The purpose of this study is to test the safety and determine the best dose of venetoclax and cytarabine when given with or without idarubicin in treating pediatric patients with acute myeloid leukemia (AML) that did not respond to treatment (refractory) or has come back after treatment (relapsed).

PRIMARY OBJECTIVE: Determine a tolerable combination of venetoclax plus chemotherapy in pediatric patients with relapsed or refractory AML or acute leukemia of ambiguous lineage. The primary endpoints are the recommended phase 2 doses (RP2D) of venetoclax plus cytarabine and venetoclax plus cytarabine and idarubicin.

SECONDARY OBJECTIVE: Estimate the overall response rate to the combination of venetoclax and chemotherapy in pediatric patients with relapsed or refractor AML or acute leukemia of ambiguous lineage. The secondary endpoints are the rates of complete remission (CR) and complete remission with incomplete count recovery (CRi) for patients treated at the RP2D.

This study will be done in two parts:

• Part 1 - Dose Escalation: The goal of Part 1 of the study is to find the highest tolerable combination and recommended phase 2 doses (RP2D) of venetoclax plus cytarabine and venetoclax plus cytarabine and idarubicin that can be given to patients with leukemia.
• Part 2 - Dose Expansion: After determination of doses in Part 1, patients will be enrolled on Part 2 to look at the effects of venetoclax plus cytarabine and venetoclax plus cytarabine and idarubicin.

Depending on when participants enroll on the study, Part 1 participants will receive one of the following courses of therapy:

• Venetoclax daily on days 1-28; cytarabine every 12 hours on days 8-17; OR
• Venetoclax daily on days 1-28; cytarabine every 12 hours on days 8-11; OR
• Venetoclax daily on days 1-28; cytarabine every 12 hours on days 8-11; idarubicin once on day 8; OR
• Venetoclax daily on days 1-28; cytarabine every 12 hours on days 8-17; idarubicin once on day 8.

Part 2 participants will receive one of the following courses of therapy:

• Venetoclax daily on days 1-28; cytarabine - to be determined from Part 1 of the study; OR
• Venetoclax daily on days 1-28; cytarabine - to be determined from Part 1 of the study; idarubicin once on day 8.

The cytarabine dosage will be that found in Part 1 to be the highest safest dose.

Those participants receiving idarubicin will also receive dexrazoxane.

Note: Part 1 has been completed. Part 2 participants receive the following determined from Part 1 of the study:

• Venetoclax daily on days 1-28; cytarabine every 12 hours days 8-11 OR
• Venetoclax daily on days 1-28; cytarabine every 12 hours days 8-11; idarubicin once on day 8.

All participants on both Part 1 and Part 2 receive one intrathecal (IT) chemotherapy before starting the first cycle. Patients with CNS disease will receive weekly IT therapy until the cerebrospinal fluid becomes free of leukemia (minimum of 4 doses). Bone marrow aspiration and biopsy to assess response will be performed between days 28 and 42 of cycle 1. Patients who achieve complete remission/complete remission with incomplete count recovery/partial remission (CR/CRi/PR) and who do not experience unacceptable toxicity during cycle 1 may receive up to four cycles of chemotherapy.

Cohort C (Amendment 5.0): Treatment of participants enrolled in cohort C will include: Venetoclax daily on days 1-21; cytarabine every 12 hours days 8-11; azacytidine days 1-7. Participants will receive one intrathecal (IT) chemotherapy before starting the first cycle. Participants with CNS disease will receive weekly ITMHA until the cerebrospinal fluid becomes free of leukemia.

The rolling-6 design will be used to determine the safety of cohort C. After cohort C is deemed to be safe, additional patients will be enrolled, if necessary, so that at least 6 patients are treated in cohort C to confirm tolerability. After tolerability is confirmed, 6 additional patients will be treated to explore activity.

Inclusion Criteria:

• Participants must have a diagnosis of AML or acute leukemia of ambiguous lineage (acute undifferentiated leukemia or mixed phenotype acute leukemia) and have refractory leukemia, defined as persistent leukemia after at least two courses of induction chemotherapy; or relapsed leukemia, defined as the re-appearance of leukemia after the achievement of remission.

• Patients in all categories above must have ≥ 5% blasts in the bone marrow as assessed by morphology or ≥ 1 blasts in the bone marrow as assessed by flow cytometry. However, if an adequate bone marrow sample cannot be obtained, patients may be enrolled if there is unequivocal evidence of leukemia with ≥ 5% blasts in the peripheral blood. In addition, patients in all categories must not be eligible to undergo curative therapy, such as immediate SCT, because of disease burden, time needed to identify a stem cell donor, or other reasons.

  * Adequate organ function defined as the following:

• Direct bilirubin ≤ 1.5 x institutional upper limit of normal (ULN)

• AST (SGOT) and ALT (SGPT) ≤ 4 x ULN

• Normal creatinine for age or a calculated creatinine clearance ≥ 60 mL/min/1.73 m2

• Left ventricular ejection fraction ≥ 40% or shortening fraction ≥ 25%

  • St. Jude patients must be between 2 years and ≤ 21 years of age, on therapy (active patient), or within 3 years of completion of therapy. Patients treated at collaborating sites must be ≤ 24 years old.
  • Performance status: Lansky ≥ 50 for patients who are ≤ 16 years old and Karnofsky ≥ 50% for patients who are > 16 years old.
  • Patients must have fully recovered from the acute effects of all prior therapy and cannot have evidence of graft-versus-host disease (GVHD)

Exclusion Criteria:

• Must not be pregnant or breastfeeding. Male or female of reproductive potential must agree to use effective contraception for the duration of study participation.
• Patients with Down syndrome, acute promyelocytic leukemia, juvenile myelomonocytic leukemia, or bone marrow failure syndromes are not eligible.
• Uncontrolled infection. Infections controlled on concurrent anti-microbial agents are acceptable, and anti-microbial prophylaxis per institutional guidelines are acceptable.
• Impairment of GI function or GI disease that may significantly alter the absorption of venetoclax.