This randomized phase II trial is comparing three different combination chemotherapy regimens to see how well they work in treating patients with relapsed or refractory acute myeloid leukemia. Drugs used in chemotherapy work in different ways to stop the growth of cancer cells, either by killing the cells or by stopping them from dividing. Giving more than one drug (combination chemotherapy) may kill more cancer cells. It is not yet known which combination chemotherapy regimen is more effective in treating patients with relapsed or refractory acute myeloid leukemia.

PRIMARY OBJECTIVES:

I. To determine the complete remission (CR) + cytogenic complete remission (CRc) + morphologic complete remission with incomplete blood count recovery (CRi) rate of carboplatin and topotecan; flavopiridol, mitoxantrone and cytosine arabinoside and sirolimus, mitoxantrone, etoposide and cytosine arabinoside in adult patients with refractory or relapsed acute myeloid leukemia (AML).

NOTE: Since the CRc patients are required to be either CR or CRi, we only report the rate of CR+CRi in the results section.

II. To determine the rate of treatment failure of these regimens. III. To determine the incidence and severity of toxicities of these regimens. IV. To analyze the predictive value of blast cell properties that have been suggested to determine response. (Correlative laboratory studies) V. To determine whether pretreatment levels of B-cell chronic lymphocytic leukemia (CLL)/lymphoma 2 (Bcl-2) or, alternatively, whether a therapy-induced change in topoisomerase I levels correlates with response to this regimen. (Correlative laboratory studies) VI. To assess the impact of clonal evolution by comparing cytogenetic abnormalities at the time of relapse with those at initial diagnosis and correlating these abnormalities and changes with response to the treatment regimens in this protocol. (Cytogenetic and fluorescent in situ hybridization [FISH] studies) VII. Panel FISH studies for common AML rearrangements will be performed on relapse AML specimens to determine the presence of these recurrent AML abnormalities and to evaluate for subtle additional abnormalities consistent with clonal evolution in these relapse specimens. (Cytogenetic and FISH studies)

OUTLINE: Patients are randomized to 1 of 3 treatment arms.

INDUCTION THERAPY:

ARM A: Patients receive carboplatin and topotecan hydrochloride intravenously (IV) continuously over 24 hours on days 1-5.

ARM B: Patients receive alvocidib IV over 4.5 hours once daily (QD) on days 1-3, cytarabine IV continuously over 72 hours on days 6-8, and mitoxantrone hydrochloride IV over 1-2 hours on day 9.

ARM C: Patients receive sirolimus orally (PO) QD on days 2-9, mitoxantrone hydrochloride IV over 15 minutes QD, etoposide IV over 1 hour QD, and cytarabine IV over 3 hours QD on days 4-8 or 5-9. (Closed to accrual)

After completion of induction therapy, patients in all arms undergo bone marrow aspirate and biopsy. Patients with persistent leukemia (i.e., leukemic blasts >= 10%) are removed from study and are offered alternative therapy at the discretion of the investigator. Patients who achieve CR proceed to consolidation therapy or receive alternative therapy at the discretion of the investigator.

CONSOLIDATION THERAPY: Beginning within 2-6 weeks after documentation of CR, patients may receive up to 2 additional courses of the same treatment they received during induction therapy. Courses repeat every 4-10 weeks in the absence of disease progression or unacceptable toxicity.

After completion of study therapy, patients are followed periodically for up to 3 years.

Inclusion Criteria

Induction Therapy:

• Patients must have morphologic proof (from bone marrow aspirate, smears or touch preps of marrow biopsy) of acute myelogenous leukemia (AML) with >= 10% blasts within two weeks prior to induction randomization

  • NOTE: Patients must be registered to E3903 (Ancillary Laboratory Protocol for Collecting Diagnostic Material on Patients Considered for Eastern Cooperative Oncology Group [ECOG]-American College of Radiology Imaging Network [ACRIN] Treatment Trials for Leukemia or Related Hematologic Disorders) and must undergo eligibility testing for the study by multiparameter flow cytometry
  • All immunodiagnoses are eligible for E1906, except acute promyelocytic leukemia (APL) (proven by the presence of promyelocytic leukemia (PML)/retinoic acid receptor (RAR) alpha); cases of APL can become eligible if the patient is ineligible for an ECOG-ACRIN APL trial or if all-trans retinoic acid or arsenic trioxide is not planned as part of the treatment regimen

• Patients must qualify for one of the following:

  • Relapse =< 6 months after first CR, dated from documentation of CR to documentation of relapse
  • Relapse between 6-12 months after first CR

• Refractory to conventional initial induction chemotherapy (=< 2 courses) or to first reinduction (=< 1 course)

• Normal cardiac ejection fraction by pretreatment multi gated acquisition scan (MUGA) or echocardiogram within 4 weeks prior to randomization (resting ejection fraction >= 50% or >= 5% increase with exercise), shortening fraction by echocardiogram >= 24%, or to within the normal range of values for the institution

• Prior treatment to doses of any of the following:

  • < 300 mg/m^2 of doxorubicin
  • < 300 mg/m^2 of daunorubicin
  • < 100 mg/m^2 of idarubicin
  • < 100 mg/m^2 of mitoxantrone

• Serum creatinine =< 2.0 mg/dL

• Serum direct bilirubin < 2.0 mg/dL

• Serum glutamic oxaloacetic transaminase (SGOT) (aspartate aminotransferase [AST]) < 4 x upper limit of normal

  • The above stipulation for normal hepatic function does not apply if liver dysfunction is due to leukemia infiltration

• Prior to study entry, patients must have recovered from toxicities of prior chemotherapy and radiotherapy; for patients refractory to induction chemotherapy (patient subgroup outlined above), marrow documentation of residual leukemia post chemotherapy and qualification for remaining eligibility criteria are needed prior to study entry (this does not require >= 30% marrow blasts to be evident but a minimum of 10% blasts must be present in the marrow)

  • NOTE: Hydroxyurea is permitted within 4 weeks of study entry

• ECOG performance status 0, 1 or 2

• Patients with a history of central nervous system (CNS) leukemia are eligible if there is documentation of no current CNS involvement on cerebrospinal fluid (CSF) examination, (i.e., negative CSF by lumbar puncture)

Consolidation therapy:

• Patients must have an ECOG performance status 0, 1 or 2
• Patients must have documented CR
• Patients must have an absence of infection or have infection controlled by antibiotics; patients who are septic will be excluded
• Patients must have a serum creatinine clearance > 50 cc/minute
• Patients must have a serum direct bilirubin < 2.0 mg/dl and alkaline phosphatase and SGOT (AST) < 4 x upper limits of normal
• Patients must have a normal cardiac ejection fraction by MUGA or echocardiogram prior to consolidation (resting ejection fraction >= 50% or >= 5% increase with exercise), shortening fraction by echocardiogram >= 24%, or to within normal range of values for the institution prior to the first and second cycle of consolidation for arms B and C

Exclusion Criteria

Induction therapy:

• Patients who have relapsed > 1 year after achieving first CR or are in >= second relapse
• Patients who have had a prior allogeneic OR autologous stem cell transplant
• History of recent myocardial infarction (within three months), uncontrolled congestive heart failure, or uncontrolled cardiac arrhythmia
• Prior treatment with carboplatin, topotecan, flavopiridol, or sirolimus
• Pregnant or breast feeding. Women of childbearing potential and sexually active males should use an accepted and effective method of contraception
• Intercurrent organ damage or medical problems that would prohibit therapy; no active or unresolved infection
• Current evidence of invasive fungal infection; such evidence includes positive blood or deep tissue cultures or stains
• Have another (i.e., prior) tumor which is currently active and likely to interfere with the patient's treatment for AML or which is likely to compromise the patient's morbidity or mortality substantially

Consolidation therapy:

• Intercurrent organ damage or medical problems that will jeopardize the outcome of therapy
• For arms B and C, patients have exceeded the following anthracycline doses or their equivalents:
• < 300 mg/m^2 of doxorubicin
• < 300 mg/m^2 of daunorubicin
• < 100 mg/m^2 of idarubicin
• < 100 mg/m^2 of mitoxantrone