RATIONALE: Drugs used in chemotherapy use different ways to stop cancer cells from dividing so they stop growing or die. Thalidomide may stop the growth of cancer cells by stopping blood flow to the tumor. Combining chemotherapy with thalidomide may kill more cancer cells.

PURPOSE: Phase II trial to study the effectiveness of combining fludarabine, carboplatin, and topotecan with thalidomide in treating patients who have relapsed or refractory acute myeloid leukemia, chronic myelogenous leukemia, or advanced myelodysplastic syndromes.

OBJECTIVES:

• Determine the response rate of patients with relapsed/refractory or high-risk acute myeloid leukemia, chronic myelogenous leukemia, or advanced myelodysplastic syndromes treated with fludarabine, carboplatin, topotecan, and thalidomide.
• Determine the non-hematologic toxicity profile and time to hematopoietic recovery in patients treated with this regimen.
• Determine the effects of this regimen on changes in biologic parameters that may predict response in these patients.
• Correlate bone marrow microvascular density before and after treatment with response in these patients.
• Determine the prognostic value of pretreatment plasma and serum levels of vascular endothelial growth factor (VEGF) and/or the modulation of serum levels of VEGF during treatment in predicting response in these patients.

OUTLINE: Patients are stratified according to diagnosis (previously untreated acute leukemia vs other).

Patients receive fludarabine IV over 5-10 minutes and carboplatin IV over 24 hours on days 1-5 followed by topotecan IV continuously over 72 hours. Patients receive oral thalidomide daily beginning within days 1-3 and continuing in the absence of disease progression or unacceptable toxicity.

Patients with residual disease on day 16-18 may receive a second course of chemotherapy as above. Patients who achieve remission may receive a third course of chemotherapy as above as consolidation beginning 4-8 weeks after completion of prior chemotherapy.

Patients are followed monthly for 6 months.

PROJECTED ACCRUAL: A total of 40 patients (20 per stratum) will be accrued for this study.

DISEASE CHARACTERISTICS:

• Diagnosis of 1 of the following:

  • Acute myeloid leukemia meeting 1 of the following criteria:

    • Previously untreated and not a candidate for anthracycline-based chemotherapy
    • In first or second relapse or refractory
    • Secondary to chemotherapy or an antecedent hematologic disorder and treated with no more than 1 prior intensive induction regimen

  • Chronic myelogenous leukemia in blast crisis at diagnosis or after prior imatinib mesylate

  • Myelodysplastic syndromes (MDS)

    • Refractory anemia with excess blasts (RAEB) or RAEB in transformation

    • Must meet at least 1 of the following criteria:

      • Absolute neutrophil count no greater than 500/mm^3
      • Platelet or red cell transfusion-dependent after no more than 1 prior intensive induction chemotherapy

  • Acute promyelocytic leukemia

    • t(15, 17)
    • Failed prior treatment with tretinoin and arsenic

  • Relapsed disease at least 3 months after prior autologous stem cell transplantation

• No active CNS leukemia

PATIENT CHARACTERISTICS:

Age

• 18 and over

Performance status

• ECOG 0-3

Life expectancy

• At least 8 weeks

Hematopoietic

• See Disease Characteristics

Hepatic

• Bilirubin no greater than 2.0 mg/dL
• AST and ALT less than 3 times upper limit of normal

Renal

• Creatinine clearance at least 50 mL/min

Cardiovascular

• Ejection fraction at least 40%
• No poorly controlled cardiac disease

Pulmonary

• No poorly controlled pulmonary disease

Other

• Not pregnant or nursing
• Negative pregnancy test
• Fertile female patients must use 1 highly effective and 1 additional method of contraception for 4 weeks before, during, and for at least 4 weeks after study
• Male patients must use effective contraception during and for 4 weeks after study
• Willing and able to comply with the System for Thalidomide Education and Prescribing Safety (STEPS) program
• HIV negative
• No poorly controlled infection
• No other active malignancy
• No severe peripheral neuropathy

PRIOR CONCURRENT THERAPY:

Biologic therapy

• See Disease Characteristics
• Prior thalidomide allowed for MDS
• At least 5 days since prior hematopoietic growth factors
• At least 2 weeks since prior biologic therapy
• No prior allogeneic bone marrow transplantation

Chemotherapy

• See Disease Characteristics
• At least 24 hours since prior hydroxyurea

Endocrine therapy

• At least 24 hours since prior corticosteroids

Radiotherapy

• Not specified

Surgery

• Not specified

Other

• At least 2 weeks since prior cytotoxic anticancer therapy
• Prior amifostine allowed for MDS