This phase I/II trial studies the best dose of sorafenib when given together with busulfan and fludarabine in treating patients with acute myeloid leukemia that has come back or does not respond to treatment and who are undergoing donor stem cell transplant. Sorafenib may stop the growth of cancer cells by blocking some of the enzymes needed for cell growth. Drugs used in chemotherapy, such as busulfan and fludarabine, work in different ways to stop the growth of cancer cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Giving sorafenib with busulfan and fludarabine may work better in treating patients with recurrent or refractory acute myeloid leukemia.

PRIMARY OBJECTIVES:

I. To identify the maximum tolerated dose (MTD) of sorafenib when combined with busulfan and fludarabine conditioning regimen.

II. To obtain preliminary evidence of efficacy.

SECONDARY OBJECTIVES:

I. To determine safety of this regimen as per National Cancer Institute (NCI) toxicity criteria.

II. To determine time to neutrophil and platelet engraftment. III. To determine incidence of acute and chronic graft versus host disease (GVHD).

IV. To determine relapse incidence. V. To determine non relapse mortality. VI. To determine overall survival.

TERTIARY OBJECTIVES:

I. To study chemotherapy resistance. II. To study deoxyribonucleic acid (DNA) damage. III. To study immune recovery and cytokines (both in plasma and cells).

OUTLINE: This is a phase I, dose escalation study of sorafenib, followed by a phase II study.

PRE-STEM CELL INFUSION: Patients receive sorafenib orally (PO) once daily (QD) or twice daily (BID) on days -24 to -5, busulfan intravenously (IV) over 3 hours on days -20 and -13 and -6 and -3, and fludarabine IV over 1 hour on days -6 to -3 in the absence of disease progression or unacceptable toxicity.

STEM CELL INFUSION: Patients receive allogeneic hematopoietic stem cell transplant (HSCT) IV in the absence of disease progression or unacceptable toxicity.

POST-STEM CELL INFUSION: Patients receive cyclophosphamide IV over 3 hours on days 3 and 4, tacrolimus PO BID beginning day 5 for about 50 days, filgrastim subcutaneously (SC) on day 7 and sorafenib PO BID beginning between days +30 and +120 for up to 1 year in the absence of disease progression or unacceptable toxicity. Patients with matched unrelated donor receive mycophenolate mofetil PO thrice daily (TID) or IV over 2 hours TID beginning on day 5 for up to 90 days for longer.

Inclusion Criteria:

• Age >= 18 and =< 70 years
• Patients with acute myeloid leukemia both flt3 positive and negative
• Human leukocyte antigen (HLA)-identical sibling or 8/8 matched unrelated donor available
• Life expectancy of at least 12 weeks (3 months)
• Direct bilirubin =< 1 mg/dL
• Alanine transaminase (ALT) =< 3 x upper limit of normal
• Serum creatinine =< 1.5 x the upper limit of normal
• Creatinine clearance >= 50
• Diffusing capacity for carbon monoxide (DLCO) > 50% of predicted corrected for hemoglobin
• Left ventricular ejection fraction (LVEF) >= 50%
• Subjects must be able to understand and be willing to sign the written informed consent form. A signed informed consent form must be appropriately obtained prior to the conduct of any trial-specific procedure
• Women of childbearing potential must have a negative serum pregnancy test performed within 7 days prior to the start of study drug. Post-menopausal women (defined as no menses for at least 1 year) and surgically sterilized women are not required to undergo a pregnancy test
• Subjects (men and women) of childbearing potential must agree to use adequate contraception beginning at the signing of the informed consent form (ICF) until at least 30 days after the last dose of study drug. The definition of adequate contraception will be based on the judgment of the principal investigator or a designated associate
• Subject must be able to swallow and retain oral medication

Exclusion Criteria:

• Acute myeloid leukemia in first complete molecular remission and favorable risk disease as defined by presence of t(8:21) or inv (16)

• Patients with a comorbidity score > 3. The principal investigator is the final arbiter of eligibility for comorbidity score > 3

• Uncontrolled hypertension (systolic pressure > 140 mm Hg or diastolic pressure > 90 mm Hg [NCI-Common Terminology Criteria for Adverse Events (CTCAE) version (v)4.0] on repeated measurement) despite optimal medical management

• Active or clinically significant cardiac disease including:

  • Congestive heart failure - New York Heart Association (NYHA) > class II
  • Active coronary artery disease
  • Cardiac arrhythmias requiring anti-arrhythmic therapy other than beta blockers or digoxin
  • Unstable angina (anginal symptoms at rest), new-onset angina within 3 months before randomization, or myocardial infarction within 6 months before randomization

• Evidence or history of bleeding diathesis or coagulopathy. Patients with bleeding due to prior thrombocytopenia are permitted

• Subject with any pulmonary hemorrhage/bleeding event of NCI-CTCAE v. 4.0 grade 2 or higher within 4 weeks before randomization; any other hemorrhage/bleeding event of NCI-CTCAE v. 4.0 grade 3 or higher within 4 weeks before randomization

• Subjects with thrombotic, embolic, venous, or arterial cerebrovascular event (including transient ischemic attacks) within 6 months of informed consent

• Subjects who have used strong CYP3A4 inducers (e.g., phenytoin, carbamazepine, phenobarbital, St. John's wort [Hypericum perforatum], dexamethasone at a dose of greater than 16 mg daily, or rifampin [rifampicin], and/or rifabutin) within 28 days before randomization

• Subjects with any previously untreated or concurrent cancer except cervical cancer in-situ, treated basal cell carcinoma, or superficial bladder tumor. Subjects surviving a cancer that was curatively treated and without evidence of disease for more than 3 years before randomization are allowed. All cancer treatments must be completed at least 3 years prior to study entry (i.e., signature date of the informed consent form)

• Presence of a non-healing wound, non-healing ulcer, or bone fracture

• History of organ allograft (including corneal transplant)

• Known or suspected allergy or hypersensitivity to any of the study drugs, study drug classes, or excipients of the formulations given during the course of this trial

• Any malabsorption condition

• Women who are pregnant or breast-feeding

• Inability to comply with the protocol and/or not willing or not available for follow-up assessments

• Any medical, psychological, or psychosocial condition which, in the investigator's opinion, makes the subject unsuitable for trial participation

• Major surgery within 30 days prior to start of study drug

• Patients who received inotuzumab and/or gemtuzumab in the past

• Therapeutic anticoagulation with vitamin-K antagonists (e.g., warfarin) or with heparins and heparinoids

  • However, prophylactic anticoagulation as described below is allowed:

    • Low dose warfarin (1 mg orally, once daily) with prothrombin time international normalized ratio (PT-INR). =< 1.5 x upper limit of normal (ULN) is permitted. Infrequent bleeding or elevations in PT-INR have been reported in some subjects taking warfarin while on sorafenib or capecitabine therapy. Therefore, subjects taking concomitant warfarin should be monitored regularly for changes in PT, PT-INR or clinical bleeding episodes
    • Low dose aspirin (=< 100 mg daily)
    • Prophylactic doses of heparin or low molecular weight heparin