This randomized phase II trial studies how well WEE1 inhibitor AZD1775 with or without cytarabine works in treating patients with acute myeloid leukemia or myelodysplastic syndrome that has spread to other places in the body and usually cannot be cured or controlled with treatment. WEE1 inhibitor AZD1775 may stop the growth of cancer cells by blocking some of the enzymes needed for cell growth. Drugs used in chemotherapy, such as cytarabine, work in different ways to stop the growth of cancer cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. It is not yet known whether giving WEE1 inhibitor AZD1775 works better with or without cytarabine in treating patients with advanced acute myeloid leukemia or myelodysplastic syndrome.

PRIMARY OBJECTIVES:

I. To estimate the clinical efficacy of AZD1775 (WEE1 inhibitor AZD1775) in combination with AraC (cytarabine) in patients with newly diagnosed acute myeloid leukemia (AML) by assessing complete response (complete remission [CR] plus CR with incomplete blood count recovery [CRi]) rates.

II. To estimate the clinical efficacy of AZD1775 alone or in combination with AraC in patients with relapsed/refractory AML and hypomethylating agent failure myelodysplastic syndrome (MDS) by assessing complete response (CR plus CRi) rates.

SECONDARY OBJECTIVES:

I. To determine the safety and tolerability of AZD1775 alone or combined with AraC in the study population.

II. To estimate additional measures of clinical benefit (i.e. hematological improvements, transfusion requirements).

III. To measure the duration of response of AZD1775 alone or combined with AraC.

IV. To measure time to response of AZD1775 alone or combined with AraC. V. To measure time to progression of AZD1775 alone or combined with AraC. VI. To measure overall survival of AZD1775 alone or combined with AraC. VII. To measure time to AML (for MDS subjects) of AZD1775 alone or combined with AraC.

TERTIARY OBJECTIVES:

I. To determine the pharmacokinetics (PK) of AZD1775 alone or combined with AraC in the study population.

II. To conduct correlative research studies characterizing underlying molecular events and solidifying putative mechanism of action in vivo and to identify potential pharmacodynamic/biomarkers of response to AZD1775 alone or combined with AraC.

III. To evaluate quality of life (QOL) and patient-reported symptoms in subjects treated with AZD1775 alone or combined with AraC.

OUTLINE: Elderly newly diagnosed patients are assigned to arm A.

ARM A (ELDERLY NEWLY DIAGNOSED PATIENTS): Patients receive cytarabine subcutaneously (SC) twice daily (BID) on days 1-5 and 8-12 and WEE inhibitor AZD1775 orally (PO) daily on days 1-5 and 8-12.

Patients are randomized to 1 of 2 treatment arms.

ARM B: Patients receive cytarabine and WEE1 inhibitor AZD1775 as in Arm A.

ARM C: Patients receive WEE inhibitor AZD1775 PO daily on days 1-5, 8-12, 15-19, and 22-26.

In all arms, courses repeat every 28 days in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up every 3-6 months for 2 years.

Inclusion Criteria:

• Patient population (histological or cytologically confirmed diagnosis):

  • Untreated elderly (> 60 years) AML if in the intermediate and poor-risk cytogenetic group and not candidates (as judged by treating doctor of medicine [MD]) for or willing to undergo standard induction therapy (i.e. elderly unfavorable cytogenetic AML) or any untreated AML age > 65 years

    • Note: previous therapy with a hypomethylating agent (HMA) for a diagnosis of MDS is allowed

  • Relapsed or refractory AML (>= 18 years)

  • Any MDS (>= 18 years) having failed or been intolerant to prior hypomethylating agent (HMA) treatment

    • Failure is defined as any disease progression while on HMA, relapse after HMA treatment or no response after 4 cycles of 5-Azacitidine or decitabine

    • Patients with isolated 5q-/5q- syndrome must have failed, not tolerated, or lenalidomide in addition to having failed or been intolerant to HMA treatment

      • Note: patients with chronic myelomonocytic leukemia (CMML) and MDS/myeloproliferative neoplasms (MPN) overlap are allowed if meeting other study eligibility criteria
      • Note: for all patient groups, therapy as part of a plan as a bridge to transplant is allowed

• Total bilirubin =< 1.5 mg/dL (except Gilbert's syndrome or known hemolysis or leukemic infiltration)

• Aspartate aminotransferase (AST)(serum glutamic oxaloacetic transaminase [SGOT]) and alanine aminotransferase (ALT)(serum glutamate pyruvate transaminase [SGPT]) =< 2.5 x upper limit normal (ULN) or < 5 x ULN if organ involvement

• Alkaline phosphatase < 5 x ULN

• Serum creatinine =< 2 x ULN or 24 hour creatinine (Cr) clearance > 30 ml/min

• Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1 or 2

• Ability to provide informed written consent and be able to adhere to the study visit schedule and other protocol requirements

• Willing to return to enrolling institution for follow-up (during the active monitoring phase of the study)

• Willing to provide blood and bone marrow aspirate samples for correlative research purposes

• Negative serum pregnancy test done =< 7 days prior to registration, for women of childbearing potential only

• Men and women must be willing to use appropriate contraception throughout study and for 6 months after

• Male patients who are sexually active with a female partner of childbearing potential must be either surgically sterilized or agree to use barrier contraception (ie, condoms) for the duration of study participation, and for 90 days after the final dose of study drug; cessation of birth control after this point should be discussed with a responsible physician

• Patients who have undergone stem cell transplantation (SCT), autologous or allogeneic, are eligible provided that they are > 60 days from stem cell infusion, have graft-versus-host disease (GVHD) =< grade 1 and are off immunosuppressive agents for > 28 days at time of registration

Exclusion Criteria:

• Uncontrolled intercurrent illness including, but not limited to, active uncontrolled infection, known positive for active infectious hepatitis, type A, B or C (past infection allowed), or psychiatric illness/social situations that would limit compliance with study requirements; Note: ongoing infection controlled on antibiotics/antifungal/antiviral medications are allowed

• Any of the following prior therapies:

  • Cytotoxic chemotherapy =<14 days prior to registration

  • Immunotherapy =< 14 days prior to registration

  • Biologic therapy (i.e. antibody therapies) =< 14 days prior to registration

  • Radiation therapy =<14 days prior to registration

  • Targeted therapies (i.e. kinase inhibitors, =< 7 days or 5 half-life's whichever is shorter)

    • For steroids or other non-cytotoxics given for blast count control, patient must be off for > 24 hours (hrs) before starting therapy; NOTE: hydroxyurea (HU) is allowed for blast count control throughout study

  • Receiving any other investigational agent which would be considered as a treatment for the primary neoplasm =< 14 days prior to registration

• Active uncontrolled central nervous system (CNS) leukemia; NOTE: positive (cyto)pathology is allowed and patient can receive intrathecal chemotherapy

• Immunocompromised patients and patients known to be human immunodeficiency virus (HIV) positive and currently receiving antiretroviral therapy; NOTE: patients known to be HIV positive, but without clinical evidence of an immunocompromised state, are eligible for this trial

• Any previous treatment with AZD1775 or allergic reactions to excipients of AZD1775

• Acute promyelocytic leukemia (APL, M3) unless failed regimens that included tretinoin, arsenic trioxide, anthracyclines and cytarabine and currently NOT candidates for stem cell transplantation

• Major surgery =< 28 days prior to registration

• Clinically significant heart disease, including the following:

  • Active severe angina pectoris within 3 months prior to registration

  • Acute myocardial infarction within 3 months prior to registration

  • New York Heart Association classification IV cardiovascular disease or symptomatic class III disease

    • Note: patients with any of the above may be allowed after discussion amongst the investigators including the principal investigator

• Any of the following:

  • Pregnant women
  • Nursing women
  • Men or women of childbearing potential who are unwilling to employ adequate contraception

• Subject has had prescription or non-prescription drugs or other products known to be sensitive cytochrome P450 family 3, subfamily A, polypeptide 4 (CYP3A4) substrates or CYP3A4 substrates with a narrow therapeutic index, or to be moderate to strong inhibitors/inducers of CYP3A4 which cannot be discontinued two weeks prior (alternatively 5 half lives if T1/2 is known) prior to day 1 of dosing and withheld throughout the study until 2 weeks after the last dose of study drug

  • NOTE: co-administration of aprepitant or fosaprepitant during this study is prohibited
  • Note: individual drugs exerting CYP interactions may be continued on a case by case basis if felt essential for patient management, after discussions and discretion of the treating physician
  • The preferred azole anti-fungal medication is fluconazole (alternatively posaconazole) which can be given during treatment with AZD1775 at the treating physician's discretion, however with dose reductions of AZD1775 by 25-75% (i.e. from AZD1775 200mg to 150 or 100mg)

• Patients may not be on an inhibitor of breast cancer resistance protein (BCRP)

  • NOTE: AZD1775 is an inhibitor of breast cancer resistance protein (BCRP); the use of statins including atorvastatin which are substrates for BCRP are therefore prohibited and patients should be moved on to non-BCRP alternatives

• Not willing to avoid grapefruit, grapefruit juices, grapefruit hybrids, Seville oranges, pummelos, and exotic citrus fruits from 7 days prior to the dose of study medication and during the entire study; NOTE: orange juice is allowed

• Corrected QT interval (QTc) > 470 msec (as calculated per institutional standards) at study entry or congenital long QT syndrome