This phase II trial evaluates the effect of azacitidine or decitabine and venetoclax in treating patients with acute myeloid leukemia that has not been treated before (treatment naive) or has come back (relapsed). Chemotherapy drugs, such as azacitidine, decitabine, and venetoclax, work in different ways to stop the growth of cancer cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading.

PRIMARY OBJECTIVE: I. To evaluate the efficacy of venetoclax plus alternative hypomethylating agent (HMA), as defined by the primary endpoint of overall response rate, for patients with treatment naive acute myeloid leukemia (AML) eligible for venetoclax plus HMA with prior HMA failure.

SECONDARY OBJECTIVES:

I. To further examine the efficacy of venetoclax plus alternative HMA for patients with treatment naive AML eligible for venetoclax plus HMA with prior HMA failure using additional efficacy endpoints.

II. To further evaluate the safety of venetoclax plus alternative HMA for patients with treatment naive AML eligible for venetoclax plus HMA with prior HMA failure.

Inclusion Criteria:

• Ability to understand and the willingness to sign a written informed consent
• Diagnosis of AML by World Health Organization (WHO) 2016 criteria (Arber 2016)
• Age >= 18 years
• Treatment naïve and eligible for venetoclax plus HMA: * Age >= 75 OR * Age >= 18-74 with at least one of the following co-morbidities: ** Eastern Cooperative Oncology Group (ECOG) performance status of 2 or 3 ** Cardiac history of chronic heart failure (CHF) requiring treatment or left ventricular ejection fraction (LVEF) =< 50% or chronic unstable angina ** Carbon monoxide diffusing capability (DLCO) =< 65% or forced expiratory volume in 1 second (FEV1) =< 65% ** Creatinine clearance >= 30 mL/min to =< 45 mL/min ** Moderate hepatic impairment with total bilirubin > 1.5 to =< 3 x upper limit of normal (ULN) ** Any other situation that the investigator judges to be incompatible with intensive chemotherapy must be reviewed with the study chair before study enrollment
• Patient experienced HMA failure for an antecedent hematologic disorder (e.g. myelodysplastic syndrome) prior to study entry (Santini 2019), defined as: * Disease progression or stable disease as best response to >= 4 cycles of HMA or >= 2 cycles of HMA combination therapy (primary resistance) OR * Relapse or progression after prior response to HMA (secondary resistance)
• Prior decitabine and/or azacitidine, including oral formulations, for antecedent hematologic disorder is required. The patient should be treatment naïve for the AML diagnosis
• Prior allogeneic hematopoietic transplant for antecedent hematologic disorder is allowed if done at least 3 months prior to enrollment and there is no evidence of active graft versus host disease (GVHD) or requirement for systemic immune suppression
• ECOG performance status of: * 0 to 2 for subjects >= 75 years of age OR * 0 to 3 for subjects >= 18-74 years of age
• Whole blood cell (WBC) >= 25,000/mm^3 at the start of study therapy (leukapheresis and hydroxyurea areallowed to meet this criteria)
• Total bilirubin =, 1.5 x institution's ULN unless related to AML or Gilbert's syndrome (subjects who are >= 18-74 may have a total bilirubin of =< 3 x institution's ULN)
• Aspartate aminotransferase (AST)/serum glutamic oxaloacetic transaminase (SGOT) and alanine aminotransferase (ALT)/serum glutamic pyruvic transaminase (SPGT) =< 3 x institutional ULN unless related to AML
• Creatinine clearance >= 30 mL/min (calculated by the Cockcroft Gault formula or measured by 24-hours urine collection)
• Women of child-bearing potential and men with partners of child-bearing potential must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and for 90 days following completion of therapy. Should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately. * A woman of child-bearing potential is any female (regardless of sexual orientation, having undergone a tubal ligation, or remaining celibate by choice) who meets the following criteria: ** Has not undergone a hysterectomy or bilateral oophorectomy; or ** Has not been naturally postmenopausal for at least 12 consecutive months (i.e., has had menses at any time in the preceding 12 consecutive months)
• Women of child-bearing potential has negative pregnancy test prior to initiating study drug dosing
• Able to swallow and retain oral medication

Exclusion Criteria:

• Current or anticipated use of other investigational agents within 14 days or 5 half-lives (whichever is shorter) prior to the first dose and throughout venetoclax administration
• Diagnosis of acute promyelocytic leukemia
• Active central nervous system involvement by AML
• Anticancer therapies, including investigational therapy, chemotherapy, targeted small molecule agents, or radiotherapy within 14 days or 5 half-lives (whichever is shorter) prior to the first dose and throughout venetoclax administration. Biologic agents (e.g. monoclonal antibodies) given for anti-neoplastic intent within 30 days prior to the first dose and throughout venetoclax administration
• Prior therapy with venetoclax
• Known diagnosis of human immunodeficiency virus (HIV) infection or known active hepatitis A, B or C infection with the exception of those with an undetectable viral load and CD4+ T-cell (CD4+) counts >= 350 cells/μL within 3 months of starting study treatment. Should have no titers within 28d of day 1. Patients with hepatitis C virus (HCV) infection should have completed curative antiviral treatment
• Known strong and/or moderate CYP3A inducers within 7 days prior to the initiation of study treatment
• Subject has consumed grapefruit, grapefruit products, Seville oranges or starfruit within 3 days prior to the initiation of study treatment
• Severe or uncontrolled medical disorder that would, in the investigator's opinion, impair ability to receive study treatment (i.e., uncontrolled diabetes, chronic renal disease, chronic pulmonary disease or active, uncontrolled infection, psychiatric illness/social situations that would limit compliance with study requirements)
• History of other malignancies, except for malignancy treated with curative intent with no known active disease present for >= 1 year; treated non-melanoma skin cancer; and localized, cured prostate and cervical cancer
• Evidence of uncontrolled active systemic infection requiring therapy (viral, bacterial, or fungal). Fever of unknown origin is not an exclusion criterion, as this may be disease related
• History of allergic reactions attributed to compounds of similar chemical or biologic composition to agents used in study
• Subject has a malabsorption syndrome of other condition that precludes enteral route of administration
• Subjects with a cardiovascular disability status of New York Heart Association class greater than 2
• Pregnant or nursing. There is a potential for congenital abnormalities and for this regimen to harm nursing infants