This phase I trial studies the side effects and best dose of selinexor when given after stem cell transplant in treating patients with acute myeloid leukemia that is at intermediate or high risk of spreading or coming back (intermediate- or high-risk), or myelodysplastic syndrome that is at high risk of spreading or coming back (high-risk). Selinexor works to stop cancer growth by blocking an enzyme, which may cause cancer cells to die and also kill cells that cause the cancer to grow, which commonly do not respond to regular chemotherapy.

PRIMARY OBJECTIVES:

I. To determine the maximum tolerated dose (MTD) of selinexor in patients with hematologic malignancies, especially acute myeloid leukemia (AML)/myelodysplastic syndrome (MDS), after allogeneic (allo)-stem cell transplant (SCT).

SECONDARY OBJECTIVES:

I. To evaluate the toxicities of selinexor as maintenance treatment after allo-SCT.

II. To determine the incidence of non-relapse mortality. III. To determine 2 years post SCT progression-free survival (PFS) and overall survival rates.

IV. To determine the incidence of acute and chronic graft-versus-host disease (GVHD).

V. To assess lymphoid and myeloid chimerism post transplantation.

TERTIARY OBJECTIVES:

I. To analyze donor immune re-constitution after allo-SCT with selinexor maintenance.

II. To monitor minimal residual disease (MRD) by Wilms tumor 1 (WT1) polymerase chain reaction (PCR) during selinexor treatment in AML/MDS patients.

III. To characterize the physiopathology of the leukemia initiating cells (LIC) at the time of disease relapse on selinexor maintenance and compare that at initial diagnosis of the disease.

OUTLINE: This is a dose-escalation study.

Beginning on day 60-100 after allo-SCT without evidence of GVHD above grade 1 and disease relapse with stable hematopoietic recovery, patients receive selinexor orally (PO) on day 1 of each week or on days 1 and 3 of weeks 1-3. Treatment repeats every 28 days for up to 12 courses in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up for 1 year.

Inclusion Criteria:

• Signed, written informed consent in accordance with federal, local, and institutional guidelines
• Patients underwent allo-SCT with intermediate risk and high risk AML and high risk MDS (defined by American Society for Blood and Marrow Transplantation [ASBMT] criteria), who are within 60 to 100 days after allo-SCT
• There is no evidence of disease relapse at the time of screening, and minimal residual disease (MRD) is acceptable
• Eastern Cooperative Oncology Group (ECOG) performance status of =< 2
• Absolute neutrophil count (ANC) > 1000 uL
• Platelets >= 20,000 without platelet transfusion
• Creatinine clearance > 30 cc/min calculated using the Cockcroft and Gault (1976) formula or measured
• Total bilirubin =< 2 mg/dl unless high indirect bilirubin is due to a congenital disorder
• Transaminases (aspartate aminotransferase [AST] or alanine aminotransferase [ALT]) =< 2.0 x upper limit of normal (ULN)
• Prothrombin time (PT) and partial thromboplastin time (PTT) =< 2 x ULN
• Willingness and ability to comply with scheduled visits, treatment plans, laboratory tests and other study procedures
• It is important patients understand the need to use birth control while on this study; female patients of child-bearing potential must agree to use dual methods of contraception and have a negative serum pregnancy test at screening (< 3 days prior to first dose), male patients with partners of childbearing potential must agree to use effective contraception during the study period and a period of 3 months after the last dose of study drug; for both male and female patients, effective methods of contraception must be used throughout the study and for three months following the last dose

Exclusion Criteria:

• Patients with acute GVHD grade II-IV

• Treatment with any investigational agent within three weeks prior to first dose in this study

• Major surgery within 2 weeks of first dose of study drug; patients must have recovered from the effects of any surgery performed greater than 2 weeks previously

• Patient has a concurrent active malignancy under treatment

• Unstable cardiovascular function:

  • Symptomatic ischemia, or
  • Uncontrolled clinically significant conduction abnormalities (i.e., ventricular tachycardia on antiarrhythmic agents are excluded; 1st degree atrioventricular (AV) block or asymptomatic left anterior fascicular block/right bundle branch block (LAFB/RBBB) will not be excluded), or
  • Congestive heart failure (CHF) NYHA class >= 3, or
  • Myocardial infarction (MI) within 3 months

• Uncontrolled infection requiring parenteral antibiotics, antivirals, or antifungals within one week prior to first dose; infections controlled on concurrent anti-microbial agents are acceptable, and anti-microbial prophylaxis per institutional guidelines are acceptable

• Known active hepatitis B virus (HBV) or C virus (HCV) infection; or known to be positive for HCV ribonucleic acid (RNA) or HBsAg (HBV surface antigen)

• Known human immunodeficiency virus (HIV) infection

• Any medical condition which, in the investigator's opinion, could compromise the patient's safety

• Patients unable to swallow tablets or patients with malabsorption syndrome, or any other disease significantly affecting gastrointestinal function