This phase I trial studies the side effects and best dose of alisertib when combined with fractionated stereotactic radiosurgery in treating patients with high-grade gliomas that have returned after previous treatment with radiation therapy (recurrent). Alisertib may stop the growth of tumor cells by blocking an enzyme needed for the cells to divide. Radiation therapy uses high energy x rays to kill tumor cells. Stereotactic radiosurgery uses special positioning equipment to send a single high dose of radiation directly to the tumor and cause less damage to normal tissue. Delivering stereotactic radiosurgery over multiple doses (fractionation) may cause more damage to tumor tissue than normal tissue while maintaining the advantage of its accuracy.

PRIMARY OBJECTIVES:

I. Evaluate the safety and tolerability of the study treatment. II. To determine the maximum tolerated dose (MTD) of daily oral alisertib when combined with fractionated stereotactic radiation treatment for recurrent high grade glioma.

SECONDARY OBJECTIVES:

I. To estimate 6 month progression free survival rate. II. To estimate the median time to progression. III. To estimate overall survival. IV. To estimate the impact on quality of life (QoL).

OUTLINE: This is a dose-escalation study of alisertib.

CONCURRENT PHASE: Patients undergo fractionated stereotactic radiosurgery once daily (QD) every weekday for 10 days and receive alisertib orally (PO) twice daily (BID) concurrently with radiation therapy for 10 days.

MAINTENANCE PHASE: Patients receive alisertib PO BID on days 1-7. Treatment repeats every 21 days for up to 18 courses in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up every 3 months for 1 year and then every 4-6 months thereafter.

Inclusion Criteria:

1. Patients must have a previously histologically or cytologically confirmed high grade glioma (astrocytic or oligodendroglial supratentorial tumors grade 3 or 4) that has been previously treated with fractionated radiation therapy and now shows evidence of recurrence.
2. Patients must have recovered from the toxic effects of prior therapy.
3. Patients must have recovered from the effects of surgery. There must be a minimum of 21 days from the day of surgery to the day of protocol treatment. For core or needle biopsy, a minimum of 7 days must have elapsed prior to the day of protocol treatment.
4. Prior treatment with cytotoxic and biological agents is permissible. There should be at least a 2-week break between prior treatment and the protocol treatment.
5. Prior treatment with fractionated radiation therapy (up to 60Gy) is an eligibility criterion, however there should not have been a second course of fractionated radiotherapy to the supratentorial area.
6. One prior single fraction radiosurgical procedure within the treatment field is acceptable if V12<5 cc (V12 is the volume of normal brain (outside GTV) receiving 12 or more Gy). Additional radiosurgical procedures outside of the treatment area are acceptable.
7. Subject must be able to take oral medication and to maintain a fast, is required for 2 hours before and 1 hour after MLN8237 administration.
8. ANC > 1500/mm³, platelets > 100,000/mm³, Hgb > 9 g/dL. Values must be obtained without need for myeloid growth factor or platelet transfusion support within 14 days of registration. However, erythrocyte growth factor is allowed as per published ASCO guidelines.
9. Total bilirubin ≤ ULN, SGOT (AST) and SGPT (ALT)< 1.5 x ULN, within 14 days of registration.
10. Adequate renal function as defined by: calculated creatinine clearance must be ≥40 mL/minute (Cockcroft-Gault), within 14 days of registration.
11. Age >18 years.
12. ECOG performance status <2 (see Appendix I).
13. Life expectancy of greater than 2 months.
14. Women of childbearing potential must have a negative β-HCG pregnancy test documented within 7 days prior to registration.
15. Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for 4 months after last dose.
16. Ability to understand and the willingness to sign a written informed consent document.

Exclusion Criteria:

1. Known history of uncontrolled sleep apnea syndrome and other conditions that could result in excessive daytime sleepiness such as severe chronic obstructive pulmonary disease requiring supplemental oxygen.
2. Systemic infection requiring IV antibiotic therapy within 14 days preceding the first dose of study drug, or other severe infection.
3. Myocardial infarction within 6 months prior to enrollment, New York Heart Association (NYHA) Class III or IV heart failure, uncontrolled angina, severe uncontrolled ventricular arrhythmias, or electrocardiographic evidence of acute ischemia or active conduction system abnormalities. Prior to study entry, any ECG abnormality at Screening has to be documented by the investigator as not medically relevant.
4. Female subject is pregnant or breast-feeding. Confirmation that the subject is not pregnant must be established by a negative serum β-human chorionic gonadotropin (β-hCG) pregnancy test result obtained during screening. Pregnancy testing is not required for post-menopausal or surgically sterilized women.
5. Patient has received other investigational drugs within 14 days before enrollment
6. Serious medical or psychiatric illness likely to interfere with participation in this clinical study.
7. Other severe acute or chronic medical or psychiatric condition, including uncontrolled diabetes, malabsorption, resection of the pancreas or upper small bowel, requirement for pancreatic enzymes, any condition that would modify small bowel absorption of oral medications, or laboratory abnormality that may increase the risk associated with study participation or investigational product administration or may interfere with the interpretation of study results and, in the judgment of the investigator, would make the patient inappropriate for enrollment in this study.
8. Treatment with clinically significant enzyme inducers, such as the enzyme-inducing antiepileptic drugs phenytoin, carbamazepine or phenobarbital, or rifampin, rifabutin, rifapentine or St. John's wort within 14 days prior to the first dose of MLN8237 and during the study.
9. Known history of human immunodeficiency virus (HIV) infection, hepatitis B, or hepatitis C. Testing is not required in the absence of clinical findings or suspicion.
10. Patients have a history of any other malignancy from which the patient has been disease-free for less than 2 years, with the exception of adequately treated basal or squamous cell carcinoma of skin, superficial bladder cancer or carcinoma in situ of cervix, AJCC (version 7.0) stage 0 or I breast cancer, AJCC (version 7.0) stage I, or II prostate cancer.
11. Radiation therapy to more than 25% of the bone marrow. Whole pelvic radiation is considered to be over 25%.
12. Patients who cannot swallow whole tablets (i.e. medication tablets)