This phase I trial studies the side effects and best dose of alisertib when given together with irinotecan hydrochloride in treating patients with advanced solid tumors or colorectal cancer. Irinotecan hydrochloride and alisertib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.

PRIMARY OBJECTIVES:

I. To investigate the feasibility and safety of MLN8237 (alisertib) when given in combination with irinotecan (irinotecan hydrochloride) to patients with advanced solid tumors.

SECONDARY OBJECTIVES:

I. To determine the maximum tolerated dose (MTD) of MLN8237 when given in combination with irinotecan to patients with advanced solid tumors and to recommend a phase II dose for the combination.

II. To obtain preliminary evidence of efficacy for this combination. III. To compare the pharmacokinetics of SN-38 (the active metabolite of irinotecan) with and without concurrent administration of MLN8237 in an expanded cohort of patients with advanced colorectal cancer.

OUTLINE: This is a dose-escalation study of alisertib.

Patients receive irinotecan hydrochloride intravenously (IV) over 30 minutes on days 1 and 8 and alisertib orally (PO) twice daily (BID) on days 1-3 and 8-10. Courses repeat every 21 days in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up for 30 days.

Inclusion Criteria:

• Eligibility for dose escalation cohort: Histologically or cytologically confirmed metastatic or unresectable solid tumor for which standard curative or palliative measures do not exist or are no longer effective OR solid tumor for which irinotecan monotherapy is considered standard
• Eligibility for the expansion cohort: Histologically or cytologically confirmed colon or rectal adenocarcinoma for which curative treatment does not exist; patients must have documented progression or intolerance to at least one prior regimen containing 5-fluorouracil or capecitabine and oxaliplatin
• Zubrod (Eastern Cooperative Oncology Group [ECOG]) performance status 0 - 2
• Patients may have measurable or non-measurable disease; x-rays and/or scans for disease assessment of measurable disease must have been completed within 28 days prior to registration; non-measurable disease must also be assessed within 28 days prior to registration
• Absolute neutrophil count (ANC) >= 1,500/mm3
• Platelet count >= 100,000/mm3
• Total bilirubin within institutional normal limits
• Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) =< 1.5 times institutional upper limit of normal or =< 5.0 times institutional upper limit of normal in the presence of liver metastases
• Creatinine =< 1.5 times institutional upper limit of normal OR creatinine clearance >= 60 mL/min/1.73 m^2 measured by 24-hour urine collection
• Any number of prior chemotherapy regimens is allowed
• Any prior chemotherapy, immunotherapy or targeted therapy must have been completed at least 4 weeks prior to start of this protocol and all side effects (except alopecia, lymphopenia and hyperglycemia) resolved to grade 1 or less; any prior radiation must have been completed at least 2 weeks prior to start of therapy
• Pregnant or nursing women are ineligible because of the risk to the fetus; women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation
• Ability to understand and the willingness to sign a written informed consent document
• Ability to swallow and retain oral medications
• Voluntary written informed consent before performance of any study-related procedure not part of normal medical care, with the understanding that consent may be withdrawn by the subject at any time without prejudice to future medical care
• Female subject is either post-menopausal or surgically sterilized or willing to use an acceptable method of birth control (i.e., a hormonal contraceptive, intra-uterine device, diaphragm with spermicide, condom with spermicide, or abstinence) for the duration of the study
• Male subject agrees to use an acceptable method for contraception during the entire study treatment period through 4 months after the last dose of MLN8237; male patients, even if surgically sterilized (i.e., status postvasectomy) must agree to practice effective barrier contraception during the entire study treatment period and through four months after the last dose of study drug, or completely abstain from heterosexual intercourse
• Subject must be able to take oral medication and to maintain a fast as required for 2 hours before and 1 hour after MLN8237 administration

Exclusion Criteria:

• Prior treatment with irinotecan or aurora A-targeted agents, including MLN8237
• A history of known Gilbert's syndrome or homozygous presence of the uridine diphosphate (UDP)-glucuronosyltransferase 1 family, polypeptide A1 (UGT1A1)*28 allele on pre-treatment testing
• Symptomatic or uncontrolled brain metastasis; patients with neurological symptoms must undergo a computed tomography (CT) scan/magnetic resonance imaging (MRI) of the brain to exclude brain metastasis; previously treated brain metastases will be allowed as long as the patient is neurologically stable and is off steroids and anticonvulsants at the time of registration
• Prior radiation to greater than 25% of the bone marrow or whole pelvis radiation
• Patients requiring full therapeutic anticoagulation including warfarin, heparin, low-molecular weight heparin, or direct factor Xa inhibitor are ineligible because therapy on this trial may result in frequent and recurrent thrombocytopenia; patients requiring prophylactic dose anticoagulation may be eligible after discussion with the principal investigator
• Patients with a diagnosis of active human immunodeficiency virus (HIV) infection, on anti-retroviral therapy, or with a cluster of differentiation (CD)4 count less than 200 are ineligible due to potential interactions between irinotecan and anti-retroviral medications as well as possible immunosuppressive activity of the study treatment; testing is not required in the absence of clinical findings or suspicion
• Patients with a diagnosis of chronic hepatitis B are ineligible due to the possibility of immunosuppression on study treatment
• Active clinically serious infection > Common Terminology Criteria for Adverse Events (CTCAE) grade 2 or systemic infection requiring IV antibiotic therapy within 14 days preceding the first dose of study drug
• Serious non-healing wound, ulcer, or bone fracture
• Major surgery, open biopsy or significant traumatic injury within 4 weeks of first dose of study drug
• Patients may not take known strong cytochrome P450, family 3, subfamily A, polypeptide 4 (CYP3A4) inducers such as phenytoin, carbamazepine, phenobarbital, rifampin or St. John's wort or strong CYP3A4 inhibitors such as ketoconazole, diltiazem, or verapamil
• Known or suspected allergy to irinotecan or MLN8237, or any agent given in the course of this trial
• Any condition that impairs patient's ability to swallow whole pills; patients with feeding tubes, intractable nausea or vomiting, or a malabsorption syndrome are not eligible
• Any clinically significant medical or psychiatric condition that would interfere with protocol treatment
• Prior allogeneic bone marrow or organ transplantation
• Patient has >= grade 2 peripheral neuropathy within 14 days before enrollment
• Known history of uncontrolled sleep apnea syndrome and other conditions that could result in excessive daytime sleepiness, such as severe chronic obstructive pulmonary disease; requirement for supplemental oxygen
• Requirement for constant administration of proton pump inhibitors, H2 antagonists, or pancreatic enzymes; intermittent uses of antacids or H2 antagonists are allowed
• Myocardial infarction within 6 months prior to enrollment or has New York Heart Association (NYHA) class III or IV heart failure, uncontrolled angina, severe uncontrolled ventricular arrhythmias, or electrocardiographic evidence of acute ischemia or active conduction system abnormalities; prior to study entry, any electrocardiogram (ECG) abnormality at screening has to be documented by the investigator as not medically relevant
• Female subject is pregnant or breast-feeding; confirmation that the subject is not pregnant must be established by a negative serum beta-human chorionic gonadotropin (beta-hCG) pregnancy test result obtained during screening; pregnancy testing is not required for post-menopausal or surgically sterilized women
• Patient has received other investigational drugs with 14 days before enrollment
• Other severe acute or chronic medical or psychiatric condition, including uncontrolled diabetes, malabsorption, resection of the pancreas or upper small bowel, requirement for pancreatic enzymes, any condition that would modify small bowel absorption of oral medications, or laboratory abnormality that may increase the risk associated with study participation or investigational product administration or may interfere with the interpretation of study results and, in the judgment of the investigator, would make the patient inappropriate for enrollment in this study