Phase I portion completed. Phase II will open pending amendment approval.

This phase I/II trial studies the side effects and the best dose of MLN8237 when given together with erlotinib hydrochloride and to see how well it works in treating patients with recurrent locally advanced or metastatic non-small cell lung cancer (NSCLC). MLN8237 and erlotinib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.

PRIMARY OBJECTIVES:

I. Safety and tolerability of the combination treatment. (Phase I) II. Maximum tolerated dose (MTD) of MLN8237 (alisertib) when given in combination with standard dose erlotinib (erlotinib hydrochloride). (Phase I) III. Progression free survival. (Phase II)

SECONDARY OBJECTIVES:

I. Pharmacokinetic (PK) parameters of erlotinib and MLN8237, including, but not limited to maximum concentration of drug (Cmax), time of occurrence for maximum drug concentration (Tmax), and area under the curve from time zero to the last measurable concentration(AUC0-t last).

II. Overall response rate (ORR), duration of response (DOR), time to progression (TTP), and overall survival (OS). (Phase II) III. Adverse events (AEs), serious adverse events (SAEs), assessments of clinical laboratory values. (Phase II)

OUTLINE: This is a phase I, dose-escalation study of alisertib followed by a phase II study.

Patients receive alisertib orally (PO) twice daily (BID) on days 1-7 and erlotinib hydrochloride PO once daily (QD) on days 1-21. Courses repeat every 21 days for up to 2 years in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up every 3 months for 2 years, every 6 months for 2 years, and then annually thereafter.

Inclusion Criteria:

• Patients with cytologically or histologically confirmed recurrent locally advanced or metastatic NSCLC have received at least one prior recognized systemic therapy for therapy for advanced disease, (recognized therapy must include a platinum doublet unless contraindicated due to organ dysfunction)
• Eastern Cooperative Oncology Group (ECOG) Performance Status 0-1
• Measurable disease by Response Evaluation Criteria in Solid Tumors- (RECIST) 1.1 criteria; previous irradiated tumor is acceptable if there is at least a 20% increase in the size of the previously irradiated lesion
• Absolute neutrophil count (ANC) >= 1500/mm^3
• Platelets >= 100,000/mm^3
• Hemoglobin (Hgb) >= 9g/dL
• Total bilirubin =< upper limit of normal (ULN)
• Serum glutamic oxaloacetic transaminase (SGOT) (aspartate aminotransferase [AST]) and serum glutamic pyruvate transaminase (SGPT) (alanine aminotransferase [ALT]) =< 1.5 x ULN; AST and/or ALT may be up to 5 X ULN if with known liver mets
• Adequate renal function as defined by: calculated creatinine clearance must be at least 40 mL/minute (Cockcroft-Gault)
• Serious, active infections must be controlled; patients may be enrolled while still on antibiotics as long as clinical signs of active infection are absent
• Previous radiation allowed provided the patient has recovered from the acute and chronic side effects to =< grade 1 (National Cancer Institute [NCI] Common Terminology Criteria for Adverse Events v. 4.0 [CTCAE v 4.0])
• Availability of archival diagnostic tissue (paraffin tissue block of resected tumor, core biopsy, fine needle aspirate cell block, or if block cannot be submitted 20-25 [5 micron] unstained slides cut from a block representative of tumor, is required)
• Able and willing to sign an informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization
• Able and willing to swallow and absorb orally administered medications and maintain a fast as required before and after MLN8237 administration
• Women of childbearing potential (WOCBP) and men who are sexually active, even if surgically sterilized, with WOCBP must agree to use effective methods of contraception during active treatment, for the duration of the study, and for 3 months after the completion of the study

Exclusion Criteria

• Prior treatment with an investigational or marketed inhibitor of the epidermal growth factor receptor (EGFR) pathway or an Aurora Kinase inhibitor
• Patients with mutations in the EGFR gene; the mutational status of all patients will be determined prior to study entry
• Prior malignancy within the past 3 years other than complete resection of basal or squamous cell carcinoma of the skin, any in situ malignancy, or low-risk prostate cancer after curative therapy
• Prior systemic therapy within 14 days of initiating protocol treatment
• Radiation therapy to more than 25% of the bone marrow; whole pelvic radiation is considered to be over 25%; (ongoing small field radiation therapy for palliation only is allowed)
• Treatment with clinically significant enzyme inducers, such as the enzyme-inducing antiepileptic drugs phenytoin, carbamazepine or phenobarbital, or rifampin, rifabutin, rifapentine or St. John's wort within 14 days prior to the first dose of MLN8237 and during the study; anticonvulsants at stable doses are allowed
• Treatment with Proton Pump Inhibitor (PPI); patients on PPI therapy prior to enrollment must stop using the PPI for at least 4 days prior to the first dose of MLN8237
• Known central nervous system (CNS) disease, except for treated brain metastasis; treated brain metastases are defined as having no evidence of progression or hemorrhage after treatment and no ongoing requirement for dexamethasone, as ascertained by clinical examination and brain imaging (magnetic resonance imaging [MRI] or computed tomography [CT]) during the screening period; anticonvulsants (stable doses) are allowed; treatment for brain metastases may include whole brain radiotherapy (WBRT), radiosurgery (RS; Gamma Knife, linear accelerator [LINAC], or equivalent) or a combination as deemed appropriate by the treating physician; patients with CNS metastases treated by neurosurgical resection or brain biopsy performed < 4 weeks prior to Day 1 will be excluded
• Uncontrolled or unstable medical or psychiatric co-morbidities which would clearly preclude use of MLN8237 or erlotinib
• Current, recent (within 2 weeks of enrollment of this study), or planned participation in an experimental drug study
• Unstable angina
• New York Heart Association (NYHA) Grade III or greater congestive heart failure
• History of myocardial infarction within 6 months of enrollment
• Abnormal electrocardiogram (EKG): severe uncontrolled ventricular arrhythmias, or evidence of acute ischemia or active conduction system abnormalities; prior to study entry, any EKG abnormality at screening has to be documented by the investigator as not medically relevant
• Pregnant (positive serum pregnancy test) or breast feeding
• History of any disease that could lead to impaired absorption of drugs
• Known hypersensitivity to any component of the trial agents
• Inability to comply with study and/or follow-up procedures
• Prior allogeneic bone marrow or organ transplantation
• Patient has >= grade 2 (CTCAE v 4.0) peripheral neuropathy within 14 days before enrollment
• Known history of uncontrolled sleep apnea syndrome and other conditions according to enrolling investigator that could result in excessive daytime sleepiness, such as severe chronic obstructive pulmonary disease; requirement for supplemental oxygen therapy; or requirement of recurrent thoracentesis to manage pleural effusions
• Known history of human immunodeficiency virus (HIV) infection, hepatitis B, or hepatitis C; testing is not required in the absence of clinical findings or suspicion