This phase I/II trial studies the side effects and best dose of hydroxychloroquine when given together with palbociclib and letrozole before surgery in treating patients with estrogen receptor positive, HER2 negative breast cancer. Hydroxychloroquine is a substance that decreases immune responses in the body. Palbociclib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Estrogen can cause the growth of breast cancer cells. Drugs, such as letrozole, may lessen the amount of estrogen made by the body. Giving hydroxychloroquine, palbociclib, and letrozole before surgery may work better than palbociclib and letrozole in treating patients with breast cancer.

PRIMARY OBJECTIVES:

I. To determine the safety of adding hydroxychloroquine (HCQ) to continuous low dose palbociclib and letrozole and to determine the recommended phase II dose (RP2D) for hydroxychloroquine (HCQ) for the subsequent Phase II study. (Phase I) II. To determine the dose responsiveness of 2 dose levels (400 mg and recommended phase II dose [RP2D]) of hydroxychloroquine added to low dose palbociclib and letrozole on pre and post HCQ breast tumor proliferation index (Ki67), autophagy, senescence and cell cycle control. (Phase II, Part I) III. To determine whether hydroxychloroquine added to low dose palbociclib and letrozole can increase the proportion of patients whose tumors achieve complete cell cycle arrest (CCCA, defined as the Ki67 =< 2.7%) comparing T2 to T1. (Phase II, Part II)

SECONDARY OBJECTIVES:

I. To determine the response rate and clinical benefit rate at 8 weeks of the assigned dose of hydroxychloroquine (HCQ) plus continuous low dose palbociclib and letrozole. (Phase I) II. Determine longer term clinical tumor responsiveness (tumor volume) and tumor biomarker indices (for patients who have extended pre-operative therapy, maximum 24 weeks). (Phase II, Part I) III. Perform exploratory studies on blood-based tumor protein, deoxyribonucleic acid (DNA) and ribonucleic acid (RNA) biomarkers with a focus on pathways of cell proliferation, autophagy, senescence and cell cycle control. (Phase II, Part I) IV. To determine the impact of adding hydroxychloroquine to low dose palbociclib and letrozole on breast tumor indices of proliferation, autophagy, senescence, cell cycle control and other intersecting pathways. (Phase II, Part II) V. Determine longer term clinical tumor responsiveness and tumor biomarkers indices (for patients who have extended pre-operative therapy, maximum 24 weeks). (Phase II, Part II) VI. To determine the dose responsiveness of HCQ (400 mg vs. RP2D) on the primary (proportion with CCCA) and secondary clinical/biological endpoints. (Phase II, Part II) VII. To perform exploratory studies on blood-based tumor protein, DNA and RNA biomarkers. (Phase II, Part II) VIII. Obtain additional safety information for the combination of low dose palbociclib, letrozole and hydroxychloroquine. (Phase II, Part II)

OUTLINE: This is a phase I, dose-escalation study of hydroxychloroquine followed by a phase II study.

PHASE I: Patients with advanced, metastatic (stage IV) breast cancer receive hydroxychloroquine orally (PO) once daily (QD), palbociclib PO QD, and letrozole PO QD on days 1-28. Cycles repeat every 28 days for up to 1 year in the absence of disease progression or unacceptable toxicity.

PHASE II: Patients with early stage (stage I-III) breast cancer receive hydroxychloroquine PO QD on days 15-28 of cycle 1 and on days 1-28 of subsequent cycles. Patients also receive palbociclib PO QD, and letrozole PO QD on days 1-28, followed by standard of care surgery at week 5. If there is a proliferative benefit with complete cell cycle arrest (CCCA) by biopsy at 4 weeks, cycles may repeat every 28 days for up to 20-24 weeks in the absence of disease progression or unacceptable toxicity, followed by standard of care surgery during weeks 20-24.

After completion of study treatment, patients are followed up within 30 days or every 4 weeks.

Inclusion Criteria:

• Signed written informed consent
• Diagnosis of estrogen positive breast cancer, estrogen receptor-positive and HER2-negative by American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) criteria
• Eastern Cooperative Oncology Group (ECOG) performance status 0-1
• Postmenopausal defined by: a. Age >= 55 years and 1 year or more of amenorrhea b. Age < 55 years and 1 year or more of amenorrhea with luteinizing hormone (LH) and/or follicle stimulating hormone (FSH) levels in the postmenopausal range c. Age < 55 with prior hysterectomy but intact ovaries with LH and/or FSH levels in the postmenopausal range d. Chemotherapy or medically induced ovarian suppression with 1 year or more of amenorrhea and with LH and/or FSH levels in the postmenopausal range e. Status after bilateral oophorectomy (>= 28 days prior to first study treatment)
• Absolute neutrophil count (ANC) >= 1500 cells/ul
• Platelet count >= 100,000/ul
• Serum creatinine concentration < 1.5 x upper limit of normal (ULN)
• Bilirubin level < 1.5 x ULN
• Aspartate aminotransferase (AST) or alanine aminotransferase (ALT) < 3 x ULN
• Alkaline phosphatase =< 2.5 ULN
• Metastatic cohorts (Phase I): Diagnosis of stage IV estrogen positive breast cancer, estrogen receptor-positive and HER2-negative by ASCO/CAP criteria
• Metastatic cohorts (Phase I): Must be a candidate for treatment with CDK4/6 inhibitor and hormonal therapy with an aromatase inhibitor as standard of care
• Metastatic cohorts (Phase I): No prior exposure to CDK 4/6 inhibitors
• Neoadjuvant cohorts (Phase II): Diagnosis of stage I-III estrogen positive breast cancer, estrogen receptor-positive and HER2-negative by ASCO/CAP criteria. If stage I, clinical tumor size must be >= 1.5 cm
• Neoadjuvant cohorts (Phase II): Baseline tumor Ki67 > 5%
• Neoadjuvant cohorts (Phase II): Surgical candidate and appropriate for pre-operative endocrine therapy

Exclusion Criteria:

• Prior exposure to CDK 4/6 inhibitor therapy
• History of retinal disease or active visual disturbances (normal baseline study-specified retinal exam required)
• Acute illness, including infections requiring medical therapy, known bleeding diathesis or need for anticoagulation
• Treatment with any of the following medications within 4 weeks before the baseline diagnostic biopsy is taken: a. Oral estrogens, including hormone replacement therapy (but prior depot estrogen use not allowed). b. Investigational agents (or 5 half-lives, whichever is longer)
• Required concomitant use of any drug that is a strong CYP3A inhibitor or inducer
• Psychological, familial, sociological or geographical conditions that do not permit compliance with the study protocol
• Life expectancy of less than 6 months
• Pregnancy, lactation or planning to be pregnant.
• Neo-adjuvant cohorts (Phase II): Prior therapy for breast cancer (medical, surgical or radiation therapy)
• Neo-adjuvant cohorts (Phase II): Clinical T4 disease
• Neo-adjuvant cohorts (Phase II): Inoperable or metastatic breast cancer based on standard evaluation