This study will test the safety and effectiveness of a combination of pembrolizumab, olaparib, and temozolomide to see how well these drugs work when given together in people with a glioma that either did not respond to previous treatment or came back after treatment.

Inclusion Criteria:

Safety Lead-In and Cohort A specific inclusion:

1. Histologically confirmed grade II or III IDH-mutated glioma (absence of known CDKN2A/B deletion) that has recurred after first line therapy (consisting of at least maximum feasible surgical resection). There is no limit on the number of prior therapies or types of therapies patients can have received.
2. Measurable disease by RANO criteria
3. Stable dose of corticosteroids for ≥ 4 weeks prior to baseline MRI. Steroid dose not to exceed 2mg/day dexamethasone (or equivalent).

Cohort B specific inclusion:

1. Histologically confirmed IDH-wildtype glioma that has recurred following therapy (consisting of at least maximum feasible surgical resection and radiation therapy).
2. Standard of care next generation sequencing via a CLIA certified platform must be available or planned and at a minimum include IDH status.
3. Patient with known or suspected deleterious mutations in at least 1 of the specified 15 genes involved in homologous recombination repair (BRCA1, BRCA2, ATM, BARD1, BRIP1, CDK12, CHEK1, CHEK2, FANCL, PALB2, PPP2R2A, RAD51B, RAD51C, RAD51D, and RAD54L)
4. Measurable disease by RANO criteria
5. Stable dose of corticosteroids for ≥ 4 weeks prior to baseline MRI. Steroid dose not to exceed 2mg/day dexamethasone (or equivalent).

All Cohorts:

1. Patients or their Legally Authorized Representative (LAR) must provide written informed consent prior to any screening procedures

2. Age 18 or older

3. ECOG 0 or 1 (KPS ≥ 70) (A lower KPS may be acceptable with prior approval from PI)

4. Willing and able to comply with scheduled visits, treatment plan, and laboratory tests

5. Patient must be able to swallow and retain oral medication

6. Patient must have adequate organ function as defined in the following table. Stable dose of corticosteroids for ≥ 5 days prior to baseline MRI.

7. Before starting study treatment, patients must have recovered to grade 1 from prior therapy (except for residual alopecia or grade 2 peripheral neuropathy).

8. At least 5 half-lives must have elapsed since any prior signaling pathway modulators (e.g., EGFR, FGFR, or other tyrosine kinase inhibitors), at least 3 weeks must have elapsed since temozolomide, 4 weeks must have elapsed since carboplatin or cisplatin, and at least 6 weeks must have elapsed from nitrosoureas (e.g., BCNU, CCNU). At least 5 half-lives much have elapsed since prior IDH-inhibitor use. In general, at least 4 weeks must have elapsed from any other anticancer drug therapy (e.g. bevacizumab).

9. Patients must be able to undergo contrast-enhanced MRI scans.

10. Patients must have shown unequivocal evidence for tumor progression by MRI in comparison to a prior scan

11. At least 12 weeks elapsed since prior radiotherapy

12. Life expectancy greater than 12 weeks

13. A woman of childbearing potential (WOCBP) must not have a positive urine pregnancy test within 72 hours prior to allocation.Women of reproductive potential must agree to use highly effective methods of birth control during the period of therapy and for 12 months after the last dose of the study therapy.

14. Male participants must agree to use a contraception as detailed in Appendix 3 of this protocol during the treatment period and for at least 120 days following the last dose of study treatment and refrain from donation sperm during this period.

15. A female participant is eligible to participate if she is not pregnant, not breastfeeding, and at least one of the following conditions applies:

    i. Not a WOCBP or ii. A WOCBP who agrees to follow the contraceptive guidance during the treatment period for at least 120 days after the last dose of study treatment. Note: Cases of pregnancy that occur during maternal exposures to treatment should be reported. If a patient is determined to be pregnant following treatment initiation, treatment must be discontinued immediately.

16. Women must agree not to breast feed while on therapy and for at least 120 days following the last dose of study drug.

    • Hematological Absolute neutrophil count (ANC) ≥1500/μL Platelets ≥100 000/μL Hemoglobin ≥9.0 g/dL or ≥5.6 mmol/L
    • Renal Creatinine OR Measured or calculated creatinine clearance (GFR can also be used in place of creatinine or CrCl) ≤1.5 × ULN OR ≥30 mL/min for participant with creatinine levels <1.5 × institutional ULN
    • Hepatic Total bilirubin ≤1.5 ×ULN OR direct bilirubin ≤ULN for participants with total bilirubin levels >1.5 × ULN AST (SGOT) and ALT (SGPT) ≤2.5 × ULN (≤5 × ULN for participants with liver metastases)
    • Coagulation International normalized ratio (INR) OR prothrombin time (PT) Activated partial thromboplastin time (aPTT) ≤1.5 × ULN unless participant is receiving anticoagulant therapy as long as PT or aPTT is within therapeutic range of intended use of anticoagulants

Exclusion Criteria:

1. No limit on number of prior therapies
2. Evidence of significant intracranial hemorrhage
3. No other investigational or standard anti-tumor therapy allowed
4. Patients must not have a known history of allergic reaction attributed to study drugs or compounds of similar chemical or biologic composition unless allergic reaction was managed by pre-medication.
5. Patients must not have a serious pre-existing medical condition or uncontrolled intercurrent illness that would preclude participation in this study (for example, uncontrolled ventricular arrhythmia, interstitial lung disease, severe dyspnea at rest of requiring oxygen therapy, history of major surgical resection involving the stomach or bowel, or pre-existing Crohn's disease or ulcerative colitis or other autoimmune disease,) or psychiatric illness/social situations that would limit compliance with study requirements
6. Patients must not have a diagnosis of immunodeficiency or be receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug.
7. Patients must not have an active systemic fungal and/or known viral infection (for example human immunodeficiency virus antibodies, hepatitis B surface antigen, or hepatitis C antibodies).
8. Patients must not have a history of active tuberculosis.
9. Patients must not have an active infection requiring systemic therapy
10. Patients must not have known history of, or any evidence of active, non-infectious pneumonitis
11. Concomitant use of known strong CYP3A inhibitors (itraconazole, telithromycin, clarithromycin, protease inhibitors boosted with ritonavir or cobicistat, indinavir, saquinavir, nelfinavir, boceprevir, telaprevir)
12. Concomitant use of known strong CYP3A inducers (phenobarbital, enzalutamide, phenytoin, rifampicin, rifabutin, rifapentine, carbamazepine, nevirapine, and St. John's Wort)
13. A woman of childbearing potential (WOCBP) who has a positive urine pregnancy test within 72 hours prior to allocation.
14. Patients must not have other active concurrent malignancy. Exceptions include basal cell carcinoma of the skin or squamous cell carcinoma of the skin that has undergone potentially curative therapy or in situ cervical cancer.
15. Patients must not have active autoimmune disease that required systemic treatment in the past 2 years (i.e. with use of disease modifying agents, corticosteroids, or immunosuppressive drugs). Replacement therapy (e.g. thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc) is not considered a form of systemic treatment.
16. Patients must not have received a live vaccine within 30 days of planned start of study (Note: Seasonal influenza vaccines for injection are generally inactivated flu vaccines and are allowed; however, intranasal influenza vaccines are live attenuated vaccine and are not allowed)
17. Concurrent treatment on another clinical trial. Supportive care trials or nontherapeutic trials (i.e. quality of life) are allowed.

Memorial Sloan Kettering Cancer Center supports the international committee of medical journal editors (ICMJE) and the ethical obligation of responsible sharing of data from clinical trials. The protocol summary, a statistical summary, and informed consent form will be made available on clinicaltrials.gov when required as a condition of Federal awards, other agreements supporting the research and/or as otherwise required. Requests for deidentified individual participant data can be made beginning 12 months after publication and for up to 36 months post publication. Deidentified individual participant data reported in the manuscript will be shared under the terms of a Data Use Agreement and may only be used for approved proposals. Requests may be made to: crdatashare@mskcc.org