This phase Ib/II trial studies the side effects, best dose, and effectiveness of enfortumab vedotin (EV) in combination with pembrolizumab and radiation therapy for treating patients with muscle invasive bladder cancer. Standard of care treatment for muscle invasive bladder cancer is chemotherapy, to shrink the tumor before the main treatment is given (neoadjuvant), followed by surgery to remove all of the bladder as well as nearby tissues and organs (radical cystectomy). In cases where patients are not candidates for the standard of care approach or prefer a bladder sparing option, tri-modality therapy with transurethral resection of bladder tumor (TURBT) followed by combined chemotherapy and radiation therapy is used. Enfortumab vedotin is a monoclonal antibody, enfortumab, linked to an anticancer drug called vedotin. It works by helping the immune system to slow or stop the growth of tumor cells. Enfortumab attaches to a protein called nectin-4 on tumor cells in a targeted way and delivers vedotin to kill them. It is a type of antibody-drug conjugate. Immunotherapy with monoclonal antibodies, such as pembrolizumab, may help the body's immune system attack the cancer, and may interfere with the ability of tumor cells to grow and spread. Intensity-modulated radiation therapy is a type of 3-dimensional radiation therapy that uses computer-generated images to show the size and shape of the tumor. Thin beams of radiation of different intensities are aimed at the tumor from many angles. This type of radiation therapy reduces the damage to healthy tissue near the tumor. Giving enfortumab vedotin with pembrolizumab and radiation therapy may work better in treating patients with muscle invasive bladder cancer.

PRIMARY OBJECTIVES:

I. To determine the recommended phase II dose (RP2D) of enfortumab vedotin given in combination with pembrolizumab and concurrent radiation therapy (RT). (Phase Ib).

II. To assess the toxicity and safety of enfortumab vedotin given in combination with pembrolizumab and concurrent radiation therapy (RT). (Phase Ib).

III. To evaluate the rate of clinical complete response to treatment (cCR) at the RP2D based on 6-month post-treatment cystoscopy, TURBT, cytology, and cross sectional imaging. (Phase II).

IV. To characterize the safety of enfortumab vedotin at the RP2D given in combination with pembrolizumab and concurrent radiation therapy (RT). (Phase II).

SECONDARY OBJECTIVES:

I. To evaluate the rate of cCR 6 months post-treatment start based on cystoscopy, TURBT, and cross sectional imaging. (Phase Ib) II. To evaluate the preliminary efficacy of enfortumab vedotin given in combination with pembrolizumab and concurrent radiation therapy (RT) as measured by 1-year recurrence free survival rate, 2-year cystectomy-free survival rate, 2-year overall survival rate, the median recurrence free survival (RFS) rate, median overall survival, and median cystectomy-free survival. (Phase Ib and Phase II) III. To assess the downstaging to ≤ pT1N0 or less following completion of treatment, based on 6-month post-treatment cystoscopy, for participants treated at RP2D of enfortumab vedotin (EV). (Phase Ib and Phase II) IV. To assess the downstaging to ≤ pT1N0 or less following completion of treatment, based on 6-month post-treatment cystoscopy, for participants across all EV dose levels. (Phase Ib and Phase II).

EXPLORATORY OBJECTIVES:

I. Assessment of change in tumor gene expression signatures using single-cell ribonucleic acid (RNA) sequencing (scRNA-Seq) following initiation of combination treatment.

II. Assessment of the change in the populations of tumor-infiltrating immune cells (TIICs) induced by initiation of the combination treatment.

III. Determination of the impact of EV/pembrolizumab combination treatment on programmed death-ligand 1 (PD-L1) expression in tumor cells and TIICs, as well as on other immunologic predictive markers.

IV. Assessment of modulation of tumor microenvironment pre- and post-initiation of combination treatment using multiplex immunohistochemistry (IHC).

V. Assessment of the modulation of circulating immune cells following initiation of combination treatment using mass cytometry (cytometry by time of flight, or CyTOF).

VI. Assessment of the change in T-cell receptor repertoire by scRNA-Seq following initiation of combination treatment.

VII. Change in tumor expression of nectin cell adhesion molecule 4 (Nectin-4) following combination treatment relative to baseline.

VIII. Assessment in the change in CD3+ T cell density (T cell count/μm^2) from baseline biopsy to post-RT biopsy in participants with residual tumor, across all EV dose levels and at RP2D.

OUTLINE:

This is a phase Ib, dose escalation study of enfortumab vedotin followed by a phase II study.

Participants receive enfortumab vedotin and pembrolizumab for up to 5 cycle of enfortumab vedotin and up to 17 cycles of pembrolizumab in the absence of disease progression or unacceptable toxicity. Beginning on cycle 1 day 1, participants also undergo non-investigational, standard of care intensity modulated radiation therapy (IMRT) over 6.5-8 weeks.

After completion of study treatment, participants are followed up at 90 days and then every 12 weeks for up to 5 years.

Inclusion Criteria:

• Biopsy-confirmed muscle-invasive bladder cancer (cT2,T3,T4a). (Note: Tissue samples are required.) (Participants with cT3/T4a staged disease will be capped at 25% of patients treated at RP2D).

• Urothelial histology present. Mixed histologies other than small cell/neuroendocrine are allowed as long as some urothelial histology is present. Neuroendocrine histology of any component and pure variant (non-urothelial) histology tumors will be excluded. (Patients with < 50% urothelial histology will be capped at 25% of patients treated at RP2D).

• Must be judged by the investigator to be ineligible for radical cystectomy or electing not to undergo radical cystectomy.

• Must be eligible for and agree to receive bladder irradiation as determined by the treating investigator.

• Must have a TURBT within 8 weeks of combination treatment start with viable tumor content. If no viable tumor content is present on TURBT, the patient will be replaced in the study.

• Patients who have autoimmune disease will be evaluated on a case-by-case basis and can only enroll so long as participants are not on active immunosuppression with a corticoid steroid allowance exceeding 10mg of prednisone or equivalent per day.

• Age >= 18 years.

• Eastern Cooperative Oncology Group (ECOG) performance status <= 1.

• Absolute neutrophil count ≥ 1,500/microliter (mcL).

• Platelets >= 100,000/mcL.

• Hemoglobin >= 9.0 g/dL or ≥ 5.6 mmol/L.

  * Criteria must be met without erythropoietin dependency and without packed red blood cell (pRBC) transfusion within last 2 weeks

• Total bilirubin <= 1.5 × upper limit of normal, unless elevated due to Gilbert's syndrome and direct bilirubin is within normal limits.

• Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase (SGOT)) <= 2.5 X institutional upper limit of normal.

• Alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase (SGPT)) <= 2.5 X institutional upper limit of normal.

• Creatinine clearance glomerular filtration rate (GFR) >= 30 mL/min/1.73 m^2, calculated by Cockcroft-Gault or measured using 24-hour creatinine clearance.

• International normalized ratio (INR) OR prothrombin time (PT) <= 1.5 × upper limit of normal (ULN).

  * If participant is receiving anticoagulant therapy, as long as PT or activated partial thromboplastin time (aPTT) is within therapeutic range of intended use of anticoagulants, participant is eligible.

• Activated partial thromboplastin time (aPTT) <= 1.5 × ULN.

  * If participant is receiving anticoagulant therapy, as long as PT or aPTT is within therapeutic range of intended use of anticoagulants, participant is eligible.

• Ability to understand and the willingness to sign a written informed consent document.

• Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial.

• Participants who are hepatitis B virus surface antigen (HBsAg) positive are eligible if they have received hepatitis B virus (HBV) anti-viral therapy for at least 4-weeks, and have undetectable HBV viral load prior to randomization. Participants should remain on anti-viral therapy throughout study intervention and follow local guidelines for HBV anti-viral therapy post completion of study intervention.

  * Note: Hepatitis B screening tests are not required unless patients have a known history of HBV infection.

• Participants with a history of hepatitis C virus (HCV) infection are eligible if HCV viral load is undetectable at screening.

  * Note: Participants must have completed curative anti-viral therapy at least 4 weeks prior to cycle 1 day 1.

• Women of child-bearing potential and men with sexual partners of childbearing potential must agree to use adequate contraception for the duration of study participation. Enfortumab vedotin (EV) may cause fetal harm. Women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Men with female partners who are women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Acceptable methods include barrier method, hormonal method, as well as intrauterine devices

• Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 8 weeks after last administration of study treatment.

Exclusion Criteria:

• Presence of distant metastases on imaging (M1 disease).

• Presence of ≥ N2 disease on imaging (N1 disease allowed, but participants with N1 disease will be capped at 25% of patients treated at RP2D).

• Presence of small cell / neuroendocrine histology in tumor sample (any content).

• Absence of urothelial histology in TURBT tumor sample (pure variant histology).

• Presence of untreated upper tract urothelial cancer.

• Presence of severe hydronephrosis precluding therapy in the judgement of the treating physician.

• Presence of extensive carcinoma in situ (CIS) is exclusionary; moderate CIS that could still benefit from radiation treatment in the judgement of treating physician is allowed.

• Baseline neuropathy grade 2 (G2) or greater.

• Baseline uncontrolled diabetes mellitus.Uncontrolled diabetes is defined as hemoglobin A1c (HbA1c) ≥ 8% or HbA1c 7 to < 8% with associated diabetes symptoms (polyuria or polydipsia) that are not otherwise explained.

  * Note: Patients with prior diagnosis but with disease under control are eligible

• Prior treatment with systemic immunotherapy or chemotherapy for urothelial cancer. (with the exception of prior systematic therapy treatment >12 months prior). Note: Prior bacillus calmette-guerin (BCG) and intravesical treatments are allowed

• Has received prior systemic anti-cancer therapy including investigational agents within 4 weeks prior to cycle 1 day 1.

• Has received prior radiotherapy within 2 weeks of cycle 1 day 1 or had radiation-related toxicities requiring corticosteroids.

• Has received a live vaccine or live-attenuated vaccine within 30 days before the first dose of study intervention.

  * Note: Administration of killed vaccines is allowed. Any licensed coronavirus 2019 (COVID-19) vaccine (including for emergency use) is allowed in the study as long as they are messenger ribonucleic acid (mRNA) vaccines, replication-incompetent adenoviral vaccines, or inactivated vaccines.

• Major surgery within 2 weeks prior to first dose of EV.

  * Note: Cataract surgery, standard tissue biopsies, and standard of care cardiac devices, such as a pacemaker or stent placed on an elective basis, are allowable procedures.

• Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug.

  * Note: Inhaled or topical steroids are permitted.

• HIV-infected participants with a history of Kaposi's sarcoma and/or Multicentric Castleman's Disease.

• Has known active CNS metastases and/or carcinomatous meningitis.

• History of another significant life-limiting malignancy requiring systematic treatment within 2 years prior to the first dose of study drugs, or any evidence of residual disease from a previously diagnosed malignancy.

  * Note: Patients with nonmelanoma skin cancer, curatively treated localized prostate cancer, curatively treated upper tract urothelial cancer, or carcinoma in situ of any type (if complete resection was performed) are allowed.

• Hypersensitivity to pembrolizumab or enfortumab vedotin, or any of their excipients.

• Prior allogeneic stem cell or solid organ transplant.

• Has a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or has current pneumonitis/interstitial lung disease.

• Has an active infection requiring systemic therapy.

• Has a history or current evidence of any condition, therapy, or laboratory abnormality or other circumstance that might confound the results of the study, interfere with the participant's participation for the full duration of the study, such that it is not in the best interest of the participant to participate, in the opinion of the treating investigator.

• History of Hepatitis B with detectable HBV viral load (participants who are HBsAg positive are eligible if they have received HBV anti-viral therapy for at least 4 weeks, and/or have undetectable HBV viral load prior to randomization) or known active hepatitis C virus (defined as detectable HCV RNA .[qualitative]) infection.

  * Note: Testing for Hepatitis B or C is not required unless clinically indicated or if there is a known history of hepatitis infection. Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial.

• Pregnant and breast feeding participants are excluded from this study because targeted chemotherapy and radiation have the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with EV+pembrolizumab, breastfeeding should be discontinued if the mother is treated with these investigational products.