This phase I trial investigates how well short-course radiation therapy followed by combination chemotherapy works in treating patients with stage II-III rectal cancer. Radiation therapy uses high energy x-rays to kill tumor cells and shrink tumors. Chemotherapy drugs, such as leucovorin, fluorouracil, oxaliplatin, and capecitabine, work in different ways to stop the growth of tumor cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Giving short-course radiation therapy and combination chemotherapy may reduce the need for surgery and therefore improve quality of life.

PRIMARY OBJECTIVE:

I. Complete clinical response (cCR) rate of patients with clinical T3 and/or node-positive M0 rectal cancer being treated with short-course radiation therapy (SCRT) followed by 16 weeks of modified leucovorin, fluorouracil, and oxaliplatin (mFOLFOX)/capecitabine and oxaliplatin (CapeOX).

SECONDARY OBJECTIVES:

I. 1-year local recurrence free survival and 1-year progression free survival of the entire cohort, the cohort that initially undergoes non-operative management (NOM), and the cohort that initially undergoes total mesorectal excision (TME).

II. Physician-reported acute and late >= grade 3 toxicity rates. III. 1-year post-treatment patient health-related quality of life and anorectal function as per Patient Reported Outcomes Measurement and Information System (PROMIS).

IV. Explore how Signatera's residual disease test correlates with patient's cCR rates, local recurrence, progression-free, and overall survival rates.

V. Explore radiomics features from longitudinal diffusion weighted magnetic resonance imaging (MRI) (diffusion weighted imaging [DWI]) data and build a predictive model for treatment effect (complete response) in rectal cancer patients undergoing SCRT.

OUTLINE:

Patients undergo SCRT in the form of intensity-modulated radiation therapy (IMRT) over 5 fractions daily for 5 consecutive days. Beginning 11-18 days after the last day of radiation therapy, patients receive either oxaliplatin intravenously (IV) and leucovorin IV on day 1 and fluorouracil IV on days 1-3 (mFOLFOX6) or oxaliplatin IV on day 1 and capecitabine orally (PO) twice daily (BID) on days 1-14 (CapeOX). Treatment with mFOLFOX6 repeats every 2 weeks for up to 8 cycles, and treatment with CapeOX repeats every 3 weeks for up to 6 cycles in the absence of disease progression or unacceptable toxicity. At 8-12 weeks after completion of all therapy, patients with residual tumor undergo TME. Patients with cCR undergo NOM.

After completion of study treatment, patients who underwent NOM are followed up every 3 months for 2 years, then every 6 months for 3 years. TME patients are followed up every 3-6 months for 2 years, then every 6 months for 3 years.

Inclusion Criteria:

• Histologically confirmed rectal adenocarcinoma
• Patients must have stage II (cT3, cN0) or stage III (cT1-3, cN1-3) tumor as staged by MRI
• No evidence of metastatic disease
• Resectable primary lesion
• Karnofsky performance status (KPS) >= 70 or Eastern Cooperative Oncology Group (ECOG) 0-2
• Absolute neutrophil count (ANC) > 1.5 cell/mm^3
• Hemoglobin (Hgb) > 8.0 gm/dL
• Platelets (PLT) > 150,000/mm^3
• Total bilirubin < or equal to 1.5 x upper limit of normal
• Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) < or equal to three times upper limit of normal
• If a woman is of childbearing potential, a negative serum pregnancy test must be documented prior to initiation of radiation therapy

Exclusion Criteria:

• Active treatment of a separate malignancy
• Distant metastatic disease as assessed by staging positron emission tomography (PET)/computed tomography (CT) or CT of the chest and abdomen within 6 weeks of starting radiation therapy
• Prior radiotherapy to the region of the study cancer that would result in overlap of radiation therapy fields
• Pregnant and/or breastfeeding
• Medical/psychological contraindication to MRI

Analysts blinded to patient outcome and sample order