This phase I trial studies the side effects and best dose of ipilimumab when given together with whole brain radiation therapy or stereotactic radiosurgery in treating patients with melanoma with brain metastases. Monoclonal antibodies, such as ipilimumab, can block tumor growth in different ways. Some block the ability of the tumor to grow and spread. Others find Tumor cells and help kill them or carry tumor-killing substances to them. Radiation therapy, such uses high-energy x-rays and other types of radiation to kill tumor cells. Giving radiation therapy in different ways may kill more tumor cells. Giving ipilimumab together with whole-brain radiation therapy or stereotactic radiosurgery may kill more tumor cells.

Melanoma patients with brain metastases have a very poor outcome. Most patients with brain metastases will die from CNS related death. Radiation therapy is an effective treatment for patients with brain metastases for symptom palliation and survival benefit. Patients with multiple metastases are typically treated with whole brain radiation treatment (WBRT). For patients with a few metastases, stereotactic radiosurgery (SRS) alone can be used as an alternative. CTLA-4 (Cytotoxic T-Lymphocyte Antigen 4) antibody, Ipilimumab has shown efficacy in metastatic melanoma and unresectable melanoma. Treatment with high doses of radiation therapy would result in tumor cell death, releasing tumor debris and liberating potential tumor antigens. We hypothesize that combining radiation therapy with Ipilimumab will facilitate immune recognition of these novel tumor-specific antigens, yielding a synergistic effect. Further, the hypothesis of this study is that this combination could focus the immune system on tumor antigens and minimize the aberrant immune activation in normal tissues, consequently reducing the incidence of irAE's (immune related adverse effect). Combination of radiation treatment with Ipilimumab will likely result in better local control, decrease the risk of developing new brain metastases, and improved overall survival. However, the safety profile and toxicities of combining Ipilimumab with brain radiation treatment are unknown. The current phase I study will assess the safety profile of combining different doses of Ipilimumab with standard dose radiation treatment either with WBRT or SRS. The MTD (maximum tolerated dose) will be determined, as well as a recommended phase II trial dose of Ipilimumab.

Inclusion Criteria:

1. Patient age is >= 18 years

2. Patient has an Eastern Cooperative Oncology Group (ECOG) performance status of 0, or 1.

3. Ability to provide written informed consent obtained prior to participation in the study and any related procedures being performed

4. Patients must meet the following laboratory criteria:

   • WBC >= 2000/uL
   • ANC >= 1000/uL
   • Platelets >= 75 x 10^3/uL
   • Hemoglobin >= 9 g/dL (>= 80 g/L; may be transfused)
   • AST/ALT <= 2.5 x ULN for patients without liver metastasis
   • AST/ALT <= 5 times for liver metastases
   • Bilirubin <= 2.0 x ULN, (except patients with Gilbert's Syndrome, who must have a total bilirubin less than 3.0 mg/dL)
   • Serum creatinine <= 2.0 x ULN or 24-hour creatinine clearance >= 50 ml/min
   • Total serum calcium (corrected for serum albumin) or ionized calcium >= lower limit of normal (LLN)
   • Serum potassium >= LLN
   • Serum sodium >= LLN
   • Serum albumin >= LLN or 3g/dl
   • Patients with any elevated Alkaline Phosphatase due to bone metastases can be enrolled

5. No active or chronic infection with HIV, Hepatitis B, or Hepatitis C.

6. Clinically euthyroid. Note: Patients are permitted to receive thyroid hormone supplements to treat underlying hypothyroidism.

7. Women of childbearing potential (WOCBP) must be using an adequate method of contraception to avoid pregnancy throughout the study and for up to 26 weeks after the last dose of investigational product, in such a manner that the risk of pregnancy is minimized. WOCBP include any female who has experienced menarche and who has not undergone successful surgical sterilization (hysterectomy, bilateral tubal ligation, or bilateral oophorectomy) or is not post-menopausal. Post-menopause is defined as:

   • Amenorrhea >= 12 consecutive months without another cause, or
   • For women with irregular menstrual periods and taking hormone replacement therapy (HRT), a documented serum follicle stimulating hormone (FSH) level >= 35 mIU/mL.
   • Women who are using oral contraceptives, other hormonal contraceptives (vaginal products, skin patches, or implanted or injectable products), or mechanical products such as an intrauterine device or barrier methods (diaphragm, condoms, spermicides) to prevent pregnancy, or are practicing abstinence or where their partner is sterile (eg, vasectomy) should be considered to be of childbearing potential.
   • WOCBP must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of HCG) within 72 hours before the start of ipilimumab.

8. Men of fathering potential must be using an adequate method of contraception to avoid conception throughout the study (and for up to 26 weeks after the last dose of investigational product) in such a manner that the risk of pregnancy is minimized.

9. A pathological diagnosis of melanoma is required, from either the primary or a metastasis. This also includes uveal melanoma.

10. A radiological diagnosis (CT or MRI) of one or more brain metastases is required.

11. Resolution of all acute toxic effects of prior chemotherapy or radiotherapy or surgical procedures to NCI CTCAE Version 4.0 grade <= 1.

12. Specific eligibility criteria for the two arms

    • Arm A WBRT and Ipilimumab:

      • Patients have 5 or more brain metastases, or patients have any brain metastases exceeding the limit for SRS (maximum diameter is > 4 cm).
      • OR Patient has only one brain metastasis and completely resected, the resection cavity is > 4 cm in diameter.

    • Arm B SRS and Ipilimumab:

      • Patients have 4 or fewer brain metastases. All the brain metastases are <= 4 cm in diameter.
      • Patients have only one brain metastasis and completely resected, the resection cavity is <= 4 cm in diameter.
      • OR If a patient is found to have progression of brain metastases that exceed 4 cm in diameter based on the MRI scan on the time of SRS procedure, the patient should be re-assigned to WBRT arm or withdrawn from the study. The study PI should be notified.
      • OR If a patient is found to have progression of brain metastases that exceed 4 lesions based on the MRI scan on the time of the SRS procedure, the patient can either be treated with SRS to all the lesions (up to 10 lesions), re-assigned to WBRT arm, or withdrawn from the study per the treating physician. The study PI should be notified.

Exclusion Criteria:

1. Patient with leptomeningeal carcinomatosis.

2. Autoimmune disease: Patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's Disease, are excluded from this study, as are patients with a history of symptomatic disease (eg, rheumatoid arthritis, systemic progressive sclerosis [scleroderma], systemic lupus erythematosus, autoimmune vasculitis [eg, Wegener's Granulomatosis]); motor neuropathy considered of autoimmune origin (e.g. Guillain-Barre Syndrome and Myasthenia Gravis).

3. Major surgery or radiation therapy within 2 weeks of starting the study treatment.

4. If patients are receiving chemotherapy or other investigational drugs, they must be discontinued 4 weeks prior to enrollment.

5. NCI CTCAE grade 3 hemorrhage within 4 weeks of starting the study treatment.

6. Any of the following within the 6 months prior to study drug administration:

   myocardial infarction, severe/unstable angina, coronary/peripheral artery bypass graft, symptomatic congestive heart failure, cerebrovascular accident or transient ischemic attack, or pulmonary embolism. Patients with known cardiac disease will be required to have an ECHO or MUGA scan at baseline and at the completion of study.

7. Ongoing cardiac dysrhythmias of NCI CTCAE grade >= 2.

8. Hypertension that cannot be controlled by medications (>150/100 mm Hg despite optimal medical therapy).

9. Pre-existing thyroid abnormality with thyroid function that cannot be maintained in the normal range with medication.

10. Known human immunodeficiency virus (HIV) or acquired immunodeficiency syndrome (AIDS)-related illness or other active infection.

11. Pregnancy or breastfeeding. Female subjects must be surgically sterile or be postmenopausal, or must agree to use effective contraception during the period of therapy. All female subjects with reproductive potential must have a negative pregnancy test (serum or urine) prior to enrollment. Male subjects must be surgically sterile or must agree to use effective contraception during the period of therapy. The definition of effective contraception will be based on the judgment of the principal investigator or a designated associate.

12. Other severe acute or chronic medical or psychiatric condition or laboratory abnormality that may increase the risk associated with study participation or study drug administration, or may interfere with the interpretation of study results, and in the judgment of the investigator would make the subject inappropriate for entry into this study.

13. Concomitant therapy with any of the following: IL-2, interferon, or other non-study immunotherapy regimens, cytotoxic chemotherapy, immunosuppressive agents, vermurafenib, or other investigational therapies.

14. All efforts need to be taken to avoid/minimize the use of cortical steroid during radiation period (1 week prior to start radiation, during radiation, and 1 week after finishing radiation). Any steroid used considered necessary by the treating physician should be closely documented, including medication, route of administration, dose, and duration.

15. Active infection with hepatitis B, or hepatitis C.

16. Patients who had prior brain radiation treatment.

17. Patients are excluded if they have a history of any other malignancy from which the patient has been disease-free for less than 2 years, with the exception of adequately treated basal or squamous cell carcinoma of skin, superficial bladder cancer or carcinoma in situ of cervix, AJCC (version 7.0) stage 0 or I breast cancer, AJCC (version 7.0) stage I, or II prostate cancer.

18. Patients are excluded if they have a history of autoimmune disease, as follows: Patients with a history of inflammatory bowel disease are excluded from this study as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis [scleroderma], Systemic Lupus Erythematosus, autoimmune vasculitis [e.g., Wegener's Granulomatosis]). Patients with motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre Syndrome and Myasthenia Gravis) are excluded. Patients with a history of autoimmune thyroiditis are eligible if their current thyroid disorder is treated and stable with replacement or other medical therapy.

19. Patients who are allergic to Ipilimumab.

20. Patients who received ipilimumab before.

21. Patients who are allergic to MRI contrast agent or have contraindication for MRI.