This randomized phase I/II clinical trial is studying the side effects and best dose of gamma-secretase/notch signalling pathway inhibitor RO4929097 when given together with vismodegib and to see how well they work in treating patients with advanced or metastatic sarcoma. Vismodegib may slow the growth of tumor cells. Gamma-secretase/notch signalling pathway inhibitor RO4929097 may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Giving vismodegib together with gamma-secretase/notch signalling pathway inhibitor RO4929097 may be an effective treatment for sarcoma.

PRIMARY OBJECTIVES:

I. To determine the maximum-tolerated dose (MTD) of gamma-secretase inhibitor RO4929097 (RO4929097) when given in combination with fixed-dose Hedgehog antagonist GDC-0449 (GDC-0449) which will become the recommended dose for the phase II portion of this study. (Phase Ib) II. To assess the progression-free survival (PFS) of the combination of RO4929097 with and without GDC-0449 in two arms of patients with advanced sarcoma. (Phase II)

SECONDARY OBJECTIVES:

I. To describe the tolerability and adverse event profile of daily GDC-0449 administered orally in combination with daily RO4929097 administered orally for 21 consecutive days. (Phase Ib) II. To describe the pharmacokinetics of the combination of the combination of GDC-0449 and RO4929097. (Phase Ib) III. To assess Response Evaluation Criteria In Solid Tumors (RECIST) 1.1 overall response rates (complete and partial response [CR+PR]) for combination therapy. (Phase Ib and II) IV. To conduct pharmacodynamic studies in tissue biopsies (pre- and post-study) for explorative and hypothesis-generating studies. (Phase Ib and II) V. To assess overall survival. (Phase II) VI. To further describe the pharmacokinetics and pharmacodynamics of the combination of GDC-0449 and RO4929097 at the phase II dose at the continuous schedule. (Phase II) V. To conduct pharmacodynamic studies in tissue biopsies (pre- and post- study drug[s]) for explorative and hypothesis generating studies. (Phase II)

OUTLINE: This is a phase I, dose-escalation study of gamma-secretase/notch signalling pathway inhibitor RO4929097 followed by a phase II study.

PHASE IB:

PART A: Patients receive vismodegib orally (PO) once daily (QD) on days 1-21.

PART B: Beginning within 7 days of finishing part A, patients receive vismodegib PO and gamma-secretase/notch signalling pathway inhibitor RO4929097 PO QD on days 1-21. Courses repeat every 21 days in the absence of disease progression or unacceptable toxicity.

PHASE II: Patients are randomized to 1 of 2 treatment arms.

ARM I: Patients receive gamma-secretase/notch signalling pathway inhibitor RO4929097 PO QD on days 1-21.

ARM II: Patients receive vismodegib PO and gamma-secretase/notch signalling pathway inhibitor RO4929097 PO QD on days 1-21.

In both arms, courses repeat every 21 days in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up for 4 weeks.

Inclusion Criteria:

• Patients must have histologically or cytologically confirmed sarcoma

• All Patients must have measurable disease as defined by RECIST 1.1

• Eastern Cooperative Oncology Group (ECOG) performance status =< 2 (Karnofsky >= 60%)

• There is a minimum of 1 prior therapy; however, there are no minimum systemic therapy requirements for well differentiated or de-differentiated liposarcoma, clear cell sarcoma, chondrosarcoma, alveolar soft part sarcoma and chordomas which have no effective therapies; for Phase Ib, there are no maximum limits to number of prior therapies; for Phase II, there is a maximum of 5 prior chemotherapy regimens including tyrosine kinase inhibitors (TKI); the last dose of systemic therapy (including TKI) must have been given at least 2 weeks prior to initiation of therapy; patients receiving nitrosourea (such as BCNU) or mitomycin C must have received their last dose of such therapy at least 6 weeks prior to initiation of therapy; patients receiving bevacizumab must wait at least 4 weeks; patients receiving experimental immunotherapy or antibody based therapies must wait a minimum of 4 weeks or 4 half-lives, whichever is longer; this should be discussed with the principal investigator before registration; tumor biopsies should be performed only after meeting these requirements; patients should recover to less than Common Terminology Criteria for Adverse Events (CTCAE) grade 2 toxicities related to previous therapies to be eligible

• Patients with metastatic or locally advanced (inoperable) gastrointestinal stromal tumor (GIST) must have progressed on imatinib and sunitinib or be intolerant to both drugs; the last dose of tyrosine kinase inhibitors imatinib or sunitinib should be given at least 2 weeks prior to initiation of therapy

• Patients with brain metastasis that have been treated with definitive surgery or radiation and have been clinically stable for 3 months following the procedure with no neurological signs or symptoms and no requirement for systemic glucocorticoids are eligible for study

• Patients must not have current evidence of another malignancy except non-melanoma skin cancer and superficial bladder cancer

• Leukocytes >= 3,000/mcL

• Absolute neutrophil count >= 1,500/mcL

• Hemoglobin >= 9 g/dl

• Platelets >= 100,000/mcL

• Total bilirubin =< upper limit of normal

• Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase [SGOT])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 3 X upper limit of normal

• Creatinine =< 1.5 or creatinine clearance >= 60 mL/min/1.73 m^2 for patients with creatinine levels above 1.5

• Patients treated at Memorial Sloan-Kettering Cancer Center may consent to optional tumor biopsies before and after initiation of study drug; tumor biopsies should be obtained after fulfilling requirements

• Women of child-bearing potential and men must use two forms of contraception (i.e., barrier contraception and one other method of contraception) at least 4 weeks prior to study entry, for the duration of study participation, and for at least 12 months post-treatment; should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately; women of childbearing potential are required to have a negative pregnancy test (with a sensitivity of at least 25 mIU/mL) within 7 days and within 24 hours prior to the first dose of GDC-0449 and/or RO4929097 (serum or urine); a pregnancy test (serum or urine) will be administered every 3 weeks to all women of childbearing potential, at the start of each drug cycle; a positive urine test must be confirmed by a serum pregnancy test; prior to dispensing GDC-0449 and/or RO4929097, the investigator must confirm and document the patient's use of two contraceptive methods, dates of negative pregnancy test, and confirm that patient understands the teratogenic potential of GDC-0449 and/or RO4929097

  • Female patients of childbearing potential are defined as follows:

    • Patients with regular menses
    • Patients, after menarche with amenorrhea, irregular cycles, or using a contraceptive method that precludes withdrawal bleeding
    • Women who have had tubal ligation

  • Female patients may be considered to NOT be of childbearing potential for the following reasons:

    • The patient has undergone total abdominal hysterectomy with bilateral salpingo-oophorectomy or bilateral oophorectomy
    • The patient is medically confirmed to be menopausal (no menstrual period) for 24 consecutive months.
    • The patient is post-menopausal defined by amenorrhea for at least 1 year in a woman > 45 years old

• Ability to understand and the willingness to sign a written informed consent document

Exclusion Criteria:

• Patients may not receive other investigational agents within 2 weeks of enrollment in this study; patients treated with bevacizumab should be off therapy for 4 weeks; other experimental or immuno therapies should wait for 4 half-lives or 4 weeks, whichever is longer; prior exposure to Notch or Hedgehog inhibitors is not allowed; patients who have not recovered to less than CTCAE grade 2 from prior therapies are ineligible
• History of allergic reactions attributed to compounds of similar chemical or biologic composition to GDC-0449 or RO4929097 used in the study
• Patients taking medications with narrow therapeutic indices that are metabolized by cytochrome P450 (CYP450), including warfarin sodium (Coumadin®) are ineligible; patients on warfarin may be considered for enrollment after cessation of warfarin and appropriate transition to alternate anti-coagulation agents
• Preclinical studies indicate that RO4929097 is a substrate of cytochrome P450, family 3, subfamily A, polypeptide 4 (CYP3A4) and inducer of CYP3A4 enzyme activity; caution should be exercised when dosing RO4929097 concurrently with CYP3A4 substrates, inducers, and/or inhibitors; furthermore, patients who are taking concurrent medications that are strong inducers/inhibitors or substrates of CYP3A4 should be switched to alternative medications to minimize any potential risk; the following medications with strong potential for interaction are not allowed: indinavir, nelfinavir, ritonavir, clarithromycin, itraconazole, ketoconazole, nefazodone
• Caution should be exercised when dosing GDC-0449 concurrently with medications that are substrates of cytochrome P450 family 2, subfamily C, polypeptide 8 (CYP2C8), CYP2C9, and CYP2C19 and have narrow therapeutic windows; caution should be exercised when dosing GDC-0449 concurrently with inhibitors of CYP3A4
• Patients must be able to swallow pills; patients with malabsorption syndrome or other condition that would interfere with intestinal absorption
• Patients with clinically active liver disease, including active viral or other hepatitis or cirrhosis, are ineligible
• Patients with uncontrolled electrolyte abnormalities including hypophosphatemia, hypocalcemia, hypomagnesemia, hyponatremia or hypokalemia or defined as less than the lower limit of normal for the institution, despite adequate electrolyte supplementation are excluded from this study
• Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, a history of torsades de pointes or other significant cardiac arrhythmias that require antiarrhythmics or other medications known to prolong corrected QT interval (QTc); psychiatric illness/social situations that would limit compliance with study requirements
• Pregnant women are excluded from this study; breastfeeding should be discontinued if the mother is treated with GDC-0449 and/or RO4929097
• Human immunodeficiency virus (HIV)-positive patients on combination antiretroviral therapy are ineligible
• Cardiovascular: baseline QTc > 450 msec (male) or QTc > 470 msec (female)