This phase I/II trial studies the side effects and best dose of adavosertib and irinotecan hydrochloride in treating younger patients with solid tumors that have come back (relapsed) or that have not responded to standard therapy (refractory). Adavosertib and irinotecan hydrochloride may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.

PRIMARY OBJECTIVES:

I. To estimate the maximum tolerated dose (MTD) and/or recommended phase 2 dose of adavosertib (AZD1755 [MK-1775]) administered on days 1 through 5 every 21 days, in combination with oral irinotecan (irinotecan hydrochloride), to children with recurrent or refractory solid tumors.

II. To define and describe the toxicities of AZD1755 (MK-1775) in combination with oral irinotecan administered on this schedule.

III. To characterize the pharmacokinetics of AZD1755 (MK-1775) in children with refractory cancer.

SECONDARY OBJECTIVES:

I. To preliminarily define the antitumor activity of AZD1755 (MK-1775) and irinotecan within the confines of a Phase 1 study.

II. To obtain initial phase 2 efficacy data on the anti-tumor activity of AZD1755 (MK-1775) in combination with irinotecan administered to children with relapsed or refractory neuroblastoma, in children with relapsed or refractory medulloblastoma/CNS PNET (central nervous system primitive neuroectodermal tumor) and in children with relapsed or refractory rhabdomyosarcoma.

III. To investigate checkpoint over-ride by AZD1755 (MK-1775) via the mechanism-based pharmacodynamic (PD) biomarker of decreased cyclin-dependent kinase 1 (CDK1) phosphorylation in correlative and exploratory studies.

IV. To evaluate potential predictive biomarkers of AZD1755 (MK-1775) sensitivity, including v-myc avian myelocytomatosis viral oncogene homolog (MYC), v-myc avian myelocytomatosis viral oncogene neuroblastoma derived homolog (MYCN), phosphorylated-WEE1 G2 checkpoint kinase (p-Wee1), enhancer of zeste homolog 2 (Drosophila) (EZH2) and gamma-H2A histone family, member gamma-(H2AX) in tumor tissues in correlative and exploratory studies.

OUTLINE: This is a phase I, dose-escalation followed by a phase II study.

Patients receive irinotecan hydrochloride orally (PO) and adavosertib PO on days 1-5. Treatment repeats every 21 days for up to 18 cycles in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up periodically.

Inclusion Criteria:

• Patients must have had histologic verification of malignancy at original diagnosis or relapse except in patients with intrinsic brain stem tumors, optic pathway gliomas, or patients with pineal tumors and elevations of cerebrospinal fluid (CSF) or serum tumor markers including alpha-fetoprotein or beta-human chorionic gonadotropin (HCG)

• Part A: Patients with relapsed or refractory solid tumors, including patients with primary or metastatic CNS tumors

• Part B: Patients with relapsed or refractory neuroblastoma

• Part C: Patients with relapsed or refractory medulloblastoma or CNS embryonal tumors formally classified as PNET (pineoblastoma, CNS neuroblastoma, CNS ganglioneuroblastoma, embryonal tumor with multi-layered rosettes, medulloepithelioma, CNS embryonal tumor with rhabdoid features [INI1 intact] and CNS embryonal tumor, not otherwise specified)

• Part D: Patients with relapsed or refractory rhabdomyosarcoma

• Part A: Patients must have a body surface area >= 0.35 m^2 at the time of study enrollment if enrolling on dose levels 1-5; patients must have a body surface area >= 0.46 m^2 at the time of study enrollment if enrolling on dose level 0

• Parts B, C, and D: Phase 2 Expansion: Patients must have a body surface area of > 0.49 m^2 at the time of study enrollment at the recommended phase 2 dose of AZD-1775

• Part A: Patients must have either measurable or evaluable disease

• Part B: Patients must have either measurable disease or must be evaluable for MIBG response without evidence of Response Evaluation Criteria in Solid Tumors (RECIST) measurable lesions; patients with neuroblastoma in bone marrow only are not eligible

• Part C: Patients must have measurable disease by computed tomography (CT) or magnetic resonance imaging (MRI)

• Part D: Patients must have measurable disease for Part D

• Patient's current disease state must be one for which there is no known curative therapy or therapy proven to prolong survival with an acceptable quality of life

• Karnofsky >= 50% for patients > 16 years of age and Lansky >= 50 for patients =< 16 years of age; note: neurologic deficits in patients with CNS tumors must have been relatively stable for at least 7 days prior to study enrollment; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score

• Patients must have fully recovered from the acute toxic effects of all prior anti-cancer chemotherapy

• At least 21 days after the last dose of myelosuppressive chemotherapy (42 days if prior nitrosourea)

• At least 14 days after the last dose of a long-acting growth factor (e.g. Neulasta) or 7 days for short-acting growth factor; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair

• At least 7 days after the last dose of a biologic agent; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair

• At least 42 days after the completion of any type of immunotherapy, e.g. tumor vaccines

• >= 21 days must have elapsed from infusion of lase dose of antibody, and toxicity related to prior antibody therapy must be recovered to grade =< 1

• At least 14 days after local palliative radiation therapy (XRT) (small port); at least 150 days must have elapsed if prior traumatic brain injury (TBI), craniospinal XRT or if >= 50% radiation of pelvis; at least 42 days must have elapsed if other substantial bone marrow radiation, including therapeutic doses of iobenguane (MIBG)

• Stem cell Infusion without TBI: no evidence of active graft vs host disease and at least 84 days must have elapsed after transplant or stem cell infusion

• Patients previously treated with irinotecan are eligible for this study

• For patients with solid tumors without known bone marrow involvement: peripheral absolute neutrophil count (ANC) >= 1000/mm^3

• For patients with solid tumors without known bone marrow involvement: platelet count >= 100,000/mm^3 (transfusion independent, defined as not receiving platelet transfusions for at least 7 days prior to enrollment)

• For patients with solid tumors without known bone marrow involvement: hemoglobin >= 8.0 g/dL (may receive red blood cell [RBC] transfusions)

• Patients with known bone marrow metastatic disease will be eligible for study provided they meet the blood counts (may receive transfusions provided they are not known to be refractory to red cell or platelet transfusions); these patients will not be evaluable for hematologic toxicity; at least 2 of every cohort of 3 patients must be evaluable for hematologic toxicity for Part A, the dose escalation part of the study; if dose-limiting hematologic toxicity is observed, all subsequent patients enrolled must be evaluable for hematologic toxicity

• Creatinine clearance or radioisotope glomerular filtration rate (GFR) >= 70 ml/min/1.73 m^2 or a serum creatinine based on age/gender as follows:

  • Age 1 to < 2 years: 0.6 mg/dL
  • Age 2 to < 6 years: 0.8 mg/dL
  • Age 6 to < 10 years: 1 mg/dL
  • Age 10 to < 13 years: 1.2 mg/dL
  • Age 13 to < 16 years: 1.5 mg/dL (males), 1.4 mg/dL (females)
  • Age >= 16 years: 1.7 mg/dL (males), 1.4 mg/dL (females)

• Bilirubin (sum of conjugated + unconjugated) =< 1.5 x upper limit of normal (ULN) for age

• Serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase [ALT]) =< 135 U/L; for the purpose of this study, the ULN for SGPT is 45 U/L

• Serum albumin >= 2 g/dL

• Correct QT interval (QTc) =< 480 msec; Note: Patients should avoid concomitant medication known or suspected to prolong QTc interval or cause torsades de pointes; if possible, alternative agents should be considered; patients who are receiving drugs that prolong the QTc are eligible if the drug is necessary and no alternatives are available

• Patients with seizure disorder may be enrolled if on non-enzyme inducing anticonvulsants and well controlled

• Nervous system disorders (Common Terminology Criteria for Adverse Events version 5.0 [CTCAE v5.0]) resulting from prior therapy must be =< grade 2, with the exception of decreased tendon reflex (DTR); any grade of DTR is eligible

• All patients and/or their parents or legally authorized representatives must sign a written informed consent; assent, when appropriate, will be obtained according to institutional guidelines

• Tissue blocks or slides must be sent if available, with exclusions; if tissue blocks or slides are unavailable, the study chair must be notified prior to study enrollment

• Patients must be able to swallow capsules

Exclusion Criteria:

• Pregnant or breast-feeding women may not be entered on this study as there is yet no available information regarding human fetal or teratogenic toxicities; pregnancy tests must be obtained in girls who are post-menarchal
• Males or females of reproductive potential may not participate unless they have agreed to use an effective double barrier contraceptive method for the entire duration of protocol therapy and for 3 months (males) and 1 month (females) after study drug discontinuation
• Patients receiving corticosteroids who have not been on a stable or decreasing dose of corticosteroid for at least 7 days prior to enrollment are not eligible
• Patients who are currently receiving another investigational drug are not eligible
• Patients who are currently receiving other anti-cancer agents are not eligible
• Patients who are currently receiving drugs that are strong or moderate inhibitors and/or inducers of cytochrome P450, family 3, subfamily A, polypeptide 4 (CYP3A4) or sensitive CYP3A4 substrates and CYP3A4 substrates with a narrow therapeutic range are not eligible; the use of aprepitant as an antiemetic is prohibited due to early drug interaction data demonstrating increased exposure to AZD1775 (MK-1775); caution should be exercised with concomitant administration of AZD1755 (MK-1775) and agents that are sensitive substrates of cytochrome P450, family 2, subfamily C, polypeptide 8 (CYP2C8), 2C9 and 2C19, or substrates of this enzyme with narrow therapeutic ranges, as well as agents that are inhibitors or substrates of permeability glycoprotein (P-gp)
• Patients who are receiving cyclosporine, tacrolimus or other agents to prevent graft-versus-host disease post bone marrow transplant are not eligible for this trial
• Patients must not have received enzyme inducing anticonvulsants for at least 14 days prior to enrollment
• Patients with cardiac diseases ongoing or in the past 6 months (e.g. congestive heart failure, acute myocardial infarction, significant uncontrolled arrhythmias) are not eligible for this trial
• Patients who have an uncontrolled infection are not eligible
• Patients who have received a prior solid organ transplantation are not eligible
• Patients who in the opinion of the investigator may not be able to comply with the safety monitoring requirements of the study are not eligible
• Patients with a history of allergic reaction to irinotecan, cephalosporins or a severe penicillin allergy are not eligible
• Patients unable to swallow capsules whole are not eligible; nasogastric or gastric (G) tube administration is not allowed