This phase I trial is studying the side effects and best dose of vorinostat when given together with isotretinoin in treating young patients with recurrent or refractory solid tumors, lymphoma, or leukemia. Drugs used in chemotherapy, such as vorinostat, work in different ways to stop the growth of cancer cells, either by killing the cells or by stopping them from dividing. Vorinostat may also stop the growth of cancer cells by blocking some of the enzymes needed for cell growth and by blocking blood flow to the cancer. Isotretinoin may cause cancer cells to look more like normal cells, and to grow and spread more slowly. Giving vorinostat together with isotretinoin may be an effective treatment for cancer.

PRIMARY OBJECTIVES:

I. Determine the maximum tolerated dose (MTD) of vorinostat (SAHA) in young patients with recurrent or refractory solid tumors or lymphomas.

II. Determine the MTD of SAHA administered in combination with isotretinoin in young patients with recurrent or refractory neuroblastoma, medulloblastoma/CNS primitive neuroectodermal tumor, or atypical teratoid rhabdoid tumor.

III. Determine the tolerability of the solid tumor MTD of SAHA in young patients with recurrent or refractory leukemia.

IV. Determine the toxic effects of SAHA administered with or without isotretinoin in these patients.

V. Determine the pharmacokinetics of this drug in these patients.

SECONDARY OBJECTIVES:

I. Determine, preliminarily, the antitumor activity of SAHA administered with or without isotretinoin in these patients.

II. Correlate the pharmacokinetics of this drug with genetic polymorphisms (e.g., UGT1A1) in these patients.

OUTLINE: This is a multicenter, dose-escalation study of vorinostat (SAHA).

Group 1 (solid tumor or lymphoma patients): Patients receive oral SAHA once daily on days 1-28. Treatment repeats every 28 days for up to 12 courses in the absence of disease progression or unacceptable toxicity.Cohorts of 3-6 patients receive escalating doses of SAHA until the maximum tolerated dose (MTD) is determined. The MTD is defined as the dose preceding that at which 2 of 3 or 2 of 6 patients experience dose-limiting toxicity. An additional 6 patients may be treated at the MTD.

Group 2 (leukemia patients): Patients receive SAHA as in group 1 at the MTD.

Group 3 (select solid tumor patients): Patients receive oral isotretinoin twice daily on days 1-14. Patients also receive SAHA once daily on days 1-28 OR once on days 1, 3, 5, 8, 10, 12, 15, 17, 19, 22, 24, and 26. Treatment repeats every 28 days for up to 12 courses in the absence of disease progression or unacceptable toxicity.The MTD of SAHA is determined as in group 1. An additional 6 patients may be treated at the MTD.

After completion of study treatment, patients are followed periodically.

Inclusion Criteria:

• Histologically confirmed* diagnosis of 1 of the following:

  • Recurrent or refractory solid tumor or lymphoma (for patients in group 1)

    • Measurable or evaluable disease

  • Recurrent or refractory leukemia (for patients in group 2)

    • Greater than 25% blasts in the bone marrow (i.e., M3 bone marrow)
    • Active extramedullary disease allowed except leptomeningeal disease

  • Recurrent or refractory CNS tumor of 1 of the following types (for patients in group 3):

    • Neuroblastoma
    • Medulloblastoma/CNS primitive neuroectodermal tumor
    • Atypical teratoid rhabdoid tumor

• No known curative therapy or therapy proven to prolong survival with an acceptable quality of life exists

• No bone marrow involvement by disease (for patients in groups 1 and 3)

• No active CNS leukemia

• Performance status - Lansky 50-100% (for patients ≤ 10 years of age)

• Performance status - Karnofsky 60-100% (for patients > 10 years of age)

• Absolute neutrophil count ≥ 1,000/mm^3 (for solid tumor patients)

• Platelet count ≥ 100,000/mm^3* (for solid tumor patients) (20,000/mm^3** for leukemia patients)

• Hemoglobin ≥ 8.0 g/dL (RBC transfusion allowed) (for solid tumor and leukemia patients)

• Triglycerides < 300 mg/dL (for patients in group 3)

• Bilirubin ≤ 1.5 times upper limit of normal (ULN)

• ALT ≤ 5 times ULN

• Albumin ≥ 2 g/dL

• Creatinine clearance or radioisotope glomerular filtration rate ≥ 70 mL/min

• Creatinine based on age as follows:

  • No greater than 0.8 mg/dL (for patients ≤ 5 years of age)
  • No greater than 1.0 mg/dL (for patients 6 to 10 years of age)
  • No greater than 1.2 mg/dL (for patients 11 to 15 years of age)
  • No greater than 1.5 mg/dL (for patients over 15 years of age)

• Negative dipstick for protein OR < 1,000 mg protein/24 hour urine collection (for patients in group 3)

• No evidence of gross hematuria (for patients in group 3)

• Not pregnant or nursing

• Negative pregnancy test

• Fertile patients must use effective contraception

• Body surface area ≥ 0.5 m^2

• Neurologic deficits in patients with CNS tumors must be stable for ≥ 1 week before study entry

• Able to swallow whole capsules

• No uncontrolled infection

• Skin toxicity < grade 1 (for patients in group 3)

• Recovered from prior immunotherapy

• At least 7 days since prior hematopoietic growth factors

• At least 7 days since prior antineoplastic biologic agents

• At least 2 months since prior stem cell transplantation or rescue

  • No evidence of active graft-versus-host disease

• No other concurrent biologic therapy or immunotherapy

• More than 3 weeks since prior myelosuppressive chemotherapy (6 weeks for nitrosoureas) and recovered

• No concurrent chemotherapy

• Patients with CNS tumors must be on a stable or decreasing dose of dexamethasone for ≥ 7 days prior to study entry

• No concurrent dexamethasone for antinausea or antiemetic therapy

• Recovered from prior radiotherapy

• At least 2 weeks since prior local, palliative, small-port radiotherapy

• At least 3 months since prior total-body irradiation, radiotherapy to the craniospinal area, or radiotherapy to ≥ 50% of the pelvis

• At least 6 weeks since other prior substantial radiotherapy to the bone marrow

• No concurrent radiotherapy

• At least 2 weeks since prior valproic acid

• No other concurrent investigational agents

• No other concurrent anticancer therapy

• No concurrent enzyme-inducing anticonvulsants