This randomized phase III trial is studying two different combination chemotherapy regimens to compare how well they work when given together with radiation therapy in treating patients with newly diagnosed rhabdomyosarcoma. Drugs used in chemotherapy, such as vincristine sulfate, dactinomycin, cyclophosphamide, and irinotecan hydrochloride, work in different ways to stop the growth of tumor cells, either by killing the cells or by stopping them from dividing. Radiation therapy uses high-energy x-rays to kill tumor cells. Giving combination chemotherapy together with radiation therapy may kill more tumor cells. It is not yet known which combination chemotherapy regimen is more effective when given together with radiation therapy in treating patients with rhabdomyosarcoma.

PRIMARY OBJECTIVES:

I. To compare the early response rates, failure-free survival (FFS), and survival of patients with intermediate-risk rhabdomyosarcoma (RMS) treated with surgery, radiotherapy, and vincristine (vincristine sulfate), dactinomycin and cyclophosphamide (VAC) or VAC alternating with vincristine, irinotecan (irinotecan hydrochloride) (VI).

SECONDARY OBJECTIVES:

I. To compare FFS, local control, and survival of patients with intermediate-risk RMS treated with VAC and early (week 4) radiotherapy vs delayed (week 10) radiotherapy, using data from Intergroup Rhabdomyosarcoma Study (IRS)-IV for historic comparison.

II. To compare the acute and late effects of VAC to VAC alternating with VI, including the toxicity associated with concurrent VI and radiotherapy.

III. To compare the acute and late effects of VAC as delivered on this study to D9803 VAC.

IV. To correlate change in fludeoxyglucose F-18 positron emission tomography (FDG-PET) maximum standard uptake value (SUVmax) from week 1 to week 4 and 15 with FFS.

V. For VI treated patients, to correlate patient UDP glucuronosyltransferase 1 family, polypeptide A1 (UGT1A1) genotype with VI toxicity. VI. To correlate cytochrome P450, family 2, subfamily B, polypeptide 6 (CYP2B6), cytochrome P450, family 2, subfamily C, polypeptide 9 (CYP2C9), and glutathione S-transferase alpha 1 (GSTA1) genotypes with VAC toxicity.

VII. To prospectively evaluate and validate gene expression values with the intent to define the best diagnostic predictors and more powerful prognostic classifiers.

VIII. To assess the frequency of bladder dysfunction in patients with bladder, prostate, and pelvic sites of RMS 3-6 years after study enrollment.

OUTLINE: Patients are randomized to 1 of 2 treatment arms within 42 days of initial surgery or biopsy.

ARM I (VAC): Patients receive VAC chemotherapy comprising vincristine sulfate IV over 1 minute on day 1 of weeks 1-13, 16, 19-25, 28, 31-37, and 40; dactinomycin IV over 1-5 minutes on day 1 of weeks 1, 4, 13, 16, 19, 22, 25, 28, 31, 34, 37, and 40; and cyclophosphamide IV over 1 hour on day 1 of weeks 1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, and 40.

ARM II (VAC/VI): Patients receive VAC chemotherapy alternating with VI chemotherapy comprising vincristine sulfate IV over 1 minute on day 1 of weeks 1-13,16, 17, 19, 20, 22-26, 28, 31-34, 37, 38, and 40; dactinomycin IV over 1-5 minutes on day 1 of weeks 1,13, 22, 28, 34, and 40; cyclophosphamide IV over 1 hour on day 1 of weeks 1,10, 13, 22, 28, 34, and 40; and irinotecan hydrochloride IV over 90 minutes on days 1-5 of weeks 4, 7, 16, 19, 25, 31, and 37.

In both arms, treatment continues in the absence of disease progression or unacceptable toxicity. Patients* in both arms also undergo radiotherapy 5 days a week for 4-6 weeks beginning in week 4 (except patients with alveolar RMS rendered group I by amputation OR patients needing week 1 emergency radiotherapy for symptomatic spinal cord compression).

NOTE: *Individualized local control plan that deviates from protocol-mandated radiotherapy allowed for patients =< 24 months of age.

After completion of study treatment, patients are followed up every 2-4 months for 4 years and then annually for 5-10 years.

Inclusion Criteria:

• Patients with newly diagnosed embryonal RMS, botryoid or spindle cell variants of embryonal RMS, ectomesenchymoma, or alveolar RMS are eligible for this study

• Enrollment on COG-D9902 to confirm local histologic diagnosis with central pathology review is required for all patients

• Patients may be enrolled on ARST0531 and start protocol treatment prior to receipt of central pathology review results

• Patient must have Intermediate-risk RMS defined as:

  • Embryonal, botryoid, or spindle cell RMS, or ectomesenchymoma: stage 2 or 3 and group III OR
  • Alveolar RMS: stage 1-3 and group I-III

• Staging ipsilateral retroperitoneal lymph node dissection (SIRLND) is required for all patients >= 10 years of age with paratesticular tumors and for patients < 10 years with clinically or radiographically involved lymph nodes (except when extensive lymph node involvement, defined as two or more lymph nodes > 2 cm in dimension, is identified by imaging studies)

• Regional lymph node sampling or sentinel lymph node procedure is required for histologic evaluation in patients with extremity tumors

• Clinically or radiographically enlarged nodes should be sampled for histologic evaluation

• Detection of metastasis by optional FDG PET (not required for study enrollment); FDG PET may detect abnormalities suggestive of metastasis not identified by bone scan, computed tomography (CT), or bone marrow aspiration/biopsy; the prognostic significance of FDG PET-detected abnormalities is not clear; FDG PET-detected abnormalities MUST be confirmed to be metastases by an additional imaging modality (such as magnetic resonance imaging [MRI] or CT) OR pathologic confirmation; unless FDG PET abnormalities are confirmed by another imaging modality or biopsy, FDG PET abnormalities will NOT be considered evidence of metastasis

• Patients must have a performance status of 0, 1, or 2; the Lansky performance score should be used for patients < 16 years and the Karnofsky performance score for patients >= 16 years

• Patients who have received prior chemotherapy (excluding steroids) or radiation therapy, except for patients transferring from ARST0331 (low-risk study), are not eligible

• Creatinine clearance or radioisotope glomerular filtration rate (GFR) >= 70 ml/min/1.73 m^2 or a serum creatinine based on age/gender as follows:

  • 1 month to < 6 months: 0.4 mg/dL
  • 6 months to < 1 year: 0.5 mg/dL
  • 1 to < 2 years: 0.6 mg/dL
  • 2 to < 6 years: 0.8 mgt/dL
  • 6 to < 10 years: 1 mg/dL
  • 10 to < 13 years: 1.2 mg/dL
  • 13 to < 16 years: 1.5 mg/dL (males) or 1.4 mg/dL (females)
  • >= 16 years: 1.7 mg/dL (males) or 1.4 mg/dL (females)

• Patients with urinary tract obstruction by tumor must meet the renal function criteria AND must have unimpeded urinary flow established via decompression of the obstructed portion of the urinary tract

• Total bilirubin =< 1.5 x upper limit of normal for age

• Peripheral absolute neutrophil count (ANC) >= 750/uL

• Platelet count >= 75,000/uL (transfusion independent)

• No evidence of uncontrolled infection

• Patients must be able to undergo radiation therapy, if necessary, as specified in the protocol

• Female patients of childbearing potential must have a negative pregnancy test

• Female patients who are breast feeding must agree to stop breast feeding

• Sexually active patients of childbearing potential must be willing to use effective contraception during therapy and for at least 1 month after treatment is completed

• All patients and/or their parents or legal guardians must sign a written informed consent

• All institutional, Food and Drug Administration (FDA), and National Cancer Institute (NCI) requirements for human studies must be met