RATIONALE: Giving chemotherapy before a peripheral stem cell transplant stops the growth of tumor cells by stopping them from dividing or killing them. Giving colony-stimulating factors, such as G-CSF, and certain chemotherapy drugs, helps stem cells move from the bone marrow to the blood so they can be collected and stored. More chemotherapy is then given to prepare the bone marrow for the stem cell transplant. The stem cells are then returned to the patient to replace the blood-forming cells that were destroyed by the chemotherapy.

PURPOSE: This phase I trial is studying the side effects and best dose of topotecan when given together with cyclophosphamide, paclitaxel, melphalan, and cisplatin, followed by an autologous peripheral stem cell transplant in treating patients with stage III, stage IV, or recurrent ovarian epithelial cancer, primary peritoneal cancer, or fallopian tube cancer.

OBJECTIVES:

• To establish the maximum tolerated dose (MTD) of continuous infusion intravenous topotecan hydrochloride when administered with intraperitoneal (IP) cisplatin and intravenous melphalan in patients with stage III, stage IV, or recurrent ovarian epithelial cancer, primary peritoneal cancer, or fallopian tube cancer.
• To describe the toxicities of each dose studied.
• To evaluate the pharmacokinetics of topotecan hydrochloride when administered at the maximum tolerated dose and cisplatin.
• To confirm the pharmacokinetic advantage of high-dose IP cisplatin and IP paclitaxel.
• To obtain tissue at the time of peritoneal catheter placement in order to evaluate the molecular determinants of apoptosis (including p53 status, p21 gene expression, bcl-2 gene expression, and bcl-2/bax ratio) and the extent of apoptosis by the TdT assay.
• To evaluate the molecular determinants of DNA damage and repair, including expression levels of ERCC1 and MDR1, and HER2/neu expression by immunohistochemistry.

OUTLINE: This is a dose-escalation study of topotecan hydrochloride.

Patients undergo surgical placement of an intraperitoneal (IP) catheter. Tumor biopsies are obtained during surgery for laboratory analysis of molecular determinants of apoptosis (including p53 status, p21 gene expression, bcl-2 gene expression, bcl-2/bax ratio) and molecular determinants of DNA damage and repair (including expression levels of ERCC1 and MDR1, and HER2/neu expression by immunohistochemistry). The extent of apoptosis is also assessed using the TdT assay.

• Course 1: Patients receive paclitaxel IP on day 1, cyclophosphamide IV on day 2, and filgrastim (G-CSF) subcutaneously (SC) beginning on day 3 and continuing until apheresis is completed. Patients undergo apheresis until ≥ 2.5 X 10^6 CD34-positive cells/kg are collected. Two weeks later, patients proceed to course 2.
• Course 2: Patients receive cisplatin IP and melphalan IV on days -11 and -4 and topotecan hydrochloride by continuous infusion over 120 hours on days -10 to -6. Patients receive 25% of their peripheral blood stem cells (PBSCs) on day -3 and G-CSF IV beginning on day -3 and continuing until blood counts recover. Patients receive their remaining PBSCs on day 0.

Patients undergo daily blood sample collection during topotecan hydrochloride administration for pharmacokinetic studies. Patients treated at the maximum tolerated dose of topotecan hydrochloride undergo additional blood sample collections for pharmacokinetic studies.

After completion of study therapy, patients are followed every 3 months.

DISEASE CHARACTERISTICS:

• Histologically confirmed ovarian epithelial carcinoma, primary peritoneal cavity carcinoma, or epithelial carcinoma of the fallopian tubes, meeting 1 of the following criteria:

  • Stage III or IV disease that was treated with initial therapy comprising a standard platinum-containing regimen

    • Must have < 2 cm of residual disease with no evidence of disease progression after initial chemotherapy AND have no disease progression immediately prior to stem cell collection
    • Patients initially presenting with stage IV disease who have achieved a clinical response (complete response [CR] or partial response [PR]) after initial therapy are eligible

  • Responding recurrent disease

    • Patients who have had recurrence with elevated CA 125 levels (> 100 U/mL) and who have achieved a reduction of CA 125 level by 50% for 4 weeks following the most recent course of reinduction chemotherapy are eligible
    • Patients who have achieved a CR or PR after salvage chemotherapy for relapsed disease are eligible

• Patients with measurable or evaluable disease must have achieved a PR after prior therapy

• No clinically significant pleural effusions

PATIENT CHARACTERISTICS:

• Karnofsky performance status 70-100%
• ANC > 1,000/μL
• Platelet count > 100,000/μL
• Serum bilirubin < 1.5 mg/dL
• SGOT and SGPT ≤ 2.5 times normal
• Creatinine clearance ≥ 60 mL/min
• No active cardiac disease that, in the opinion of the investigator, would preclude safe administration of chemotherapy
• Cardiac ejection fraction normal at rest by MUGA
• No history of potentially disabling psychiatric disorders
• Hepatitis B antigen, hepatitis C antibody, and HIV antibody negative
• No clinically significant peripheral neuropathy
• FEV_1 ≥ 2.0 L or ≥ 75% of the lower limit of normal

PRIOR CONCURRENT THERAPY:

• See Disease Characteristics
• At least 4 weeks since prior chemotherapy or radiotherapy
• No prior radiotherapy to the whole abdomen