This phase I trial studies the side effects and best dose of veliparib in treating patients with malignant solid tumors that do not respond to previous therapy. Veliparib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.

PRIMARY OBJECTIVES:

I. To establish the maximum tolerated dose (MTD), dose-limiting toxicities (DLT), and recommended phase II dose of chronically dosed single-agent ABT-888 (veliparib) in patients with either a refractory breast cancer (BRCA) 1/2- mutated solid cancer; platinum-refractory ovarian, fallopian tube, or primary peritoneal cancer; or basal-like breast cancer.

SECONDARY OBJECTIVES:

I. To establish the safety and tolerability of single-agent ABT-888 in the above patient population. A dose expansion at the recommended phase II dose will be performed in 6-12 evaluable patients with germline BRCA mutations.

II. To determine the effects of ABT-888 treatment on the level of poly ADP-ribose polymerase (PARP) inhibition and deoxyribonucleic acid (DNA) damage in peripheral blood mononuclear cells (PBMCs) and tumor samples or cells in malignant ascitic fluid.

III. To determine the pharmacokinetics (PK) of chronically dosed ABT-888. IV. To document any evidence of anti-tumor response.

OUTLINE: This is a dose-escalation study.

Patients receive veliparib orally (PO) twice daily (BID)* on days 1-28. Courses repeat every 28 days in the absence of disease progression or unacceptable toxicity.

NOTE: *Patients receive veliparib once on day 1 of course 1 for pharmacokinetic and pharmacodynamic studies.

After completion of study therapy, patients are followed for 4 weeks.

Inclusion Criteria:

• Patients must have histologically or cytologically confirmed solid tumors that fulfill at least one of the following 3 criteria:

  • Have a documented BRCA1/2 mutation and a BRCA related malignancy (primarily breast or ovarian cancers, but also may include prostate or pancreatic cancers); NOTE: Patients enrolled under the Dose Expansion Phase must have a documented BRCA 1/2 mutation; or
  • Platinum-refractory ovarian, fallopian tube, or primary peritoneal cancer or
  • Basal-like breast cancer whose disease has progressed following standard therapy or who have no acceptable standard treatment options

• All patients without a known, documented BRCA mutation from Myriad Genetic Laboratories must have a probability of harboring a BRCA gene mutation assessed by BRCAPRO computer program

  • All patients in whom the probability of having a genetic mutation is >= 20% must have formal BRCA testing done through Myriad Genetic Laboratories in order to participate in the study
  • Although various research based tests have been developed to detect BRCA mutations, due to the fact that these are not Food and Drug Administration (FDA) or Clinical Laboratory Improvement Amendments (CLIA) approved and therefore not reportable to patients, if a patient has diagnosis of a BRAC mutation based on a non-Myriad test, then they must undergo Myriad BRCA gene sequencing to be eligible
  • Patients are eligible whether they have a known deleterious BRCA 1 or 2 mutation or a mutation of uncertain significance
  • If a patient refuses BRCA testing, then they are ineligible for the study

• Platinum-refractory is defined as progression or recurrence within 6 months of initial platinum response; platinum-resistant is defined as having no prior response to platinum (i.e. evidence of progression within 2-3 cycles of beginning initial platinum-based treatment) and platinum-resistant patients are excluded; the only platinum-sensitive patients that are eligible are those with known BRCA mutations

• Basal-like breast cancer will be defined as estrogen and progesterone receptor negative, human epidermal growth factor receptor 2 (HER2) negative, and/or having expression profile of epidermal growth factor receptor (EGFR) and cytokeratins 5/6, consistent with basal phenotype; breast cancer patients with "triple-negative" phenotype (negative hormone and HER2 receptors) are eligible to participate in this trial; patients who are only known to be "triple-negative" but unknown basal phenotype will have their tumor blocks assessed for basal markers

• For subjects enrolled under the Dose Escalation Phase: Enrolled patients without a known BRCA mutation must have archived tumor tissue available for assessment of BRCA 1/2 protein expression by immunohistochemistry, as well as other correlative studies; it is optional for patients with a known BRCA mutation to provide archived tissue for correlative studies

• For subjects enrolled under the Dose Expansion Phase: All patients enrolled during the Dose Expansion Phase (for which a tissue biopsy is mandatory) must have a known BRCA mutation and must agree to collection or archival tumor tissue, if available

• There are no limitations on the amount of prior therapies received; however, no major surgery, radiation or chemotherapy within four weeks prior to study enrollment except for mitomycin C and nitrosoureas, in which case it is 6 weeks; patients must be recovered from toxicities of prior therapies to at least eligibility levels

• Eastern Cooperative Oncology Group (ECOG) performance status < 2 (Karnofsky > 60%)

• Life expectancy of greater than 3 months

• Transaminases =< 2.5 times upper limit of normal (ULN)

• Bilirubin =< 2.0 mg/dL

• Creatinine =< ULN or a creatinine clearance > 50 ml/minute (calculated by Cockcroft-Gault formula) if creatinine > ULN

• Neutrophils >= 1500/uL

• Platelets >= 100,000/uL

• Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation; should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately

• Ability to understand and the willingness to sign a written informed consent document

• Ability to swallow pills

• Patients with central nervous system (CNS) metastases must be stable after therapy for CNS metastases (such as surgery, radiotherapy or stereotactic radiosurgery) for > 3 months and must be off steroid treatment prior to study enrollment and must have a life expectancy of 3 months or greater to be eligible

• Patients with BRCA mutations who are enrolled in the 6-12 patient expansion group at dose level VIII (400 mg BID) must agree to tumor biopsies; therefore patients enrolled in this cohort should have tumors easily accessible for biopsies with low likelihood of complication and these patients should not be on therapeutic doses of anticoagulation

• Patients with BRCA mutations who are enrolled in the 6-12 patient expansion group at dose level VIII (400 mg BID) must agree to collection of archival tissue, if available; if not available, patient may still be enrolled as long as the patient consents to the mandatory fresh tumor tissue biopsies

Exclusion Criteria:

• Patients who have had chemotherapy, hormone therapy or radiotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier

• Patients may not be receiving any other investigational agents

  • Patients with prostate cancer must continue ongoing luteinizing-hormone-releasing hormone (LhRH) agonist therapy and discontinue antiandrogens at least 6 weeks (for bicalutamide) or 4 weeks (flutamide) prior to study entry; patients with bone metastases or hypercalcemia who began intravenous bisphosphonate treatment prior to study entry may continue this treatment

• Patients with CNS metastases must be stable after therapy for CNS metastases (such as surgery, radiotherapy or stereotactic radiosurgery) for > 3 months and must be off steroid treatment prior to study enrollment and must have a life expectancy secondary to that of 3 months or greater to be eligible

• Uncontrolled intercurrent illness including, but not limited to ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements

• Human immunodeficiency virus (HIV) infected patients on protease inhibitors are ineligible; HIV infected patients with adequate cluster of differentiation 4 (CD4) counts (> 500) and not on protease inhibitors are eligible

• Pregnant women are excluded; breastfeeding should be discontinued if the mother is treated with ABT-888

• Active seizure or history of seizure disorder