This phase I/II trial studies the side effects and best dose of cabazitaxel when given together with enzalutamide in treating patients with prostate cancer that has spread to other places in the body (metastatic) and has not responded to treatment with hormones or no longer responds to treatment with hormones (hormone-resistant). Drugs used in chemotherapy, such as cabazitaxel, work in different ways to stop the growth of tumor cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Androgen can cause the growth of prostate cancer cells. Hormone therapy using enzalutamide may fight prostate cancer by blocking the use of androgen by the tumor cells. Giving cabazitaxel together with enzalutamide may work better in treating metastatic, hormone-resistant prostate cancer.

PRIMARY OBJECTIVES:

I. To determine the safety and tolerability of combination treatment with enzalutamide and cabazitaxel (as determined by percent dose limiting toxicities [DLT], where DLT < 17% is consistent with it being a tolerable combination).

II. To determine the efficacy of treatment with the hormonal agent enzalutamide and the chemotherapy cabazitaxel in combination in men with metastatic castration-resistant prostate cancer (CRPC) (as determined by percent of patients achieving >= 90% prostate specific antigen [PSA] declines following initiation of treatment).

SECONDARY OBJECTIVES:

I. To further define the anticancer effect and safety profile of the combination of enzalutamide and cabazitaxel.

Ia. Collect toxicity data (description of adverse events). Ib. Determine PSA response (percent of patients who achieve >= 50% PSA decline and >= 30% PSA decline).

Ic. Examine pharmacokinetic (PK) data of cabazitaxel to characterize enzalutamide and cabazitaxel pharmacokinetic blood levels.

Id. Determine tumor response by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 for measurable disease and Prostate Cancer Working Group 2 criteria for non-measurable (bone) disease.

Ie. Determine overall survival.

EXPLORATORY OBJECTIVES:

I. To determine baseline (and at progression) biological tumor characteristics to evaluate for possible biomarkers indicative or predictive of response: apoptosis by cleaved caspase 3; androgen signaling axis (including but not limited to: androgen receptor expression, androgen receptor splice variants, and intratumoral androgen levels), and glucocorticoid receptor.

II. To collect circulating tumor cells (CTCs) and determine the degree to which tumor characteristics (delineated above) are shared by the CTCs.

III. To collect plasma and serum pre-treatment and at progression for assessment of circulating micro-ribonucleic acid (RNA)s and other circulating markers.

IV. To collect buffy coat to evaluate for steroid transporters.

OUTLINE: This is a dose de-escalation study of cabazitaxel.

Patients receive cabazitaxel intravenously (IV) over 1 hour on day 1 and enzalutamide orally (PO) once daily (QD) on days 1-21 (days 2-21 of cycle 1). Patients also receive prednisone PO twice daily (BID) as standard of care with cabazitaxel. Cycles repeat every 21 days for 6-10 cycles in the absence of disease progression or unacceptable toxicity. Patients may continue enzalutamide PO QD on days 1-28 in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up at 28 days and then every 6 months for up to 5 years.

Inclusion Criteria:

• Metastatic CRPC

• Willing to provide a tumor sample via biopsy from a metastatic site of disease to be collected at screening if safe and feasible per discretion of treating investigator; adequate archival metastatic tissue can be used, if available, in lieu of baseline biopsy if done when patient had CRPC; patients without a site amenable to biopsy and lack of archival tissue may still join the study

• Evidence of prostate cancer progression by any of the following criteria: radiographic or PSA criteria, or symptomatic progression related to prostate cancer

• Castrate testosterone levels (< 50 ng/dL) achieved by orchiectomy or maintenance on a luteinizing hormone-releasing hormone (LHRH) agonist or antagonist

• Histologic confirmation of original prostate cancer diagnosis per institutional standard; life expectancy of greater than 6 months

• Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1

• Leukocytes >= 3,000/mm^3

• Absolute neutrophil count >= 1,500/mm^3

• Platelets >= 100,000/mm^3

• Total bilirubin within normal institutional limits (or < 2 X the upper limit of normal in those with Gilbert's disease)

• Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase [SGOT])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 1.5 X institutional upper limit of normal

• Creatinine within less than the institutional upper limit of normal

• Creatinine clearance >= 45 mL/min/1.73 m^2 for patients with creatinine levels above institutional normal

• Subject agrees to use a double barrier method of birth control during the course of study treatment period with enzalutamide and/or cabazitaxel treatment and for at least 3 months after the study is discontinued

  • A double-barrier method of contraception involves the use of a condom in combination with 1 of the following: contraceptive sponge, diaphragm, or cervical ring with spermicidal gel or foam
  • Subject who has had a vasectomy at least 6 months prior to starting study treatment period and those whose female sexual partner(s) are more than 55 years of age and postmenopausal for at least 2 years or surgically sterile (tubal ligation, hysterectomy, or bilateral oophorectomy) agree to use at least a condom

• Ability to understand, and the willingness to sign, a written informed consent document, as well as comply with study requirements

• Must have appropriate wash out (> 6 half-lives) of androgen receptor antagonists, 5 alpha reductase inhibitors or ketoconazole prior to the start of cycle 1; if the agent is not in the table below, the washout should be 2 weeks

  • Bicalutamide; approximate half-life: 6 days; washout period required: 36 days
  • Flutamide; approximate half-life: 6 hours; washout period required: 36 hours
  • Nilutamide approximate half-life: 4 days; washout period required: 24 days
  • Finasteride; approximate half-life: 8 hours; washout period required: 48 hours
  • Aminoglutethimide; approximate half-life: 15 hours; washout period required: 4 days
  • Ketoconazole; approximate half-life: 8 hours; washout period required: 48 hours

Exclusion Criteria:

• Prior chemotherapy for mCRPC prostate cancer; chemotherapy given neoadjuvantly, adjuvantly, or for hormone sensitive metastatic disease is permitted as long as the cancer did not progress on chemotherapy AND > 6 months have elapsed
• Patients may not have received any other investigational agents within the last 14 days at the time of treatment start
• Patients may not have received enzalutamide or ARN-509 (another androgen receptor antagonist) in the past
• Patients may not have received cabazitaxel in the past
• Subject has clinical signs suggestive of high or imminent risks for pathological fracture, spinal cord compression and/or cauda equina syndrome
• History of severe hypersensitivity reaction (>= grade 3) to docetaxel, polysorbate 80 containing drugs, or any of the capsule components of enzalutamide, including Labrasol, butylated hydroxyanisole, and butylated hydroxytoluene
• Concurrent or planned treatment with strong inhibitors or strong inducers of cytochrome P450 family 3, subfamily A, polypeptide 4/5 (3A4/5); (a one-week wash-out period is necessary for patients who are already on these treatments)
• Uncontrolled, intercurrent illness including, but not limited to, ongoing or active infection, uncontrolled diabetes, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
• Subject has a history of seizure or any condition that may predispose to seizure including, but not limited to, underlying brain injury, stroke in the past 6 months, primary brain tumors, brain metastases, prior seizures
• Subject has a history of unexplained loss of consciousness or transient ischemic attack within 12 months of treatment start
• Subject is unwilling to stop using herbal supplements that can affect the PSA, such as saw palmetto or prostate cancer (PC)-SPES
• Subject has another active malignancy other than non-melanomatous skin cancer (unless it is metastatic) or superficial bladder cancer
• Must not have a gastrointestinal condition that would interfere with absorption
• Subjects may not be on other therapies that affect hormone levels, such as estrogens, testosterones, ketoconazole during this study; however, megestrol for hot flashes is permitted