This phase I trial is studying the side effects and best dose of vorinostat when given together with sorafenib tosylate in treating patients with advanced liver cancer. Sorafenib tosylate and vorinostat may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth or by blocking blood flow to the tumor.

Outline: This is a dose-escalation study of vorinostat. The purpose of this research study is to test the safety and effectiveness of a combination of two cancer drugs, sorafenib (Nexavar) and vorinostat (Zolinza), in advanced liver cancer (hepatocellular carcinoma). Advanced means that the cancer has spread too far to consider surgery. Approximately 19 people will take part in this study.

After enrollment of 6 patients at sorafenib 400 mg orally twice a day and vorinostat 300 mg orally, only 2 of the 6 patients were evaluable for DLT (no DLTs). The other 4 patients were not evaluable for DLT because of required dose modifications. Because of this dose modification need, Cohort A has been modified to include 2 dose levels: Dose level A-1a (sorafenib 400 mg orally twice a day and vorinostat 200 mg orally once a day) and dose level A1 (sorafenib 400 mg orally twice a day with vorinostat 100 mg orally once a day). The starting dose upon reopening after approval of this version will be dose level A-1a. Dose level A1 will only be used if dose level A-1a is not tolerable.

Inclusion Criteria:

• Diagnosis of HCC by biopsy-proven pathologic diagnosis or by clinical criteria as defined below:

  * Clinical criteria to be met if patient has a history of cirrhosis or chronic hepatitis B infection:

  • Imaging abnormalities > 1 cm in size with classic enhancement by magnetic resonance imaging (MRI) or triple-phase computed tomography (CT) scan
  • Alpha-fetoprotein (AFP) of any value

• Performance status Eastern Cooperative Oncology Group (ECOG) =< 1

• If cirrhosis, Child-Pugh classification A or B

• Total bilirubin =< 3.0 mg/dL

• Creatinine =< 1.5 x upper limit of normal for the laboratory

• International normalized ratio (INR) =< 1.7 (if not due to anticoagulants)

• Absolute neutrophil count (ANC) >= 1,500/mm^3

• Platelets >= 80,000/mm^3

• Hemoglobin (Hgb) >= 8.5 g/dL (transfusion or erythropoietin-like substances not permitted prior to baseline evaluation)

• Any prior therapies such as surgery, chemoembolization, radiofrequency ablation, and alcohol injection are allowed as long as toxicity from such prior therapy is =< grade 1

• Prior sorafenib is allowed as long as toxicity from ongoing is ≤ grade 2 and prior intolerance of 400 mg sorafenib PO daily is felt amenable, by the principal investigator, to supportive care measures or dose modifications.

• Measurable or evaluable disease by Response Evaluation Criteria in Solid Tumors (RECIST) criteria version (v) 1.1 mRECIST or elevated AFP

• Ability to understand and willingness to sign a written informed consent; a signed informed consent must be obtained prior to any study specific procedures

• Women of childbearing potential must have a negative pregnancy test performed within 2 weeks prior to the start of treatment

• Women of childbearing potential and men must agree to use a medically accepted form of birth control for the duration of study participation and for 4 months following completion of study treatment

Exclusion Criteria:

• Candidate for curative therapy including surgical resection or orthotopic liver transplantation

• Known central nervous system metastasis

• Any investigational agent within 4 weeks of first dose of study treatment

• Known intolerance of vorinostat

• Unable to swallow medication

• Unable to swallow medication; suspected malabsorption

• Active alcohol abuse

• Contraindication to antiangiogenic agents, including:

  • Pulmonary hemorrhage/bleeding event >= grade 2 within 4 weeks of first dose of study drug
  • Any other hemorrhage/bleeding event >= grade 3 within 4 weeks of first dose of study treatment
  • Serious non-healing wound, ulcer, or bone fracture

• Thrombolic or embolic events such as a cerebrovascular accident including transient ischemic attacks within the past 6 months; hepatic portal vein thrombus is not considered an exclusion criterion

• Major cardiac dysfunction, such as uncontrolled angina, congestive heart failure with New York Heart Association (NYHA) class III or higher, known left ventricular ejection fraction less than 40%

• Systolic blood pressure > 160 mmHg or diastolic pressure > 90 mmHg despite optimal medical management

• Significant lung disease (oxygen [O2] saturation less than 88% in room air)

• Serious uncontrolled infection; known human immunodeficiency virus (HIV)-seropositivity requiring retroviral therapy, or diagnosis of acquired immune deficiency syndrome (AIDS); diagnosis of chronic hepatitis B or C allowed

• Medical, psychological, or social conditions that, in the opinion of the investigator, may increase the patient's risk or interfere with the patient's participation in the study or hinder evaluation of the study results