RATIONALE: Radiation therapy uses high-energy x-rays to kill tumor cells. This may be an effective treatment for liver cancer.

PURPOSE: This phase I/II trial is studying the side effects and best dose of external-beam radiation therapy in treating patients with liver cancer that cannot be removed by surgery.

OBJECTIVES:

• To assess the feasibility and safety of radiotherapy (RT) in patients with hepatocellular carcinoma. (Phase I)
• To assess the safety and efficacy of RT in these patients. (Phase II)
• To generate reproducible peptide patterns of the serum proteome or specific serum sub proteomes in these patients.
• To assess changes in the proteome or sub proteome patterns after RT in these patients.
• To detect peptides that discriminate between before and after RT in these patients.
• To identify these discriminating peptides in these patients.

OUTLINE: This is a multicenter, phase I dose-escalation study followed by a phase II study.

Patients undergo radiotherapy (RT) once daily, five days a week, for 6 weeks. Intensity-modulated, 3-dimensional conformal, or fractionated stereotactic RT may be used.

After completion of study therapy, patients in the phase I portion are followed for 1 year and patients in the phase II portion are followed for 3 years.

DISEASE CHARACTERISTICS:

• Histologically, cytologically, or radiologically confirmed hepatocellular carcinoma

  • Clinical stage T2-4, N0-1, M0 (stage II, IIIA, IIIB, IIIC) OR unresectable T1, N0-1, M0 (stage I) disease

    • M1 disease allowed in phase I if at least 90% of the tumor load (volume) is in the liver

• Measurable disease (at least one liver lesion that can be measured in at least one dimension as ≥ 10 mm in multislice CT scan/MRI)

• Volumetry of liver tumor and residual liver tissue: residual liver volume (= total liver volume - gross tumor volume) has to be ≥ 800 mL and ≥ 40% of total liver volume

• No operable disease (with curative intent or planned liver transplantation)

• No presence of clinical ascites

PATIENT CHARACTERISTICS:

• WHO performance status 0-2

• Cirrhosis Child-Pugh class A or B (Child-Pugh score of ≤ 9)

• Hemoglobin ≥ 100 g/L

• ANC ≥ 1,200/mm³

• Platelet count ≥ 50,000/mm³

• ALT and AST ≤ 7 times upper limit of normal (ULN)

• AP ≤ 10 times ULN

• Bilirubin ≤ 50 μmol/L

• INR ≤ 2

• Creatinine clearance ≥ 50 mL/min

• Functional left kidney (scintigraphy mandatory for phase I, phase II only if indicated)

• Lipase ≤ 2 times ULN (phase I only)

• Able to tolerate proton-pump inhibitors or H2 antagonists during radiation therapy

• Not pregnant

• Negative pregnancy test

• Fertile patients must use effective contraception during and for 4 months after completion of study therapy

• No prior malignancy allowed, except for the following:

  • Adequately treated cervical carcinoma in situ
  • Adequately treated localized nonmelanoma skin cancer
  • Any other malignancy from which patient has been disease-free for 5 years

• No presence of medically uncontrolled encephalopathy

• No myocardial infarction within the past 6 months

• No esophageal varices ≥ grade 3, with red signs, or bleeding within the past 3 months

• No symptoms of colitis, enteritis, esophagitis, fistula, gastritis, ileus, necrosis, perforation, stricture, or ulcer

• No severe anorexia, constipation, dehydration, diarrhea, or vomiting

• No serious underlying medical condition that, in the opinion in the investigator, would preclude study participation (e.g., active autoimmune disease or uncontrolled diabetes)

  • Portal vein thrombosis allowed

• No psychiatric disorder precluding understanding of information on study related topics or giving informed consent

• No nutritional intake < 1500 calories per day (corrected)

• No weight loss ≥ 15 % within the past 3 months

PRIOR CONCURRENT THERAPY:

• At least 8 weeks since prior transarterial chemoembolization (TACE), radiofrequency ablation, or radiotherapy (RT) unless progressive disease was documented after this therapy

• At least 21 days since prior and no other concurrent treatment with experimental drugs

• At least 21 days since prior and no other concurrent treatment on another clinical trial

• At least 21 days since prior and no other concurrent anticancer therapy

• No prior RT to the abdomen or caudal chest

  • Prior RT to pelvis allowed
  • Prior RT to chest must be above D5 vertebra
  • Portal vein embolization ligation or pre-RT TACE allowed

• No concurrent treatment with steroids or non-steroidal anti-inflammatory drugs during RT (proton-pump inhibitor allowed)