Prospective Randomized Controlled Trial to Evaluate Locally Implemented Large Language Models (LLMs) for Simplifying Patient Communication in Hematology and Oncology
Prospective Randomized Controlled Trial to Evaluate Locally Implemented Large Language Models (LLMs) for Simplifying Patient Communication in Hematology and Oncology
This study tests whether patients with blood cancer or other cancers better understand their medical information when it is rewritten in plain language by an artificial intelligence (AI) system.
When patients are discharged from the hospital, they receive a medical letter summarizing their diagnosis, treatment, and next steps. These letters are often written in technical language that is difficult for patients to understand. In this study, an AI language model running on the hospital's own secure servers rewrites parts of this letter into simpler language. A physician checks the simplified version before the patient receives it.
Patients are randomly assigned to one of two groups. One group receives both the standard medical letter and the AI-simplified version. The other group receives the standard letter only. A separate group of patients who do not speak German well will receive a simplified and translated version.
After reading their letter, all participants fill out a short questionnaire about how well they understood the information. The study takes place at TUM University Hospital (Klinikum rechts der Isar) in Munich, Germany.
Background:
Studies show that up to 40-80% of medical information conveyed during physician consultations is not correctly recalled or understood by patients. This problem is particularly relevant in hematology and oncology, where treatment regimens, prognoses, and side-effect profiles are complex. Large language models (LLMs) have demonstrated the ability to convert medical texts into plain language with high accuracy. However, prospective randomized controlled trials evaluating the clinical benefit of LLM-simplified patient synopses in routine care are lacking.
Study Design:
Prospective, single-center, randomized controlled trial with parallel group design. Randomization is 2:1 (intervention : control) using permuted blocks of variable size (4-6). An additional non-randomized translation arm enrolls patients with insufficient German language proficiency.
Intervention:
The locally implemented LLM system (on-premise, no external data transmission) automatically simplifies the following sections of the discharge letter: Current Status, Medical History, Epicrisis, and Further Management. A study physician reviews and approves the simplified version before it is given to the patient. The system is not classified as a medical device and is not used for diagnosis or treatment decisions.
Endpoints:
The primary endpoint is a comprehension score measured by a 5-item scale (10-point Likert, based on PEMAT), assessing overall comprehension and comprehension of diagnosis, treatment, next steps, and medical terminology. Secondary endpoints include patient satisfaction (EORTC QLQ-INFO25 subscales), subjective uncertainty reduction, format preference, physician review time, correction rate, and translation quality.
Statistical Analysis:
The primary endpoint will be analyzed using a t-test or Mann-Whitney U-test. A clinically relevant difference of 1.5 points on the 10-point scale is assumed. With a standard deviation of 2.5, power of 80%, and alpha of 0.05 (two-sided), 136 randomized patients are required (91 intervention, 45 control). Accounting for a 10% dropout rate, 150 patients will be recruited for the randomized arms, plus 30 for the translation arm (total n=180).
Data Protection:
All data are pseudonymized and stored on secure hospital servers. No patient data are transmitted to external servers or cloud services. The study complies with GDPR.
Inclusion Criteria:
Exclusion Criteria:
krischan.braitsch@tum.de+49 089 4140 1268
lisa.adams@tum.de+49 089 4140 1084