Participants log into the study web application using a unique ID and their name. For each simulated patient case, they press "Start" to unlock the case record and initiate timing. After viewing the patient information, clinicians use CocktailAI to generate a first draft of the discharge summary. They then review and edit the draft and submit the final document by pressing "Submit." For the discharge referral of the same case, clinicians follow the same procedure. Copy-and-paste from the case record is permitted. This sequence is repeated for six simulated cases.

Participants log into the study web application using a unique ID and their name. For each simulated patient case, they press "Start" to unlock the case record and initiate timing. After viewing the patient information, clinicians write the discharge summary from scratch and submit the document by pressing "Submit." They then write the discharge referral for the same case from scratch and submit it. Copy-and-paste from the case record is permitted. This sequence is repeated for six simulated cases.

The LLM assistant generates drafts of discharge summaries and discharge referrals directly from the simulated patient records, with no clinician review or edits. These drafts are the unedited first drafts produced in the CocktailAI & Modification arm (captured before any clinician edits) and are included as a separate group.

This study uses CocktailAI, a template-based LLM assistant co-developed by the Department of Ophthalmology and Visual Sciences, Kyoto University Graduate School of Medicine, and Fitting Cloud Inc. (Kyoto, Japan). It is designed to extract relevant information from EHRs using LLMs and embed the extracted content into predefined templates. In this trial, the inputs are six simulated patient records (no real patient data). Text generation uses Gemini-2.0-flash-lite. Templates for discharge summaries and discharge referrals are pre-defined by a team member.

In the CocktailAI & Modification group and the Control group, the time spent creating documents is measured in seconds. In the CocktailAI group, the time required for document generation is measured in seconds.

Blinded to group allocation, ophthalmology experts evaluate the documents using pre-specified criteria defined before study initiation. These criteria are developed based on six domains: Medical Accuracy, Language, Conciseness, Presence of Hallucinations, Validity for Clinical Use, and Possibility of Harm. Most domains are assessed on a three-point scale, whereas Presence of Hallucinations is evaluated dichotomously (present or absent). In addition to these domain-specific ratings, experts provide a subjective overall score on a 10-point scale (with higher scores indicating better quality) and are asked to guess which study group the document belongs to.