Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
The goal of this study is to test the accuracy of Large Language Model-generated serious illness communication (SIC) summaries, the feasibility of delivering the SIC summaries, and to collect perspectives on the SIC summaries from clinicians and participants with cancer. Large Language Models (LLMs) are artificial intelligence programs that can perform various natural language processing tasks.
The study will enroll up to 70 Dana-Farber patients admitted to Brigham and Women's hospital who have an elevated mortality prediction at admission. The research study procedures include randomization to intervention or control arms on admission (3:1 intervention to control). For patients in the intervention arm, a large language model query will summarize serious illness communication documentation in their medical record screening from the prior 6 months. A summary of the documentation will be sent to the inpatient and outpatient oncology clinicians within 24 hours of admission; these clinicians will be asked to review the summaries, discuss with the patient and incorporate into care plans as appropriate. For patients with no SIC documented on day 4 of the admission, a second email will be sent. Control patients will receive usual care, which is no email sent to their oncology teams. All patients will be offered to participate in an interview on day 4 of their admission about their communication with their care teams. For patients in both arms, inpatient and outpatient oncology attendings will be sent an email survey regarding SIC for the patient and, if applicable, the utility of the SIC summaries.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Group A: LLM-Generated SIC Summary Email | Experimental | 45 participants will be randomized. Participants will have the option of completing a one-time in-person interview with study staff on their perceptions of SIC and the acceptability of LLM-generated summaries of SIC. |
|
| Group B: Usual Care | No Intervention | 15 participants will be randomized and will receive standard oncology care. Participants will have the option of completing a one-time in-person interview with study staff on their perceptions of SIC and the acceptability of LLM-generated summaries of SIC. |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| LLM SIC Summary Email | Other | A Large Language Model-based platform is integrated with the Health Vision Platform, a HIPAA-compliant, electronic health record data management system. Real-time summaries of prior SIC with be sent to clinicians caring for participants in the intervention arm. An email will be sent to the inpatient attending and responding clinician encouraging the team to review the SIC summary, discuss care preferences with the patient and ensure care is aligned with preferences and goals. If no SIC documentation is found, the email will instead encourage the outpatient oncologist to share information regarding undocumented SICs that may have occurred, as well as prompt inpatient and outpatient teams to engage the patient in SIC. For patients with no SIC documentation who are still admitted 72 hours later, the RA will send an additional email prompting SIC or asking the clinician to indicate that an SIC is not appropriate. Patients will also receive standard of care. |
| Measure | Description | Time Frame |
|---|---|---|
| Serious Illness Conversation (SIC) summary accuracy | Primary outcome: We will assess SIC summary accuracy by reviewing the source documentation for each bullet point of the 5 domains of the SIC summary. The RA will read each bullet point, read the source note for the date cited in the summary and indicate with a 1 (supported) or 0 (not supported) whether the documentation supports the summarized point. If a summarized point is scored as 0, the MPIs will independently review the item and assign a 1 or 0 to indicate whether the unsupported summary could negatively impact patient care. SIC summary accuracy will be determined at the patient level as the proportion of bullet points supported by underlying documentation divided by the total number of bullet points (i.e., a score from 0 to 1). We will determine the average SIC accuracy score across all patients for the study. We will also report the percent of LLM-SIC summary statements that are unsupported and clinically significant and provide qualitative descriptions of these instances. | 1 week |
| Measure | Description | Time Frame |
|---|---|---|
| Feasibility (successful delivery of the intervention) | Feasibility is defined as the successful delivery of the summary email, and will be calculated as the number of participants with successful delivery of the intervention email divided by the total number of intervention participants, with an 80% feasibility threshold. | 1 week |
| Measure | Description | Time Frame |
|---|---|---|
| Length of hospitalization | Length of hospitalization in days for the admission triggering enrollment (data on death during the admission, as well as discharge to hospice and re-admission within 30 days will also be collected) | 1 week |
| Days in the hospital |
Participant Inclusion Criteria:
Participant Exclusion Criteria:
-Patients with elective inpatient admissions (typically for chemotherapy or other treatments)
Clinician Inclusion Criteria:
-The clinician fills one of the following roles for the enrolled patient:
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Affiliation | Role |
|---|---|---|
| Christopher Manz, MD | Dana-Farber Cancer Institute | Principal Investigator |
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| Brigham and Women's Hospital | Boston | Massachusetts | 02115 | United States | ||
| Dana-Farber Cancer Institute |
The Dana-Farber / Harvard Cancer Center encourages and supports the responsible and ethical sharing of data from clinical trials. De-identified participant data from the final research dataset used in the published manuscript may only be shared under the terms of a Data Use Agreement. Requests may be directed to: [contact information for Sponsor Investigator or designee]. The protocol and statistical analysis plan will be made available on Clinicaltrials.gov only as required by federal regulation or as a condition of awards and agreements supporting the research.
Data can be shared no earlier than 1 year following the date of publication
Contact the Belfer Office for Dana-Farber Innovations (BODFI) at innovation@dfci.harvard.edu
Not provided
Not provided
| ID | Term |
|---|---|
| D009369 | Neoplasms |
| D003643 | Death |
| ID | Term |
|---|---|
| D010335 | Pathologic Processes |
| D013568 | Pathological Conditions, Signs and Symptoms |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
|
| Clinician Perspectives of Clinical Utility |
Clinician perspectives of clinical utility are defined as clinicians' views on the usefulness and relevance of serious illness communication (SIC) and, for intervention-arm clinicians, the LLM-generated SIC summaries for informing participant care. These perspectives will be assessed via structured REDCap surveys sent to both inpatient and outpatient clinicians, with follow-up phone interviews conducted by the research assistants for non-respondents. Clinician responses to surveys and interview questions will be summarized and stratified by intervention vs control groups. We will compare survey responses between the two study arms using Chisquared and Mann-Whitney U tests for binary and likert items, respectively. Responses to open-ended questions will be qualitatively summarized. |
| 1 week |
| Participant Perspectives on Inpatient and Outpatient Clinician Understanding of their Care Preferences and Acceptability of SIC Summaries | Participant acceptability is defined as participant perceptions of the appropriateness and usefulness of LLM-generated summaries of serious illness conversations (SICs). It will be assessed through structured in-person interviews including the validated "Heard and Understood" questionnaire and open-ended questions regarding the acceptability of SIC summaries and participant-clinician communication. Participant responses to interview questions will be summarized and stratified by intervention vs control groups. Responses to open-ended questions will be qualitatively summarized. | 1 week |
| New Documentation of SIC | New documentation of serious illness communication (SIC) during the index hospitalization is defined as any new note or entry created in the electronic health record (EHR) during the current hospital admission that reflects a discussion of the participant's values, goals, preferences, or prognosis, and meets the standardized codebook criteria for SIC. | 1 week |
Number of days in the hospital over the subsequent 90 days
| 90 days |
| Number of hospitalizations | Number hospitalizations over the subsequent 90 days | 90 days |
| ICU admissions | Number of intensive care unit admissions over the subsequent 90 days | 90 days |
| Intubation | Any occurrence of intubation over the subsequent 90 days | 90 days |
| Vital status (including date of death) | Vital status, including date of death, over the subsequent 90 days | 90 days |
| Boston |
| Massachusetts |
| 02115 |
| United States |