Not provided
| ID | Type | Description | Link |
|---|---|---|---|
| ME-2024C1-36732 | Other Grant/Funding Number | Patient-Centered Outcomes Research Institute (PCORI) |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Patient-Centered Outcomes Research Institute | OTHER |
Not provided
Not provided
Not provided
The goal of this clinical trial is to evaluate how a conversational method of collecting medical history affects patients' perceptions and experiences compared to clinical care as usual. This conversational AI intake system collects medical history information, can be completed by participants at home, and do not disrupt routine clinical care.
The primary questions this study aims to answer are:
1) Does conversational intake affect patients' perceptions of empathy during their clinical interactions?
This will be a prospective study that follows a cohort of participants for four (4) months after engaging with the AI intake system. Because each participant serves as his/her own control, both comparators will be administered within-subject, and the order of exposure (AI intake vs. usual care) will be randomized to minimize sequence effects.
After completing the AI intake method, participants will rate their experience, particularly in terms of empathy and compare it to their usual interactions with their own clinicians.
Conversational artificial intelligence (AI) systems, such as those based on Large Language Models (LLMs) like ChatGPT, offer innovative ways to engage patients in health-related conversations. Despite these advances, challenges remain regarding patient safety and system reliability. Specific concerns include biased recommendations against certain patient groups, inaccuracies or misleading responses, and mechanical, unempathic interactions, particularly during sensitive moments such as when patients express suicidal thoughts. Testing conversational AI in healthcare settings is complicated due to the diverse medical, linguistic, and behavioral characteristics exhibited by patients.
This study addresses these challenges by developing an advanced conversational AI system guided by a structured knowledge-based topic network to maintain conversation relevance and coherence. Additionally, the investigators introduce a novel patient simulator methodology that mimics diverse medical histories, linguistic styles, and behavioral interactions, enhancing pre-clinical testing rigor.
The research focuses specifically on the clinical context of depression management, aiming to optimize antidepressant selection. Currently, many patients undergo a frustrating and costly trial-and-error process to find effective antidepressants. The study compares two approaches and their impact on a patient's perceptions of empathy:
The conversational AI intake system leverage a curated, evidence-based knowledgebase of 15 commonly used antidepressants, considering factors like patient age, gender, comorbidities, and previous antidepressant use. The accuracy and completeness of the AI-generated recommendations are rigorously verified in by clinicians prior to any medication changes, adhering to FDA safety requirements.
This will be a prospective study that follows a cohort of participants for four (4) months after engaging with the AI intake system. A primary goal of the project is to evaluate how conversational AI impacts patient-centered outcomes, specifically patient perceptions of empathy and communication quality. Patients with major depressive disorder will be recruited online, enhancing participant diversity and representativeness. Because each participant serves as his/her own control, both comparators will be administered within-subject, and the order of exposure (AI intake vs. usual care) will be randomized to minimize sequence effects. Outcomes will include differences in data completeness and patient perceptions of empathy. To ensure that the AI conversation is evaluated against the best of usual care, we will select the highest empathy score achieved across multiple visits as the usual care comparator.
Beyond immediate clinical outcomes, the project's methodological advancements, particularly the development of robust, bias-mitigated conversational systems and comprehensive patient simulation for AI testing, will have broad applicability across healthcare domains. The conversational AI and patient simulator will be made publicly available at no cost, providing tools that other researchers, clinicians, and healthcare providers can utilize and adapt to various health contexts.
Patient and stakeholder engagement is integral to the study. A representative advisory board, including patients with lived experience of depression, clinicians, mental health advocates, and researchers, guides all phases of the project. This collaborative framework ensures that the research remains patient-centered and responsive to real-world clinical needs and experiences.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Conversational AI system vs Usual Care | Experimental | Participants complete medical history intake through an interactive conversational AI designed to support patient-centered, empathetic dialogue. Using large language models (LLM), the system interprets patient input, maintains context, and generates natural-language responses. A dialogue manager prioritizes medically relevant topics to support efficient data collection and reduce off-topic discussion. For safety, trained human monitors oversee conversations in real time and can intervene if risks such as self-harm arise. The AI intake is compared with patients' experiences with their clinicians through monthly follow-up questionnaires over four months. The study evaluates patients' ratings of empathy, communication quality, and engagement, not conversation content. Each participant serves as their own control, with AI intake and usual care compared within-subject and randomized by order of exposure. |
|
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Conversational AI system vs Usual Care | Other | Participants complete medical history intake through an interactive conversational AI designed to support patient-centered, empathetic dialogue. Using large language models (LLM), the system interprets patient input, maintains context, and generates natural-language responses. A dialogue manager prioritizes medically relevant topics to support efficient data collection and reduce off-topic discussion. For safety, trained human monitors oversee conversations in real time and can intervene if risks such as self-harm arise. The AI intake is compared with patients' experiences with their clinicians through monthly follow-up questionnaires over four months. The study evaluates patients' ratings of empathy, communication quality, and engagement, not conversation content. Each participant serves as their own control, with AI intake and usual care compared within-subject and randomized by order of exposure. |
| Measure | Description | Time Frame |
|---|---|---|
| Perceptions of empathy | The primary outcome assesses the impact of intake methods (conversational AI vs. structured survey) on patients' perceptions of empathy. Patient empathy perceptions will be measured using the Jefferson Scale of Empathy (JSE), a validated instrument. A higher JSE score means higher perceptions of empathy. We hypothesize that patients interacting with the conversational AI will report higher perceived empathy scores compared to those using the structured survey. For evaluation of impact of comparators, the primary analysis will focus on impact of LLM and clinician encounters on patient perceived empathy. These data will be analyzed using paired comparisons (ANOVA), with randomized order of exposure accounted for in the design. At the power of 0.80, a small-to medium effect size (d = 0.3), and significance level of 0.05, a sample size of 90 participants is needed, with 112 participants needed with attrition. In order to support higher power, a total of 130 participants will be recruited. | From enrollment up to 4 months after participation |
| Measure | Description | Time Frame |
|---|---|---|
| Communication Accommodation | The study team will assess communication accommodation to determine whether the conversational AI meets patient needs through a content analysis of conversation transcripts. This analysis will rate three key dimensions: empathy, communication quality, and effectiveness. Codes for each are drawn from validated, widely used scales. From this, the study team will generate composite scores to classify AI conversations as high or low in each dimension and overall. To validate the findings, the study team will compare content analysis scores with patient self-reports on perceived levels (high or low) of empathy, quality, and effectiveness. Together, these data will provide a robust evaluation of the AI's communication accommodation. |
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Role | Phone | Extension | |
|---|---|---|---|---|
| Farrokh Alemi, PhD | Contact | 7579459484 | rapidai@gmu.edu | |
| Kevin Lybarger, PhD | Contact | 7579459484 | rapidai@gmu.edu |
Not provided
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| George Mason University | Fairfax | Virginia | 22030 | United States |
De-identified individual participant data from this study will be shared with the Patient-Centered Outcomes Research Institute (PCORI) and made available through PCORI's designated data repository. Qualified researchers may request access to the data by following PCORI's established data access procedures, which include submission of a research proposal and a signed data use agreement.
Data will be available by June 2029, following completion of the final research report, and will remain available for at least 7 years.
Qualified researchers may request access to de-identified individual participant data and supporting documents (including the protocol, statistical analysis plan, informed consent form, and analytic code) via the PCORI-designated repository. Access to certain data elements may require submission of a research proposal and a signed data use agreement.
Not provided
Not provided
| ID | Term |
|---|---|
| D003865 | Depressive Disorder, Major |
| ID | Term |
|---|---|
| D003866 | Depressive Disorder |
| D019964 | Mood Disorders |
| D001523 | Mental Disorders |
Not provided
Not provided
This will be a prospective study that follows a cohort of participants for four (4) months after engaging with the AI intake system. Each subject serves as his/her own control, both comparators will be administered within-subject, and the order of exposure (AI intake vs. usual care) will be randomized to minimize sequence effects.
Not provided
Not provided
Not provided
Not provided
|
| From enrollment up to 4 months after participation |
| Adherence to recommendations | Adherence to the recommendations provided by the conversational AI will be evaluated approximately one month after patient participation. Follow-up includes a structured questionnaire and brief interview, where patients report (1) if they discussed system-generated recommendations with their clinician, and (2) their current medications by direct reference to medication containers. Collected data will allow classification of clinician prescriptions as either concordant or discordant with system-generated recommendations, facilitating evaluation of adherence behaviors. | From enrollment up to 4 months after participation |