Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Health Science Education Small Grants Program | UNKNOWN |
Not provided
Not provided
Not provided
Not provided
The purpose of this study is to refine and test existing enterprise-grade large language model (LLM) based on generative artificial intelligence (AI), to assess the feasibility and acceptability of LLM-based feedback, to assess the ability of LLM-based feedback to improve residents' communications,to explore the ability of standardized patients to assess residents' communication and to explore the ability of residents to self-assess their communication complexity
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Educational LLM-based feedback tool | Experimental |
| |
| Control | No Intervention |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| educational LLM-based feedback tool | Behavioral | Participants will have their verbal communications with standardized patients (SP) regarding 3 different scenarios recorded, transcribed, and analyzed in real-time by the large language model (LLM) and will receive feedback as suggestions and alternative scripts. These will be reviewed by residents between SP scenarios |
| Measure | Description | Time Frame |
|---|---|---|
| Readability discernment as assessed by a survey | This will be scored by the by Cohen's Kappa values from 1-5. Higher Cohen's kappa scores mean better outcome | end of intervention ( 1 hour after baseline) |
| Quality discernment as assessed by a survey | This will be scored by the by Cohen's Kappa values from 1-5. Higher Cohen's kappa scores mean better outcome | end of intervention ( 1 hour after baseline) |
| Correctness of recommendations as assessed by a survey | This will be reported on a 5 point Likert scale form 1 very incorrect to 5 very correct | end of intervention ( 1 hour after baseline) |
| Applicability of recommendations as assessed by a survey | This will be reported on a 5 point Likert scale form 1 very inapplicable to 5 very applicable | end of intervention ( 1 hour after baseline) |
| Perceived readability of resident-standardized patient (SP) interactions as assessed by a survey: schooling level | This will be categorically reported in the following categories: Elementary middle high college graduate | end of intervention ( 1 hour after baseline) |
| confidence in communication ability | This is scored from 1( very unconfident) to 5 (very confident) | end of intervention ( 1 hour after baseline) |
| usefulness of the LLM |
| Measure | Description | Time Frame |
|---|---|---|
| Survey feedback on the LLM interface | This is scored from 1( very unrealistic) to 5 (very realistic) | end of intervention ( 1 hour after baseline) |
| readability grade level of resident-SP transcripts as assessed by the Flesch-Kincaid Grade Level (FKGL) readability tool |
Not provided
Inclusion Criteria:
Not provided
Not provided
Not provided
Not provided
| Name | Role | Phone | Extension | |
|---|---|---|---|---|
| Krislynn M Mueck, MD, MS, MPH | Contact | (713) 500-7409 | Krislynn.M.Mueck@uth.tmc.edu | |
| William D Rieger | Contact | (713) 500-7300 | William.D.Rieger@uth.tmc.edu |
| Name | Affiliation | Role |
|---|---|---|
| Krislynn M Mueck, MD, MS, MPH | The University of Texas Health Science Center, Houston | Principal Investigator |
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| The University of Texas Health Science Center at Houston | Houston | Texas | 77030 | United States |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
|
This is scored from 1( very useless) to 5 (very useful)
| end of intervention ( 1 hour after baseline) |
| acceptability of future use | This is scored from 1( very unlikely) to 5 (very likely) | end of intervention ( 1 hour after baseline) |
Readability of resident-SP encounter transcripts will be assessed using the Flesch-Kincaid Grade Level formula, which estimates the U.S. school grade level required to understand the text. Higher scores indicate a higher reading grade level (i.e., lower readability). Formula used: Grade level= 0.39(total words/total sentences) + 11.8 (total syllables/total words)-15.59 |
| end of intervention ( 1 hour after baseline) |
| Quality based on Ensuring Quality Information for Patients (EQIP) score of resident-SP transcripts | Percentage score based on a validated questionnaire This has 20 questions and each is scored from 1(yes), 0.5(partly), 0 (no) and question is removed if it does not apply.Scores are reported as a percentage and higher percentage score indicates better quality | end of intervention ( 1 hour after baseline) |
| Perceived readability of SP-resident interactions as assessed by a standardized survey | This will be categorically reported in the following categories: Elementary middle high college graduate | end of intervention ( 1 hour after baseline) |
| confidence in communication ability | This is scored from 1( very unconfident) to 5 (very confident) | end of intervention ( 1 hour after baseline) |