Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Xuanwu Hospital, Beijing | OTHER |
Not provided
Not provided
Not provided
Not provided
This study will evaluate whether three-minute six-dimensions education(3M-6D education) can improve the reliability of large language models as medical assistants for the general public. Participants will be randomly assigned to receive or not receive 3M-6D education and then use ChatGPT, Gemini, or non-AI information resources. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
This randomized, controlled, proof-of-concept simulation trial will evaluate whether three-minute six-dimensions education (3M-6D education) can improve the reliability of large language models as medical assistants for the general public.
Eligible participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five study groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Participants in the 3M-6D education GPT and 3M-6D education Gemini groups will receive approximately three minutes of education before using ChatGPT or Gemini.Each participant will be randomly assigned one of 10 standardized clinical scenarios and complete a simulated counseling task in unrestricted natural language within approximately 10 minutes. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| 3M-6D education GPT Group | Experimental | Participants will first be trained in 3M-6D education, then use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes. |
|
| 3M-6D education Gemini Group | Experimental | Participants will first be trained in 3M-6D education, then use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes. |
|
| GPT Group | Active Comparator | Participants will use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes. |
|
| Gemini Group | Active Comparator | Participants will use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes. |
|
| Control group | No Intervention | Participants will use non-AI tools such as internet searches and medical websites to complete a consultation task in unrestricted natural language in approximately 10 minutes. |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| three minutes six dimensions education | Behavioral | 3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, the investigators identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so the investigators call this approach three minutes six dimensions education (3M-6D education). |
| Measure | Description | Time Frame |
|---|---|---|
| Relevant conditions identification of the 3M-6D education GPT group compared with the GPT group | Relevant conditions identification is defined as the proportion of participants whose final response includes the expert-defined final diagnosis or a relevant differential diagnosis. | 1 hour. |
| Disposition concordance of the 3M-6D education GPT group compared with the GPT group | Disposition concordance is defined as the proportion of participants whose final care recommendation matches the expert-defined level. The five levels are self-care, routine outpatient care, urgent outpatient care, emergency department visit, and emergency medical services. | 1 hour. |
| Relevant conditions identification of the 3M-6D education Gemini group compared with the Gemini group | 1 hour. | |
| Disposition concordance of the 3M-6D education Gemini group compared with the Gemini group | 1 hour. |
| Measure | Description | Time Frame |
|---|---|---|
| Relevant conditions identification of the 3M-6D education GPT group compared with the control group | 1 hour. | |
| Relevant conditions identification of the 3M-6D education Gemini group compared with the control group | 1 hour. |
| Measure | Description | Time Frame |
|---|---|---|
| Failure to identify red flags in the 3M-6D education GPT group compared with the GPT group | Failure to identify red flags is defined as the proportion of participants whose final response does not include the expert-defined red-flag symptoms or warning signs for the assigned standardized simulated clinical scenario. | 1 hour. |
Inclusion Criteria:
Exclusion Criteria:
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Role | Phone | Extension | |
|---|---|---|---|---|
| Xunming Ji | Contact | 01083198962 | jixm@ccmu.edu.cn | |
| Chuanjie Wu | Contact | 01083199439 | wuchuanjie@ccmu.edu.cn |
Not provided
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| Beijing Ctiy | Recruiting | Beijing | Beijing Municipality | China |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five parallel groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Each participant will complete one standardized simulated clinical scenario.
Not provided
Not provided
Outcome assessors will be blinded to group assignment when evaluating participants' outcomes. Group information will be removed from the outcomes before assessment.
|
| ChatGPT | Other | Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language. |
|
| Gemini | Other | Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language. |
|
| Disposition concordance of the 3M-6D education GPT group compared with the control group | 1 hour. |
| Disposition concordance of the 3M-6D education Gemini group compared with the control group | 1 hour. |
| Red-flag identification in the 3M-6D education GPT group compared with the GPT group | Red-flag identification is defined as the proportion of participants whose final response includes the key warning signs that experts defined for the assigned scenario. | 1 hour. |
| Red-flag identification in the 3M-6D education GPT group compared with the control group | 1 hour. |
| Red-flag identification in the 3M-6D education Gemini group compared with the Gemini group | 1 hour. |
| Red-flag identification in the 3M-6D education Gemini group compared with the control group | 1 hour. |
| NASA Task Load Index score of the 3M-6D education GPT group compared with the GPT group | NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician. It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance. Each domain is scored from 0 to 100. The total score is the mean of the six domains. Higher scores indicate greater perceived task load. | 1 hour. |
| NASA Task Load Index score of the 3M-6D education GPT group compared with the control group | NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician. It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance. Each domain is scored from 0 to 100. The total score is the mean of the six domains. Higher scores indicate greater perceived task load. | 1 hour. |
| NASA Task Load Index score of the 3M-6D education Gemini group compared with the Gemini group | NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician. It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance. Each domain is scored from 0 to 100. The total score is the mean of the six domains. Higher scores indicate greater perceived task load. | 1 hour. |
| NASA Task Load Index score of the 3M-6D education Gemini group compared with the control group | NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician. It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance. Each domain is scored from 0 to 100. The total score is the mean of the six domains. Higher scores indicate greater perceived task load. | 1 hour. |
| Relevant conditions identification of the 3M-6D education GPT group compared with the 3M-6D education Gemini group | 1 hour. |
| Disposition concordance of the 3M-6D education GPT group compared with the 3M-6D education Gemini group | 1 hour. |
| Red-flag identification in the 3M-6D education GPT group compared with the 3M-6D education Gemini group | 1 hour. |
| NASA Task Load Index score of the 3M-6D education GPT group compared with the 3M-6D education Gemini group | NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician. It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance. Each domain is scored from 0 to 100. The total score is the mean of the six domains. Higher scores indicate greater perceived task load. | 1 hour. |
| Failure to identify red flags in the 3M-6D education GPT group compared with the control group |
| 1 hour. |
| Failure to identify red flags in the 3M-6D education Gemini group compared with the Gemini group | 1 hour. |
| Failure to identify red flags in the 3M-6D education Gemini group compared with the control group | 1 hour. |
| Underestimation of disposition in the 3M-6D education GPT group compared with the GPT group | Underestimation of disposition is defined as the proportion of participants whose final care recommendation is lower than the expert-defined disposition level for the assigned standardized simulated clinical scenario. | 1 hour. |
| Underestimation of disposition in the 3M-6D education GPT group compared with the control group | 1 hour. |
| Underestimation of disposition in the 3M-6D education Gemini group compared with the Gemini group | 1 hour. |
| Underestimation of disposition in the 3M-6D education Gemini group compared with the control group | 1 hour. |