This study is a randomized controlled trial (RCT) investigating whether access to a new LLM interface can improve medical triage and diagnostic accuracy for laypeople compared to access to a standard LLM interface. It addresses previous findings where laypeople using standard LLMs performed worse than those using conventional methods (e.g., web search) due to incomplete symptom sharing and poor interpretation of AI advice. To address this, the research tests a structured LLM system that proactively asks clinical history questions before providing a standardized, easy-to-read diagnostic output.
Inclusion Criteria:
Exclusion Criteria:
ihsan.qazi@lums.edu.pk+923233333766
Participants will not be informed of which arm constitutes the "treatment" or what the study hypothesizes.
Participants can access any assistance methods they would typically employ (e.g., web search or health portals) in addition to a new LLM interface (based on GPT-4o) to complete medical scenarios. The new LLM interface uses a fixed system prompt that (a) instructs the model to ask targeted clarifying questions before providing any diagnostic or triage suggestions, and (b) requires all final responses to follow a structured template listing: possible conditions, approximate likelihood of each, and a recommended triage with brief reasoning.
Participants can use any assistance methods they would typically employ (e.g., web search or health portals) in addition to a standard LLM (GPT-4o) to complete medical scenarios. AI-overview in web searches will be disabled via an extension. They would not be allowed to access any LLMs other than the standard LLM interface.
ayeshaali@lums.edu.pk04235608368
The Diagnostic and Triage Capacity of Laypeople-large Language Model Collaboration in China
The Impact of Large Language Models on Diagnostic Reasoning Among LLM-Trained Medical Doctors
AI-assisted Rare Disease Diagnosis
AI in Respiratory Disease Prevention, Diagnosis, and Triage
Ophthalmic Diseases and AI: an RCT Study
Large Language Models Assist in Tumor MDT
Improving the Reliability of LLMs as Medical Assistants for the General Public
Effect of Perception-based Interventions on Public Acceptance of Using Large Language Models in Medicine