Evaluating the Potential of Large Language Models for Respiratory Disease Consultations

Evaluating the Potential of Large Language Models for Respiratory Disease Consultations: A Randomized Crossover Trial

The clinical trial aimes to evaluate multiple large language models in respiratory disease consultations by comparing their performance to that of human doctors across three major medical consultation scenarios.

The main question aims to answer are:

• How do large language models perform in comparison to human doctors in diagnosing and consulting on respiratory diseases across various clinical scenarios?

In three clinical scenarios including the online query section, the disease diagnosis section and the medical explanation section, research assistants or volunteers will be asked to cross-question all LLMs or real doctors using predefined online questions and their own issues. After each questioning session, a short washout period is implemented to eliminate potential biases.

Inclusion Criteria:

1. Self-reported symptoms of common respiratory diseases, such as cough, chest tightness, fever, and wheezing
2. Ability to engage in LLM dialog operations independently or with minimal peer training
3. A health status deemed suitable for study participation by the pulmonology experts

Exclusion Criteria:

1) Excessively poor health status