Evaluation of a Retrieval Augmented Large Language Model as a Diagnostic Copilot in Rheumatology

Evaluation of a Retrieval Augmented Large Language Model as a Diagnostic Copilot in Rheumatology

This trial evaluates whether providing physicians with access to Prof. Valmed, a clinical decision support medical product, improves identification of rheumatic diseases and formulation of differential diagnoses compared with conventional decision support.

Advanced AI, particularly large language models, shows promise for enhancing clinical reasoning, yet most systems such as ChatGPT are not certified as medical products. Prof. Valmed is a clinical decision support medical product designed to assist physicians in diagnostic decision making. Given frequent referral problems and diagnostic delays in rheumatology, evaluating such support is highly relevant for clinical workflows.

This randomized controlled trial will test whether access to Prof. Valmed improves physicians' diagnostic performance in cases of suspected rheumatic disease compared with conventional decision support. Participants will be randomized to either use Prof. Valmed or rely on conventional tools while working through standardized clinical cases. For each case, participants will submit up to three differential diagnoses and a confidence rating. Independent reviewers, blinded to group allocation, will adjudicate accuracy. Findings will clarify the benefits and limitations of integrating Prof. Valmed into routine practice.

Inclusion Criteria:

• Participants must be licensed physicians.
• Training in rheumatology, internal medicine, emergency medicine, family medicine, dermatology or orthopedics.

Exclusion Criteria:

• Not currently practicing clinically.