A multicentre, randomised diagnostic accuracy study to evaluate whether the rare disease-specific AI can improve diagnostic accuracy and efficiency for physicians managing real-world clinical cases.

Rare diseases collectively affect approximately 300 million individuals worldwide. This prolonged diagnostic delay is attributable in large part to the breadth of over 7,000 recognized rare conditions, which far exceeds the clinical exposure of any individual physician. A rare disease-specific diagnostic AI was developed by Peking Union Medical College Hospital (PUMCH), supporting differential diagnosis generation, clinical workup planning, and genomic variant interpretation. A balanced crossover design ensures that each enrolled physician serves as their own control, substantially reducing confounding from inter-reader variability in baseline diagnostic competency. Within each physician, cases are randomly assigned at the case level to either the AI-assisted or unassisted condition, such that each physician reads a subset of cases with AI assistance and the remaining cases without. This within-reader, case-level randomization eliminates the need for a washout period and directly controls for inter-reader differences in baseline diagnostic competency. All cases are collected from real-world clinical settings with independently confirmed gold-standard diagnoses and span a pre-specified spectrum of rare and non-rare disease categories, reflecting the differential diagnostic challenge encountered in routine clinical practice, to ensure diagnostic breadth and clinical representativeness. Physician seniority (junior vs. senior) is incorporated as a pre-specified stratification and subgroup analysis variable. Diagnostic outputs are evaluated by an independent Expert Adjudication Committee, blinded to the assistance condition, using standardized scoring criteria established prior to data collection.

Inclusion Criteria:

• 1. Licensed physicians at the junior or senior level affiliated with internal medicine, neurology, pediatrics, and rare disease-related departments.
• 2. Willingness to provide written informed consent, adhere to trial protocols, and complete all required pre-study training prior to enrollment.

Exclusion Criteria:

• 1. Prior exposure to any of the clinical cases included in the study case library.
• 2. Direct participation in the design or development of the AI model.