Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
This multicenter retrospective study aims to evaluate the diagnostic and therapeutic performance of three large language models-ChatGPT, Gemini and Deepseek-using 800 archived inpatient medical records from urology departments across four tertiary hospitals. The study will focus on the accuracy and applicability of these models in disease recognition, preliminary diagnosis and treatment recommendation generation, in order to explore their potential value and limitations in supporting clinical decision-making in real-world settings.
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Large Language Model Assessment (ChatGPT, Gemini, DeepSeek) | Other | De-identified inpatient medical records were retrospectively collected from the urology departments of four tertiary hospitals (200 cases per site, 800 in total). Each case included standardized clinical information such as demographics, chief complaint, history of present illness, past medical history, physical examination, laboratory and imaging findings, discharge diagnosis and treatment plan. To simulate the role of an AI system in a "first-visit physician" scenario, all diagnostic conclusions, differential diagnoses and treatment plans were removed before being input into the models. Three large language models (ChatGPT, Gemini and DeepSeek) were prompted with a standardized instruction: "Based on the above clinical information, provide your preliminary diagnosis, differential diagnoses and treatment recommendations." Each model generated outputs including (i) primary and secondary diagnoses, (ii) differential diagnosis lists with reasoning and (iii) preliminary treatment suggesti |
| Measure | Description | Time Frame |
|---|---|---|
| Diagnostic Accuracy: Assessed by Top-1 accuracy | Top-1: Proportion of cases where the model's first diagnosis matches the true primary diagnosis. | Through study completion, an average of 3 months |
| Diagnostic Accuracy: Assessed by Top-3 accuracy | Top-3: Proportion of cases where the true diagnosis appears in the model's top 3. | Through study completion, an average of 3 months |
| Diagnostic Completeness | Proportion of the model's diagnoses that overlap with all diagnoses (primary and secondary) in the case. | Through study completion, an average of 3 months |
| Differential Diagnosis Quality | Evaluated by experts using a Likert 5-point scale, considering factors like common disease coverage, logical clarity, and specificity | Through study completion, an average of 3 months |
| Treatment Plan Quality | Assesses whether the model's treatment suggestions align with clinical guidelines, scored by experts on completeness, appropriateness, and safety. | Through study completion, an average of 3 months |
| Analysis Time | 5.Time taken by the AI model to provide diagnoses and treatment suggestions (in seconds), reflecting real-time capability. | Through study completion, an average of 3 months |
Not provided
Not provided
Inclusion Criteria:
Exclusion Criteria:
Not provided
Not provided
Not provided
The study population was drawn from the following institutions: The First Affiliated Hospital of Fujian Medical University, The Second Affiliated Hospital of Fujian Medical University,Shishi City Hospital and Shaowu City Hospital
| Name | Role | Phone | Extension | |
|---|---|---|---|---|
| Ning Xu | Contact | +86-13235907575 | drxun@fjmu.edu.cn |
Not provided
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| The First Affiliated Hospital of Fujian Medical University | Recruiting | Fuzhou | China |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| ID | Term |
|---|---|
| D014570 | Urologic Diseases |
| ID | Term |
|---|---|
| D052776 | Female Urogenital Diseases |
| D005261 | Female Urogenital Diseases and Pregnancy Complications |
| D000091642 | Urogenital Diseases |
| D052801 | Male Urogenital Diseases |
Not provided
Not provided
Not provided
Not provided
Not provided