Evaluating an Artificial Intelligence Tool to Help Primary Care Doctors Diagnose Skin Conditions.

A Multi-Reader Multi-Case (MRMC) Study Assessing the Impact of Legit.Health Plus on the Diagnostic Accuracy and Referral Decision-Making of Primary Care Physicians for Skin Lesions.

This study aims to determine if an artificial intelligence (AI) medical device can help primary care doctors more accurately identify and manage various skin conditions. Skin issues are a frequent reason for doctor visits, but differences in expertise between general practitioners and specialists can sometimes lead to misdiagnoses or unnecessary referrals.

The researchers hypothesized that the information provided by the AI device would increase the true diagnostic accuracy of primary care practitioners for multiple dermatological conditions. To test this, the study followed a prospective, self-controlled design where each participating doctor served as their own comparison.

During the study, 9 primary care physicians evaluated 30 clinical images representing a variety of skin pathologies. For each image, the doctors followed a two-step process:

• First, they provided a diagnosis based only on the image and the patient's medical history.
• Second, they were shown the AI's analysis-including the top 5 suggested diagnoses and confidence levels-and asked to provide a final diagnosis.

The study also investigated if the AI could help doctors decide whether a patient truly needs a referral to a specialist or if the condition could be handled remotely via teledermatology. The primary question was whether using this AI support would significantly increase the number of correct diagnoses made by primary care doctors and lead to more efficient patient care.

This detailed description outlines the clinical methodology, technical framework, and data integrity protocols utilized in the investigation of the Legit Health Plus medical device for skin pathologies in primary care.

Study Design and Technical Methodology The research was conducted as a prospective observational and cross-sectional self-controlled study. It utilized a Multi-Reader Multi-Case (MRMC) framework to measure the impact of Computer-Aided Diagnosis (CAD) on clinician performance.

• Self-Controlled Framework: Each primary care practitioner (PCP) served as their own comparator, providing diagnoses first without and then with device support.
• Sequential Evaluation Workflow: Participants accessed a secure web-based platform to review 30 clinical cases. For each case, doctors provided an initial diagnosis based on an image and medical history, followed by a final diagnosis after reviewing the AI's top 5 suggested ICD-11 categories and confidence levels.
• Clinical Decision Support: The study also evaluated clinician decisions regarding dermatology referrals and the feasibility of remote management (teledermatology) based on AI-provided data, such as malignancy indices.
• Case Distribution: The 30 clinical images represented nine different conditions, including Melanoma, Basal Cell Carcinoma, Psoriasis, and Hidradenitis Suppurativa, all previously confirmed by dermatologists and anatomical pathology.

Quality Assurance and Data Management

To ensure the scientific integrity of the clinical investigation, the following quality and monitoring protocols were implemented:

• Centralized Case Report Forms (CRF): All data were collected via a secure web platform where entries were time-stamped and stored in a central database.
• Data Validation and Checks: Validation rules were applied using computer filters to automatically identify missing values or logical inconsistencies. This was supplemented by manual editing and exploratory statistical techniques to detect errors.
• Monitoring Plan: An independent clinical monitor oversaw the investigation. Monitoring included remote video or telephone meetings every three months to ensure compliance with the Clinical Investigation Plan (CIP) and ISO 14155:2020.
• Bias Minimization: Random selection of practitioners helped ensure that outcomes were not influenced by pre-existing group characteristics. Standardized protocols ensured all participants were evaluated under identical conditions.

Inclusion Criteria:

• Board-certified primary care physicians regardless of their professional experience.
• High-quality images of patients with different skin conditions.

Exclusion Criteria:

• Low-quality images of patients which can not be properly analyzed.