Correlation Between Online Case-based Training of Pathologists in Dermoscopic Images and Diagnostic Accuracy in Histopathological Interpretation of Skin Lesions Suspicious of Melanoma
Correlation Between Online Case-based Training of Pathologists in Dermoscopic Images and Diagnostic Accuracy in Histopathological Interpretation of Skin Lesions Suspicious of Melanoma
Pathologists provide the current gold standard in skin lesion diagnostics, most often primarily based on the interpretation of histological slides. Still, it has been suggested that pathologists' diagnostic accuracy and confidence could be improved if they gained access to additional clinical information and in-vivo clinical and dermoscopic images of melanocytic tumors. This study examines the effect of digital training for pathologists in interpreting dermoscopic and clinical skin tumor images.
This study aims to evaluate the impact of dermoscopy training on pathologists' assessment of melanoma-suspected skin lesions.
Data collection of DAHT cases: Department of plastic surgery, Herlev hospital, year 2020-2021,
DAHT platform: Made in 2021-2023 by Melatech,
Consensus agreement: Four dermatopathologists assess all DAHT cases, year 2023-2024
Enrollment of pathologists: Randomization and assessment DAHT cases, year 2026.
Background Several publications suggest that the increasing melanoma incidence may partly be caused by histopathological overdiagnosis (Glasziou) and that the consequences of missing a melanoma may be one reason for this (Titus, L. J.). Pathologists provide the current gold standard in skin lesion diagnostics (Elmore 2017), primarily based on the interpretation of histological slides. Still, it has been suggested that pathologists' diagnostic accuracy and confidence could be improved if they gained access to additional clinical information and in vivo clinical and dermoscopic images of melanocytic tumors (Scolyer, Elder). The inter-rater reliability is significantly enhanced when pathologists are provided with dermoscopic and clinical images of the lesion during histopathological interpretation (Bauer, J.). This effect is especially pronounced among expert dermatopathologists who are proficient at evaluating dermoscopic images (Ferrara). However, interpreting dermoscopic images is challenging, and mastery typically requires several years of clinical experience (Ternov). Chevolet et al. reported in 2015 a significant difference in the interpretation of dermoscopic images between novices and clinicians with former dermoscopy training (Chevolet). The learning journey can be significantly shortened if the trainee receives comprehensive training in pattern recognition for dermoscopy and clinical images with immediate, accurate, and individualized feedback and access to a library with a large selection of skin lesion cases (Ericsson, Nervil). Recent studies in Denmark have shown that General Practitioners can increase their diagnostic accuracy by up to 10,5% by using a Large-scale Interactive Image Repository (LIIR) app for a few hours (Nervil). This method has yet to be tested on pathologists, which is what we indeed intend to do in this project.
Previous studies have focused only on melanocytic lesions (Elmore, Elder, Piepkorn, Ferrara). Still, most pathologists will receive both melanocytic and pigmented non-melanocytic lesions (seborrheic keratoses, dermatofibromas, etc.) clinically suspected of being melanoma. The current practice in preparing histological specimens involves creating standardized sections based on the clinician's assessment of the diagnosis and the size of the skin lesion. However, there is often no consideration of potentially suspicious areas within the skin lesion, and standard sectioning, staining, and subsequent evaluation may not occur in the most clinically suspicious areas. Previous studies have highlighted the importance of clinical information for accurate assessment of skin lesions, particularly melanocytic lesions (Scolyer), yet standardized clinical information for histopathological examination has not been systematically implemented in Denmark, likely due to a lack of technology enabling easy information sharing across medical specialties and healthcare sectors. Improving pathologists' diagnostic accuracy is crucial to ensuring patients receive the correct treatment. Dermoscopic images and training in their interpretation could be the future of histopathological skin lesion training.
Hypothesis Digital training in dermoscopy and clinical image interpretation improves diagnostic accuracy, confidence, and ease of skin lesion assessment among pathologists compared to those without such training.
Aim This study aims to evaluate the impact of dermoscopy training on pathologists' assessment of melanoma-suspected skin lesions.
Method The DAHT project includes an unfiltered selection of 203 clinically melanoma-suspect skin lesions excised at a specialized surgical department and consists of dermoscopic and clinical images, along with clinical information about the patients and skin lesions; this material is referred to as "DAHT cases". These cases will be assessed by the participating pathologists in a randomized order, and both individual and group performance will be analyzed.
DAHT Case Database
Lesion data were collected from patients in 2020-2021 at the Department of Plastic Surgery, Herlev Hospital. Requirements for eligibility for the current study are:
Patients received oral and written information about the project and were asked to sign a consent form before participation. The participation did not affect the treatment, diagnostics, or follow-up of the patients included. Upon consent, the following information was collected for each lesion:
Clinical image Dermoscopic image Patient CPR number (personal ID number) Sex and age of the patient Location of skin tumor (on a 3D avatar) Medical history (former treatment, congenital nevi, if pregnant, time of appearance of skin lesion, change in appearance, symptoms, former melanoma or other skin diseases, family history of melanoma, sun exposure within the last six months) After excision, the specimen was prepared for pathological examination, and an experienced dermatopathologist chose a representative hematoxylin-eosin stain and, if available, a MelanA stain for each skin lesion for the study. These stains were subsequently digitized and linked to relevant information (dermoscopic and clinical images, tumor location, sex, age, lesion information, etc.). The Personal Identification number (CPR number) was deleted, and the clinical images were cropped, rendering the cases anonymous. All cases are stored in a database under a random, anonymous ID number. The original diagnosis made by the pathologists at the pathological department at Herlev Hospital was blinded to the principal investigator and is therefore not included in this study.
Web-based IT platform
The investigators initiated the development of an IT platform for the trial (The DAHT platform). The platform has been upgraded to enable the following features:
User login Automated randomization Case presentation Diagnostic options User tracking Study data exportation The diagnostic options are based on the standardized MPATH-Dx version 2.0 (Barnhill). After diagnosing the lesion, participating pathologists will rate both their confidence in the chosen diagnosis and assess the difficulty of the lesion on a 6-step Likert scale. They will also be asked if they would want a second opinion and/or additional tests/stains. The tracking feature will enable various analyses, including whether the participants used the histology stains, dermoscopic images, clinical images, and clinical information when diagnosing each case.
Executive phase
General pathologists and dermatopathologists will be enrolled. All participants will receive an email outlining the trial and the handling of their data before signing a digital consent form. Upon inclusion, each enrolled pathologist will be asked to fill out a digital sign-in form with the following demographic baseline variables:
Name E-mail address Sex Country Number of years interpreting skin lesions (0-2, 3-5, 6-9,10+) Caseload from melanocytic lesions per month (1-50, 51-100, 101+) Former training in dermoscopy (yes/no) Perceived relevance of dermoscopic images during histopathological evaluation on a 5-step Likert scale Routine with the use of digitized slides (yes/no)
After sign-in, all participants will be automatically randomized (allocation ratio 1:1) to either the intervention or the control group.
Participants in the intervention group will receive immediate access to a previously developed digital (tablet or smartphone) educational platform, Dermloop Learn, with training in the interpretation of dermoscopic images of melanocytic lesions and other common skin lesions, with educational material on the correlation between dermoscopy and histology.
In order to maximize the learning potential, participants will be asked to use the learning intervention for 28 days and train on at least 200 quiz cases, with a goal of at least 50 cases per. week. They will not have access to DAHT cases during this period. After finishing the learning intervention, they will get access to the DAHT platform and be able to diagnose DAHT cases.
Participants in the control group will not have access to the digital educational platform or the learning intervention, but will assess the DAHT cases immediately after signing up.
Each participant (control or intervention) will be asked to evaluate between 30 and 100 DAHT cases within 4 months.
Statistics Minimum cases per participant Case variance was fixed at 21%, derived from the observed difficulty distribution of the 203-case library (hard: 127/203, 62.6%; medium: 31/203, 15.3%; easy: 45/203, 22.2%). A sensitivity analysis was conducted across plausible ranges of person variance (10-30%) and residual variance (45-75%), estimating the minimum number of cases per participant required to achieve a generalizability coefficient Φ ≥ 0.80, using the D-study approximation. A minimum of 30 cases per participant was adopted, covering all feasible variance scenarios where person variance ≥ 18% and residual variance ≤ 60%.
Minimum number of participants A two-proportion comparison (control: 40%, intervention: 60%; OR = 2.25, Cohen's h = 0.41; two-sided α = 0.05, 80% power) yielded a naive estimate of 99 participants per group. Applying a design effect of 1.30 for the planned GLMM with crossed random effects yields a final requirement of 129 participants per group (258 total). No dropout inflation was applied; participants who withdraw will be replaced through continuous recruitment. The design effect will be confirmed by simulation-based power analysis (simr) (Green P, MacLeod CJ. SIMR: an R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol. 2016;7(4):493-498) in a planned interim analysis of the first 30 completers.
Odds ratio; Cohen's h 2.25; 0.41
Power; α (two-tailed) 80%; 0.05
Naive n per group 99
GLMM design effect Design effect 1.30
Adjusted n per group 129
Dropout Strategy; inflation applied Continuous replacement; none Final requirement 129 per group (258 total)
Justification for assumptions Pilot data could not be used directly for sample size estimation: case evaluations were sparse, and agreement proportions were identical between trained and untrained pathologists at baseline (dermoscopic accuracy: 58.3% vs 52.8%). Following training, the intervention group reached 75.0% dermoscopic accuracy on the same 203-case library (within-person improvement: +22.2 percentage points, OR = 2.65), supporting the plausibility of the assumed 20-percentage-point between-group effect. As dermoscopic and histopathological accuracy are related but distinct competencies, and within-person pre-post effects tend to exceed between-group differences, a conservative estimate of 40% vs 60% was adopted based on expert opinion and prior dermoscopy training studies (Nervil).
Case and rater requirements With 258 participants each completing a minimum of 30 cases, the expected number of evaluations per case is (258 × 30) / 203 ≈ 38, exceeding the recommended minimum of 5-10 for stable variance component estimation. Cases will be assigned in randomized order via the DAHT platform. Recruitment will continue until all 203 cases have been evaluated by the required number of participants.
Data analysis plan for study 2 All analyses will be conducted in R. Primary inferential analyses use a generalized linear mixed model (GLMM) with crossed random effects for participants and cases (lme4 package), accounting for the incomplete crossed design in which each participant evaluates a random subset of cases.
Primary analyses Two primary analyses will be conducted, both using the same GLMM structure with a binary outcome (correct/incorrect), group as a fixed effect (intervention vs control), and crossed random effects for participant and case. Results will be reported as odds ratio (OR) with 95% confidence interval and two-sided p-value at α = 0.05.
Analysis 1 - Diagnosis accuracy: A correct diagnosis is defined as an exact match with the Study 1 expert consensus diagnosis. Pre-specified prior to data collection; no partial credit applied.
correct_diagnosis ~ group + (1 | participant_id) + (1 | case_id), family = binomial Analysis 2 - Classification accuracy: Correct classification is defined as an exact match with the Study 1 expert consensus MPATH-Dx 2.0 classification (Barnhill). Pre-specified prior to data collection; one-class discrepancies are scored as incorrect.
correct_classification ~ group + (1 | participant_id) + (1 | case_id), family = binomial
Secondary and descriptive analyses Weighted Cohen's Kappa (quadratic weights) will be calculated per participant against the expert gold standard (irr package),(x) reported as median and interquartile range per group. Kappa is reported as a descriptive complement to the GLMM, characterizing the quality of agreement across the MPATH-Dx ordinal scale.
Generalizability theory (G-theory) - interim analysis A confirmatory G-study will be conducted once 30 participants have completed the study (minimum 15 per group), using the gtheory package in R. Variance components for persons (σ²(p)), cases (σ²(i)), and residual interaction (σ²(pi,e)) will be estimated and expressed as percentages of total variance. A D-study will confirm whether Φ ≥ 0.80 is achieved across the 30-100 case range. This analysis is confirmatory only; n = 258 is fixed prior to recruitment and will not be adjusted. Concurrent with the G-study, simulation-based power confirmation will be conducted using the simr package (500 Monte Carlo iterations, powerCurve extended to 258 participants) to confirm ≥ 80% power to detect OR = 2.25 under the observed variance structure.
Incomplete data Participants who complete fewer than 30 cases will be replaced and excluded from the primary analyses. Cases with incomplete coverage will be retained; the GLMM and G-theory accommodate unbalanced designs. A sensitivity analysis will compare baseline characteristics of completers and non-completers (professional grade, years of experience, self-rated confidence) to assess non-random dropout.
Statistical software Data cleaning, randomization, and all statistical analyses will be conducted using R (R Foundation for Statistical Computing). G-theory variance components and generalizability coefficients will be estimated using the gtheory package. GLMM analyses will use the lme4 package. Simulation-based power analyses will use the simr package. Results will be reported with appropriate measures of uncertainty, including confidence intervals and reliability coefficients where applicable.
Feasibility and risk mitigation Everything is ready for the study to initiate. As soon as our group has the last funding for the PI and some funding for a statistician, we will be able to begin the recruitment of participants The study has a list of requirements and risks that have been or will all be mitigated during the planning and execution of the study.
The DAHT Platform and the DAHT Cases have already been developed and prepared through funding from previous studies. Our PI conducted the initial DAHT Case patient recruitment, built the DAHT Case database, and will continue working on the DAHT projects.
The DAHT cases are ready and digitized, accessible through the thoroughly tested DAHT Platform, and vetted by four of the world's leading experts in melanocytic skin lesions, with a consensus agreement by this expert panel on both a gold-standard diagnosis and classification.
The educational intervention has been thoroughly tested, and its effect has been proven several times (Ternov, Nervil, Sigrid Christensen).
The power calculations suggest a relatively large sample size, which might be challenging to accommodate. Fortunately, we have not limited ourselves to a single country or region. With online case presentations and a web-based platform, we can enroll participants from all countries, making the large number of participants realistic. All participants who do not complete at least 30 DAHT cases will be treated as dropouts. This caveat increases the risk of low participant numbers, which we account for by continuing recruitment until the required number is achieved.
Additionally, to mitigate the risk of insufficient participants, our collaborators from several Departments of Pathology in Denmark and from the world-renowned International Melanoma Pathology Study Group (IMPSG) have agreed to spearhead this part of the project. With their network, digital distribution, and easy access to participate, we assess that recruitment will not be an issue.
The IT firm, Melatech, with whom we are collaborating, has issued a signed letter of collaboration confirming the free use of the DAHT Platform and their support throughout all phases of the study. Melatech has similarly provided access to the digital educational platform and the data generated during the study period. The Consensus Agreement study, which the PI has also been conducting, is almost done, ensuring we have the diagnoses and classifications we need for each DAHT case as a reference for accuracy in this study.
Under the signed contract between Melatech and our PI, we have provided competent and ongoing IT support throughout the study period, ensuring the study runs smoothly.
Ethics The patients in the DAHT case library were invited to sign a consent form prior to enrollment in this study and informed that their images would be fully anonymized. Their images were cropped, and their CPR numbers were deleted shortly after data extraction, making it impossible to identify the patients and ensuring that any new information gained about a lesion during the DAHT projects would not interfere with the patients' ongoing treatment.
The study has been approved by the Danish Ethics Committee and by the Capital Region of Denmark - Legal Department of Scientific Research By using images and information about a few patients, we may pave the way for improving diagnostic procedures for all future patients.
Perspective The potential consequences of the DAHT study are far-reaching. It is a technological solution that is easy to implement both nationally and internationally. As such, the results may benefit patients worldwide. As melanoma is one of the most common cancers in the world, this impact may be felt worldwide.
The technology to capture high-resolution dermoscopic images digitally and share them between medical specialties and healthcare sectors is currently being implemented in the Danish Healthcare system through Dermloop Capture and is easily transferable to other healthcare settings if the results show a benefit.
Dermloop Capture enables easy, safe documentation and sharing of patient information, and allows healthcare professionals to receive clinical feedback on their diagnostic accuracy and misdiagnosed patients - something that isn't easily and systematically available in any other current setting. Due to the system's two-way communication, implementing such a system will not only help the pathologist diagnose more accurately but also help the clinician diagnose skin lesions more effectively. Clinicians receive systematic, direct feedback from the pathologist while viewing the dermoscopic image of the lesion, making it easier to learn and retain this educational feedback. Studies show that this feedback leads to fewer unnecessary biopsies, as the fear of missing a melanoma is reduced with increasing competence.
For the patient with a skin lesion suspected of being melanoma, sharing information gives the clinician the opportunity to provide the pathologist with the information needed to make an informed decision about a patient's diagnosis and, hence, treatment, as a more accurate diagnosis translates into a more accurate treatment. This will ensure earlier correct diagnosis and treatment of patients with melanoma, as well as less treatment for patients who would previously have been overdiagnosed with a diagnosis of melanoma but who can now more safely be assured and sent home without overtreatment; No wide-margin excision, no removal of their sentinel lymph nodes. Both will benefit the patient and the healthcare system by allocating resources to patients who need them rather than to those who are overdiagnosed. The digital educational platform, Dermloop Learn, is free, available to all users, and can be downloaded to your smartphone within seconds, making it easy to implement internationally. Dermloop Capture is widely implemented in Denmark, with users across the primary sector and in hospital clinical departments. The missing link in the patient journey is providing the pathologists with sufficient information, which we believe we can make possible. If the hypothesis is correct, the implementation of systematic use of dermoscopic images in the primary sector, as well as in surgical and pathological settings, will be an obvious next step at the national and international levels.
Inclusion Criteria:
Exclusion Criteria:
- Assessment of less than 30 DAHT cases
LouisaBoenloekke@gmail.com28356465