Impact of AI Feedback on Ultrasound Biometry Accuracy Across the Expertise Levels

Evaluating the Sensitivity to Change of AI-Feedback in Ultrasound Biometry: A Stratified Randomized Controlled Trial Across the Expertise Gradient

Objective: To evaluate the impact of real-time AI feedback on fetal biometry accuracy and investigate the Expertise Reversal Effect-whether AI benefits diminish as user experience increases.

Design: A stratified randomized trial of 75 participants (25 Novices, 25 Intermediates, 25 Experts). Users are randomized 1:1 to either AI-assisted or manual measurement groups.

Outcomes:

• Primary: EFW accuracy (MAPE) compared to actual birthweight.
• Secondary: Procedure time, image quality, error relative to baseline scans, and cognitive workload (NASA-TLX).

Study Overview: This study evaluates how real-time Artificial Intelligence (AI) feedback impacts the accuracy of fetal biometry measurements in obstetric ultrasound. While AI tools are designed to assist clinicians, their effectiveness may vary depending on the user's baseline skill level-a phenomenon known as the "Expertise Reversal Effect."

Research Aim: The primary objective is to determine if AI-guided feedback significantly reduces measurement error in ultrasound fetal weight estimation to traditional manual methods. The study specifically investigates whether the benefit of AI is greater for novice users, intermediate users users than for experienced specialists.

Study Design: This is a stratified, randomized controlled trial involving 75 participants categorized into three expertise tiers:

Novices (e.g., students or residents with minimal scan experience).

Intermediate Users (e.g., physicians in mid-level training).

Experts (e.g., senior specialists).

Participants within each tier will be randomized 1:1 to either the AI-Assisted Group (receiving real-time automated plane validation and calipers) or the Control Group (performing standard manual biometry).

Primary Outcome Measure: Accuracy of Estimated Fetal Weight (EFW): The Mean Absolute Percentage Error (MAPE) of the EFW relative to the actual birthweight, assessing the clinical impact of AI assistance on weight prediction.

Secondary Outcome Measures:

• Procedural Efficiency: Total procedure time (probe-to-skin) required to complete the biometry.
• Image Quality: Objective assessment of captured planes based on standardized salomon criteria.
• Relative Measurement Error: Deviation of estimated fetal weight when compared to a standard (expert-validated) ultrasound scan.
• Subjective Workload: Evaluation of cognitive load and user effort using the NASA Task Load Index (NASA-TLX).
• Determination of Experience Threshold: Defining the 'cutoff' in clinical experience (years and total scans) for significant AI-mediated accuracy gains.

Clinical Target Population: Healthcare professionals and students, including but not limited to:

• Medical students (doing their masters.
• Resident physicians and Senior Consultants in Obstetrics and Gynecology.

Exclusion:

- If the participants do not understand and speak either Danish or English

Pregnant women:

Inclusion Criteria:

• Pre pregnancy BMI < 40
• Singelton pregnancy
• GA ≥ 37+0 at time of induction
• Intact membranes (to ensure consistent amniotic fluid index)

Exclusion Criteria:

• Major fetal anatomical anomaly
• Anhydramnios (DVP < 2 cm)
• CPR ratio < 2.5th percentile