Comparing AI Role-Play and Peer Role-Play for Informed Consent Training in Endoscopy: A Randomized Control Trial

COMPARING AI ROLE-PLAY AND PEER ROLE-PLAY FOR INFORMED CONSENT TRAINING IN ENDOSCOPY: A RANDOMIZED CONTROL TRIAL

The goal of this randomized controlled trial is to find out whether AI-based role play is effective in teaching healthcare trainees how to obtain informed consent from patients for endoscopic procedures.

The main question is:

• How does generative AI-based role play compare to traditional peer role play in training healthcare trainees to deliver informed consent for endoscopic procedures?

Researchers will compare AI-based role play with peer role play to see which method is more effective for teaching this skill.

What participants will do:

• Attend a lecture on how to obtain informed consent.
• Take part in simulation-based training using either peer role play or AI-based role play.
• Participate in a simulated clinical encounter to assess their ability to obtain informed consent.

BACKGROUND AND RATIONALE Generative artificial intelligence (AI), particularly large language models (LLMs), has recently enabled the development of interactive, customizable virtual patient simulations that support experiential learning in healthcare communication. These AI-driven platforms can simulate realistic patient encounters, incorporating variable demographics, emotional states, and clinical complexity. They also offer scalable access to individualized feedback and repeated practice, circumventing the logistical limitations of traditional peer or actor-based simulations. Preliminary studies suggest that trainees perceive these tools as high in realism and educational value; however, there remains a lack of empirical evidence directly comparing the educational efficacy of generative AI-based simulations to established pedagogical approaches. This is particularly relevant for teaching complex communication tasks such as obtaining informed consent for gastrointestinal endoscopy-an interaction that demands clarity, empathy, and a detailed explanation of procedural risks, benefits, and alternatives.

STUDY AIM The primary objective of this study is to compare the effectiveness of generative AI-based role play and traditional peer role play in training healthcare trainees to obtain informed consent for endoscopic procedures. Specifically, the study will assess differences in trainee's ability to obtain informed, communication skills in obtaining informed consent and self-reported confidence (self-efficacy) following simulation-based training using a randomized controlled trial (RCT) design.

METHODS Study Design This is a single-blinded, parallel-group randomized control trial. A total of 40 medical trainees will be recruited. Participants will be randomized in a 1:1 ratio to either the intervention group (generative AI-based role-play) or the control group (traditional peer role-play). Randomization will be conducted using a computer-generated sequence, with group assignments placed in sealed envelopes distributed by an independent study team member. Due to the nature of the interventions, participant blinding is not feasible; however, outcome assessors will be blinded to group allocation.

Participants Participants will include medical trainees enrolled in a University of Toronto medical training program. Exclusion criteria include prior experience obtaining informed consent for endoscopy 25 or more times in either clinical or simulated settings.

Baseline Assessment All participants will complete a baseline questionnaire collecting demographic data, self-efficacy in obtaining informed consent, and attitudes toward AI in healthcare education.

Educational Intervention Participants will first attend a standardized 20-minute didactic lecture on the principles and components of informed consent in gastrointestinal endoscopy. The session will cover essential topics such as the purpose and risks of upper and lower endoscopy, alternative diagnostic options, addressing patient concerns, and ethical-legal considerations. This lecture is designed to ensure a uniform knowledge base prior to the simulation-based practice sessions.

Simulation-Based Training

Participants will then engage in 30 minutes of simulation-based practice using their assigned modality:

• Control Group - Peer Role Play: Participants will be paired with fellow trainees and alternate between physician and patient roles. Using standardized case prompts, trainees will conduct informed consent discussions. After each interaction, the participant in the patient role will provide peer feedback based on a provided checklist.
• Intervention Group - Generative AI Role Play: Participants will interact individually with an AI-driven virtual patient interface. The simulation will guide a dynamic informed consent conversation in which the AI patient responds in real-time to trainee input. The platform is programmed to emulate realistic patient personas and provides automated, individualized feedback after each scenario, focusing on content completeness, tone, empathy, and clarity.

The role-play scenarios used in both groups will be matched for content, including clinical context (i.e., informed consent for combined upper endoscopy and colonoscopy), relevant patient history, and the same required communication tasks. This design aims to ensure that differences in outcomes can be attributed to the mode of delivery rather than differences in case complexity.

Post-Intervention Assessment After the simulation sessions, all participants will complete a standardized post-test involving a simulated patient encounter. In this assessment, trainees must obtain informed consent for combined upper endoscopy and colonoscopy from a standardized patient (SP, i.e., actor) following a consistent, scripted patient persona. The SP will be trained to provide consistent responses across participants.

All interactions will be video recorded for assessment purposes, ensuring that identifying facial features are not captured to maintain anonymity. Assessors will be blinded to group allocation.

Follow-Up Survey Immediately following the post-test, participants will complete a follow-up survey reassessing self-efficacy in obtaining informed consent. The intervention group will also complete a feedback questionnaire assessing the usability, perceived educational value, and acceptability of the AI-based platform.

OUTCOMES

• Primary Outcomes:

  • Ability to obtain informed consent during the SP post-test, assessed using a structured checklist (assessors blinded to group allocation)
  • Communication performance during the SP post-test, assessed using a structured global rating scale (assessors blinded to group allocation)

• Secondary Outcomes:

  o Change in self-efficacy in obtaining informed consent (pre- vs. post-intervention survey scores).

• Other Outcomes (intervention group only); o Trainee perceptions of the AI-based simulation (intervention group only), including realism, usability, and satisfaction.

STATISTICAL ANALYSIS Quantitative data will be analyzed using standard statistical methods. Between-group differences in informed consent performance scores and communication scores will be assessed using independent samples t-tests or Mann-Whitney U tests, depending on the distribution of the data. A mixed-design (repeated measures) ANOVA will be used to evaluate pre-post changes in self-efficacy and to test for interaction effects between time and group (AI vs. peer role-play). Qualitative data from open-ended survey responses will be analyzed using thematic analysis following an inductive coding approach.

Inclusion Criteria:

1) physician trainee enrolled at the University of Toronto.

Exclusion Criteria:

1) have previously obtained informed consent for an endoscopic procedure 25 or more times in either real clinical or simulated settings.