Educational Study of Multimedia in Surgical Skills Training in Colorectal Surgery

The Role of Multimedia in Cognitive Surgical Skill Acquisition in Open and Laparoscopic Colorectal Surgery

To practice independently surgeons require competency in surgical skills, encompassing a combination of technical and non-technical skills. Cognitive skills, aspects of non-technical skills, represent an integral component of surgical competency. Cognitive skills comprise factual knowledge and decision-making.

Changing work patterns in the United Kingdom, as specified by the European Working Time Directive (EWTD), have had a profound impact on the delivery of surgical skills training. Surgical trainees are now increasingly removed from normal working hours in which the majority of traditional operative training and experience is gained, leading to a net reduction in trainees' operative exposure. This reduction in operative experience means that surgical competence can no longer be assured on the basis of experience alone.

Although there is no educational technology that can replace the craft apprenticeship required to train a competent surgeon, reduction in training hours has led to rapid development of educational tools to augment surgical skills training outside the operating room environment. These tools tend to concentrate on technical skills performance without emphasis on cognitive skills.

Trainees in today's era have grown up in a multimedia environment; multimedia is media that uses a combination of text, voiceover, animation and video. Multimedia is an underdeveloped educational resource that can supplement cognitive skills training in operative surgery.

The purpose of this study was to design and develop an online multimedia educational tool in a common colorectal surgery procedure ("Anterior Resection") and determine the effectiveness of this tool in teaching and assessment of cognitive skills.

Study hypothesis: Multimedia learning is equivalent to conventional teaching "Study Day" in improving scores in cognitive surgical skills.

This study involved two main objectives:

1. Develop and design multimedia educational tools for open and laparoscopic anterior resection

   • The multimedia tools were developed in collaboration with a professional multimedia company (Digimed). Initially 31 open and 35 laparoscopic (key-hole) procedures were filmed after informed, written consent was obtained from patients. Key procedural steps/ subtasks were identified and integrated onto navigational interactive interface platforms developed using Adobe® Flash® Professional CS5 10.1.
   • Each subtask was composited into succinct video clips using Final Cut Pro®, supplemented with annotation, animation, text and voiceover. Content addressed cognitive aspects including factual and anatomical knowledge, decision-making points, instrumentation required, team positioning and use of assistance. The multimedia educational tools are available online at following the link http://www.colorectaltraining.co.uk. The colorectal training website has 1.24GB including 69 video clips in the open tool and 67 video clips in the laparoscopic tool.

2. Evaluate the effectiveness of multimedia in teaching/assessment of cognitive surgical skills

   • A randomised control study was conducted to evaluate the effectiveness of online multimedia in comparison to conventional "Study day" teaching in cognitive surgical skills acquisition. Enrolled participants were randomly allocated into either the intervention (Multimedia) or control (Study Day) Group. The methodology of the assessments was online timed (written) assessments to test cognitive surgical skills. All trainees were assessed before and after the study period.
   • Following ethics approval, permission was obtained from surgical training programme directors (TPDs) to recruit trainees into the study. The study was conducted between 6th October 2011 and 23rd December 2011. Trainees invited to participate were contacted by email. A study flyer attachment and hyperlink (http://www.colorectaltraining.co.uk) were provided for trainees to access relevant study information on a 'Participation Information Sheet' and an online consent form. After submission of the consent form, trainees were directed to separate web pages on the secure web-based service containing a Trainee Proforma (to obtain demographic participant information) and Pre-Assessment Test.
   • Participants in the multimedia group were provided with unrestricted access for 30 days and the study day was held at the end of this period for the "Study Day" group. Multimedia group participants were each emailed an individual login and password to access the multimedia tools on the colorectal training website: http://www.colorectaltraining.co.uk. The login details were requested each time the website was accessed.
   • The study day involved a series of detailed interactive lectures covering all the steps of open and laparoscopic anterior resection surgery. The content delivered was identical to that in the multimedia tool. After a period of one week, all study participants were required to complete the online Post-Assessment test. Multimedia group participants emailed a separate hyperlink to complete an online Evaluation form. All completed forms were stored securely for analysis.

3. Sample size

   • Sample size was calculated on the basis of a pilot study of 50 surgical trainees taking part in an anterior resection study day in January 2009. Trainees were asked to complete a pre-study day written assessment, followed by a post-study day assessment immediately after the lectures had been completed. The mean (SD) pre- and post-assessments scores were 12.4 (4.56) and 26.8 (3.99) respectively.
   • The sample size was calculated with the mean change under the assumption that the baseline (pre-assessment) score for both arms is the same. Mean change in score in the control is 14.4 (4.9) and an expected mean change in score of 24.4 (SD 4.9) in the intervention group is assumed. The sample size required to test the hypothesis at a type I error of 5% and power of 80% was calculated (per group) to be 10. So a total of 20 trainees need to be recruited. If the response rate is 20% and assuming a drop off rate of 20% ((20/20%)/ (1-20%)), a total of 125 trainees needed to be approached.

4. Statistical analysis

   • The baseline pre-assessment test ensured comparability of knowledge base and cognitive skills between study groups and helped to assess construct validity of the assessment tool. Basic descriptive statistics include the numbers and percentages for categorical data, mean and standard deviation for normally distributed continuous data; median and range for skewed data was also included. All data were analysed on an intention to treat basis.
   • Differences in assessment scores between multimedia and control (study day) groups were compared using either the unpaired Students' t-test (for normally distributed data) or the Mann Whitney U test (for skewed data). To analyse post-scores versus pre-test scores within the same group, paired t-test (for normally distributed data) or the Mann Whitney U test (for skewed data) was used. Data was analysed using SPSS (statistical package for social sciences version 16.0, Chicago IL). Significance levels were set at P<0.05.

Inclusion Criteria:

• Postgraduate specialist general surgical trainees at Speciality Training Year 3 (ST3) level/ Specialist Registrar (SpR) Year 1 or above

Exclusion Criteria:

• Postgraduate general surgical trainees below ST3 level/ SPR Year 1
• ALL non general surgical postgraduate trainees
• Medical students