Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Telegram Education for Surgical Learning and Application Gamified (TESLA-G) is an online, gamified quizzing platform for surgical education in medical students.The main objective of this pilot study is to assess the feasibility and acceptability of carrying out a full randomised control trial involving TESLA-G.
A pilot randomised controlled trial involving 50 undergraduate medical students will be conducted. They will be randomised into an intervention group and an active control group.
Feasibility will be determined by participant enrollment, retention rate, and quiz completion. Acceptability will be measured quantitatively via a post-intervention learner satisfaction survey and qualitatively via semi-structured interviews. Additionally, participants' scores for pre- and post-intervention knowledge tests will be compared.
Online multiple-choice question (MCQ) quizzes are a well-established summative assessment tool in medical education. However, they suffer from high drop-out rates, attributed to waning user motivation over time. Various studies have shown that incorporation of gamified elements into learning can promote engagement and motivation.
In this pilot study, we will assess the feasibility and acceptability of a novel gamified quizzing platform, Telegram Education for Surgical Learning and Application Gamified (TESLA-G), to determine the possibility of a future larger-scale randomised controlled trial.
This study entails a randomised controlled trial with two arms, an intervention group (TESLA-G) and an active control group (conventional quizzing platform). 50 first to fifth year medical students from Lee Kong Chian School of Medicine, Nanyang Technological University will be randomised into the two arms with a 1:1 allocation ratio, stratified by year of study. Participants will use the assigned quizzing platform to attempt questions on a specific topic (endocrine surgery) over a period of two weeks.
At the end of the study period, several outcomes will be assessed. Feasibility will be determined by participant enrollment, retention rate, and quiz completion. Acceptability will be measured quantitatively via a post-intervention learner satisfaction survey and qualitatively via semi-structured interviews. Additionally, participants' scores for pre- and post-intervention knowledge tests will be compared.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| TESLA-G | Experimental | Participants will be randomised into the two arms with a 1:1 allocation ratio stratified by year of study. There will be 25 participants in this arm. |
|
| Control | Active Comparator | Participants will be randomised into the two arms with a 1:1 allocation ratio stratified by year of study. There will be 25 participants in this arm. |
|
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Gamified online quizzing platform | Other | TESLA-G is a novel gamified quizzing platform designed based on Bloom's taxonomy of learning domains. Questions will be created in blocks, where each block will test a specific topic within a specialty (endocrine surgery has been selected for this study). Each block has 5 questions, and each question corresponds to each level of Bloom's taxonomy and each level in game. For this study, we aim to create 56 blocks of 5 questions, totalling 280 questions. All questions will be created by two board-certified general surgeons and one endocrinologist, and validated by the research team. The aim of the game is for players to get as many points as they can before the timer runs out. Gamification elements include levels, countdown timer, lives, a point multiplier system, leaderboard rankings and a personalised dashboard. Participants in the intervention group will be provided with a link to access TESLA-G, sent from an automated Telegram bot; this access will be provided for 14 days. |
| Measure | Description | Time Frame |
|---|---|---|
| Feasibility of intervention | The feasibility of the intervention will be evaluated quantitatively. These are the objectives:
Achieving all of these goals will prove that it is feasible to conduct a full-scale randomised controlled trial (RCT), while achieving two out of three goals will indicate that it is probably feasible. Achieving less than two goals will suggest that a full-scale RCT is not feasible with the current procedure. | 14 days |
| Acceptability of intervention | Acceptability of the intervention is measured quantitatively via a post-intervention learner satisfaction survey and qualitatively via semi-structured interviews. | Post-intervention (14 days) |
| Measure | Description | Time Frame |
|---|---|---|
| Improvement of surgical knowledge | Scores of pre- and post-intervention quizzes will be compared. The quizzes, which are on endocrine surgery, will be drafted by two board-certified general surgeons and one endocrinologist. All questions will also be validated by the research team. It should be noted however, that the power of study will be inadequate in identifying the comparative effectiveness between the control and intervention groups in terms of improvement of surgical knowledge. Hence the analysis will be primarily conducted to identify any potential adverse effects and increase in the surgical knowledge within each group, and secondarily between groups. This will be done using a confidence interval of 95% using a small effect size of 0.2. |
Not provided
Inclusion Criteria:
Exclusion Criteria:
Not provided
Not provided
Not provided
Not provided
| Name | Role | Phone | Extension | |
|---|---|---|---|---|
| Clement, Luck Khng Chia, MBBS, MS | Contact | +65 6602 2207 | Chia.clement.lk@ktph.com.sg | |
| Matthew, Song Peng Ng, MBBS | Contact | +65 9644 1701 | matthew.rocketmail@gmail.com |
| Name | Affiliation | Role |
|---|---|---|
| Clement, Luck Khng Chia, MBBS, MS | Department of General Surgery, Khoo Teck Puat Hospital, Singapore | Principal Investigator |
| Lorainne Tudor Car, MBBS, PhD | Nanyang Technological University | Principal Investigator |
Not provided
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 34368038 | Background | Douthit NT, Norcini J, Mazuz K, Alkan M, Feuerstein MT, Clarfield AM, Dwolatzky T, Solomonov E, Waksman I, Biswas S. Assessment of Global Health Education: The Role of Multiple-Choice Questions. Front Public Health. 2021 Jul 22;9:640204. doi: 10.3389/fpubh.2021.640204. eCollection 2021. | |
| 32769474 | Background |
| Label | URL |
|---|---|
| Johnson G. Optional online quizzes: College student use and relationship to achievement. Canadian Journal of Learning and Technology/La revue canadienne de l'apprentissage et de la technologie 2006;32 | View source |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
This is an online, pilot randomised controlled trial consisting of two parallel active groups - an intervention group and an active control group. Participants will be randomised into each group with a 1:1 allocation ratio, stratified by year of study.
Not provided
Not provided
Participants will be subjected to partial blinding, where the type of platform (i.e. conventional quizzing platform or TESLA-G) will not be revealed to them. Instead, participants will be informed that one of the two platforms has been randomly assigned to them. In addition, all researchers involved in statistical analysis will be blinded as far as possible. Until the data has been fully analysed, they will not be able to identify the nature of each of the two groups.
A single, non-blinded researcher will be in charge of conveying information to participants. This is to ensure that important instructions on accessing and using the platforms, which are different for each platform, are sent to the participants. This researcher will also be the point-of-contact for the participants' platform-related queries throughout the study.
|
| Conventional quizzing platform | Other | The conventional quizzing platform will be a modified version of TESLA-G with all gamification elements removed. The same set of questions as the gamified version will be used. Upon entering the quiz, a question stem and five choices will be shown. When an option is selected, the right answer along with its explanation will be indicated. The next question will then be sent, and this process repeats until the participant leaves the platform or has answered every question. Questions will be queued in blocks, with each block corresponding to a particular topic in endocrine surgery. Unlike the gamified version, questions within each block will be randomised, regardless of their level on the Bloom's taxonomy. Participants will also not be informed of the level of Bloom's taxonomy for individual questions. The access link to the platform will be sent to participants from an automated Telegram bot, and this access will last for 14 days. |
|
| Pre and post-intervention (14 days) |
| Participant satisfaction | All participants will also complete a post-intervention learner satisfaction survey, in the form of a Likert scale adapted from the System Usability Survey (SUS) (Brooke et al., 1996) and the Student Evaluation of Educational Quality (SEEQ) Questionnaire. The survey will assess if students in the intervention group are more satisfied with their experience than students in the control group. | Post-intervention (14 days) |
| Retention of surgical knowledge | After another 14 days, participants will sit for a follow-up knowledge test, which once again consists of 20 MCQs on endocrine surgery conducted over 30 minutes. At this point, participants in both control and intervention groups will no longer be able to access the learning platforms. The difficulty level of the follow-up test will be similar to that of the post-intervention test as described earlier. The difference in test scores (post-intervention and follow-up) will indicate participant retention of surgical knowledge. | 14 days after intervention |
| St-Onge C, Young M, Renaud JS, Cummings BA, Drescher O, Varpio L. Sound Practices: An Exploratory Study of Building and Monitoring Multiple-Choice Exams at Canadian Undergraduate Medical Education Programs. Acad Med. 2021 Feb 1;96(2):271-277. doi: 10.1097/ACM.0000000000003659. |
| 32789664 | Background | Ryan A, Judd T, Swanson D, Larsen DP, Elliott S, Tzanetos K, Kulasegaram K. Beyond right or wrong: More effective feedback for formative multiple-choice tests. Perspect Med Educ. 2020 Oct;9(5):307-313. doi: 10.1007/s40037-020-00606-z. |
| 31871352 | Background | Yang BW, Razo J, Persky AM. Using Testing as a Learning Tool. Am J Pharm Educ. 2019 Nov;83(9):7324. doi: 10.5688/ajpe7324. |
| 16507066 | Background | Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci. 2006 Mar;17(3):249-55. doi: 10.1111/j.1467-9280.2006.01693.x. |
| 18823514 | Background | Larsen DP, Butler AC, Roediger HL 3rd. Test-enhanced learning in medical education. Med Educ. 2008 Oct;42(10):959-66. doi: 10.1111/j.1365-2923.2008.03124.x. |
| 32865608 | Background | Jud SM, Cupisti S, Frobenius W, Winkler A, Schultheis F, Antoniadis S, Beckmann MW, Heindl F. Introducing multiple-choice questions to promote learning for medical students: effect on exam performance in obstetrics and gynecology. Arch Gynecol Obstet. 2020 Dec;302(6):1401-1406. doi: 10.1007/s00404-020-05758-1. Epub 2020 Aug 31. |
| 29492055 | Background | Ayyub A, Mahboob U. Effectiveness of Test-Enhanced Learning (TEL) in lectures for undergraduate medical students. Pak J Med Sci. 2017 Nov-Dec;33(6):1339-1343. doi: 10.12669/pjms.336.13358. |
| 29390949 | Background | Green ML, Moeller JJ, Spak JM. Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48. Med Teach. 2018 Apr;40(4):337-350. doi: 10.1080/0142159X.2018.1430354. Epub 2018 Feb 1. |
| 31526248 | Background | Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med Educ Online. 2019 Dec;24(1):1666538. doi: 10.1080/10872981.2019.1666538. |
| 29355907 | Background | Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, Tramacere I, Moja L. E-learning for health professionals. Cochrane Database Syst Rev. 2018 Jan 21;1(1):CD011736. doi: 10.1002/14651858.CD011736.pub2. |
| 25999314 | Background | Brame CJ, Biel R. Test-enhanced learning: the potential for testing to promote greater learning in undergraduate science courses. CBE Life Sci Educ. 2015 Summer;14(2):14:es4. doi: 10.1187/cbe.14-11-0208. |
| 25884641 | Background | Mitra NK, Barua A. Effect of online formative assessment on summative performance in integrated musculoskeletal system module. BMC Med Educ. 2015 Mar 3;15:29. doi: 10.1186/s12909-015-0318-1. |
| 24006398 | Background | Orr R, Foster S. Increasing student success using online quizzing in introductory (majors) biology. CBE Life Sci Educ. 2013 Fall;12(3):509-14. doi: 10.1187/cbe.12-10-0183. |
| 17848591 | Background | Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance. Adv Physiol Educ. 2007 Sep;31(3):253-60. doi: 10.1152/advan.00027.2007. |
| 33128662 | Background | van Gaalen AEJ, Brouwer J, Schonrock-Adema J, Bouwkamp-Timmer T, Jaarsma ADC, Georgiadis JR. Gamification of health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2021 May;26(2):683-711. doi: 10.1007/s10459-020-10000-3. Epub 2020 Oct 31. |
| 30920375 | Background | Gentry SV, Gauthier A, L'Estrade Ehrstrom B, Wortley D, Lilienthal A, Tudor Car L, Dauwels-Okutsu S, Nikolaou CK, Zary N, Campbell J, Car J. Serious Gaming and Gamification Education in Health Professions: Systematic Review. J Med Internet Res. 2019 Mar 28;21(3):e12994. doi: 10.2196/12994. |
| 25352673 | Background | Nevin CR, Westfall AO, Rodriguez JM, Dempsey DM, Cherrington A, Roy B, Patel M, Willig JH. Gamification as a tool for enhancing graduate medical education. Postgrad Med J. 2014 Dec;90(1070):685-93. doi: 10.1136/postgradmedj-2013-132486. Epub 2014 Oct 28. |
| 31345202 | Background | Coleman E, O'Connor E. The role of WhatsApp(R) in medical education; a scoping review and instructional design model. BMC Med Educ. 2019 Jul 25;19(1):279. doi: 10.1186/s12909-019-1706-8. |
| 29046774 | Background | Bakshi SG, Bhawalkar P. Role of WhatsApp-based discussions in improving residents' knowledge of post-operative pain management: a pilot study. Korean J Anesthesiol. 2017 Oct;70(5):542-549. doi: 10.4097/kjae.2017.70.5.542. Epub 2017 May 19. |
| 28265446 | Background | Blumenfeld O, Brand R. Real time medical learning using the WhatsApp cellular network: a cross sectional study following the experience of a division's medical officers in the Israel Defense Forces. Disaster Mil Med. 2016 Aug 9;2:12. doi: 10.1186/s40696-016-0022-7. eCollection 2016. |
| 34178316 | Background | Alhalabi N, Salloum R, Aless A, Darjazini Nahas L, Ibrahim N. Messaging apps use in undergraduate medical education: The case of National Medical Unified Examination. Ann Med Surg (Lond). 2021 Jun 5;66:102465. doi: 10.1016/j.amsu.2021.102465. eCollection 2021 Jun. |
| 32742059 | Background | Iqbal MZ, Alradhi HI, Alhumaidi AA, Alshaikh KH, AlObaid AM, Alhashim MT, AlSheikh MH. Telegram as a Tool to Supplement Online Medical Education During COVID-19 Crisis. Acta Inform Med. 2020 Jun;28(2):94-97. doi: 10.5455/aim.2020.28.94-97. |
| 34593508 | Background | Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, Boyd KA, Craig N, French DP, McIntosh E, Petticrew M, Rycroft-Malone J, White M, Moore L. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021 Sep 30;374:n2061. doi: 10.1136/bmj.n2061. |
| 32075641 | Background | Blanie A, Amorim MA, Benhamou D. Comparative value of a simulation by gaming and a traditional teaching method to improve clinical reasoning skills necessary to detect patient deterioration: a randomized study in nursing students. BMC Med Educ. 2020 Feb 19;20(1):53. doi: 10.1186/s12909-020-1939-6. |
| 30325894 | Background | Plana NM, Rifkin WJ, Kantar RS, David JA, Maliha SG, Farber SJ, Staffenberg DA, Grayson BH, Diaz-Siso JR, Flores RL. A Prospective, Randomized, Blinded Trial Comparing Digital Simulation to Textbook for Cleft Surgery Education. Plast Reconstr Surg. 2019 Jan;143(1):202-209. doi: 10.1097/PRS.0000000000005093. |
| 29403314 | Background | Bell ML, Whitehead AL, Julious SA. Guidance for using pilot studies to inform the design of intervention trials with continuous outcomes. Clin Epidemiol. 2018 Jan 18;10:153-157. doi: 10.2147/CLEP.S146397. eCollection 2018. |
| 26092476 | Background | Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016 Jun;25(3):1057-73. doi: 10.1177/0962280215588241. Epub 2015 Jun 19. |
| 34429945 | Background | Tuma F, Nassar AK. Applying Bloom's taxonomy in clinical surgery: Practical examples. Ann Med Surg (Lond). 2021 Aug 5;69:102656. doi: 10.1016/j.amsu.2021.102656. eCollection 2021 Sep. |
| 37380211 | Derived | Ng MSP, Jabir AI, Ng TR, Ang YI, Chia JL, Tan DNH, Lee J, Mahendran DCJ, Tudor Car L, Chia CLK. Evaluating TESLA-G, a gamified, telegram-delivered, quizzing platform for surgical education in medical students: protocol for a pilot randomised controlled trial. BMJ Open. 2023 Jun 28;13(6):e068740. doi: 10.1136/bmjopen-2022-068740. |
| Deterding S, Khaled R, Nacke LE. Gamification: Toward a definition. CHI 2011 gamification Published Online First: 2011. | View source |
| Morillas Barrio C, Munoz-Organero M, Sanchez Soriano J. Can gamification improve the benefits of student response systems in learning? An experimental study. IEEE Trans Emerg Top Comput 2016;4:429-38. | View source |
| Licorish SA, Owen HE, Daniel B, et al. Students' perception of Kahoot!'s influence on teaching and learning. Research and Practice in Technology Enhanced Learning 2018;13:1-23. | View source |
| Top Apps Worldwide for January 2021 by Downloads. (accessed 23 May 2022) | View source |
| Most popular messaging apps. Statista. (accessed 23 May 2022). | View source |
| Brooke J. Sus: a 'quick and dirty' usability scale. Usability evaluation in industry Published Online First: 1996. | View source |
| Soon MKS, Martinengo L, Lu J, et al. Telegram Education for Surgical Learning and Application (TESLA): An exploratory study. JMIR Preprints. 2021. | View source |
| Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann 2016. | View source |
| Grammatikopoulos V, Linardakis M, Gregoriadis A, et al. Assessing the Students' Evaluations of Educational Quality (SEEQ) questionnaire in Greek higher education. High Educ 2015;70:395-408. | View source |
| Orgill BD, Nolin J. Learning Taxonomies in Medical Simulation. In: StatPearls. Treasure Island (FL): : StatPearls Publishing 2022. | View source |