Evaluation of a Chatbot With CALMO (Chatbot-Assisted Learning Model) to Enhance Clinical Learning in Nursing Students

Development and Evaluation of a Chatbot With CALMO to Enhance Clinical Learning in Nursing Students: A Quasi-Experimental Mixed-Methods Study

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight.

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The objectives of the proposed study are: (1) Design and develop a chatbot prototype that can simulate conversations, provide clinical information, and offer guidance relevant to various clinical scenarios; (2) Evaluate the impact of the chatbot intervention on nursing students' nursing knowledge, general self-efficacy, and self-reflection and insight; (3) Explore students' perceptions, satisfaction, and acceptance of the chatbot as an educational tool.The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight. The proposed study will be conducted in two phases. Phase 1: Design and develop a chatbot prototype which can provide evidence-based clinical information, engaging in interactive dialogue, and offering learning resources tailored to different clinical scenarios. The chatbot will be accessible through a web-based platform or mobile application. Phase 2: Conduct a mixed-methods quasi-experimental design and focus group interviews to evaluate the impact of the chatbot intervention on clinical learning. A convenience sample of 174 nursing students enrolled in a nursing program from two institutions will be recruited and assigned to either the experimental group, which will receive chatbot-supported clinical learning, or the control group, which will receive traditional clinical education without chatbot integration. Quantitative data will be collected from the two groups at three-time points: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2) to assess students' nursing knowledge, general self-efficacy, and self-reflection and insight. Qualitative data will be collected through focus group interviews to explore students' perceptions, satisfaction, and experiences with chatbot-assisted learning. Quantitative data will be analyzed using MANOVA to compare the outcomes between and within groups. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to student perceptions and experiences using chatbot. Participants in the Experimental Group (EG) will receive an access code for the self-developed web-based chatbot platform called CALMO, a gender-neutral name as an acronym for Chatbot-Assisted Learning Model, which is planned to be developed during the first phase of this proposed study. The intervention will be available throughout the 12-week study.Participants in the Control Group (CG) will receive their usual clinical education support during the intervention period.A project management team will be formed to evaluate the intervention and ensure the quality and consistency of study intervention via regular face-to-face or online meetings. The fidelity of the study will be ensured by developing mechanisms for tracking participant logins, session completion rates, and engagement with intervention materials. Regular checks of the chatbot functionality will be performed by the platform developer with provision of regular service reports. Communication channels will be established for participants to provide feedback, report technical issues, or seek support from the research team during the intervention. Data will be analyzed using a mixed-methods approach, combining both quantitative and qualitative data. The statistical analysis using Statistical Package for the Social Sciences (SPSS), Version 26. The baseline characteristics of the participants in the two arms will be compared to identify any clinically meaningful differences using Chi-square test (for nominal data), Mann-Witney U test (for ordinal or non-normality data) or t-test (for normality data). Any variables considered to be significantly different will be further evaluated (e.g., included as covariates in the statistical analyses) and adjusted to account for their potential confounding effects on the outcomes. To test the hypothesis and to cater for the clustering effect with repeated measures of the outcome variables, a multivariate approach, such as a Generalized Estimating Equation (GEE), with adjustment of the baseline outcome variables to test for between-group differences will be employed. If students drop out between baseline and final data collection, intention to treat analysis will be applied. Both the Time x Group effect will be tested using a two-sided type I error rate of 5% for all tests of significance. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to students' perceptions and experiences.

Inclusion Criteria:

• Baccalaureate nursing students participating in their final clinical placement
• have sufficient knowledge of English language
• own a smartphone with internet access
• possess a valid Hong Kong phone number
• possess a valid email address

Exclusion Criteria:

• those who have been involved in the pilot test of the proposed chatbot intervention