Evaluation of a Vaccine Chatbot on HPV Vaccine Confidence and Hesitancy

Evaluation of a Vaccine Chatbot on HPV Vaccine Confidence and Hesitancy: a Randomized Controlled Trial and Implementation Science Study

The goal of this clinical trial is to learn if a chatbot powered by artificial intelligence works to improve HPV vaccination among females aged 15 to 45 in China. A randomized controlled trial and implementation science study will be conducted targeting females as participants. The main questions it aims to answer are:

1. Does the vaccine chatbot influence women's confidence, literacy, hesitancy, and uptake of the HPV vaccine.
2. What are the public acceptance of chatbot and the facilitators and barriers to its implementation in a real-world setting.

Researchers will compare a group of women who use the chatbot with a group who do not use it to see if the chatbot is effective at helping women feel confident and willing to get vaccinated against HPV.

Participants will:

1. Be recruited and randomly allocated into one of two groups. One group will be invited to use the HPV vaccine chatbot and the other group will not get access to the vaccine chatbot until the end of the trial.
2. Complete a questionnaire survey on their confidence, literacy, and hesitancy on the HPV vaccine.
3. Have their vaccination status checked at the end of trial.

A randomized controlled trial with 2 arms will be conducted to evaluate the effectiveness of an LLM-powered chatbot on improving HPV vaccination among females aged 15 to 45, followed by a pragmatic implementation science study to assess the public acceptance of chatbot and to identify the facilitators and barriers to its implementation in a real-world setting.

The sample size is calculated based on the primary outcome of the HPV vaccination uptake among females aged 15 to 45 in China. This uptake is around 10% according to the China CDC estimation, and we assume that it can increase to 15% following the intervention. With a significance level (α) of 0.05 and 80% power (1-β=0.8), a sample size of at least 683 per group is needed to detect a 5% absolute difference in vaccination rate (from an assumed baseline rate of 10% to 15% post-intervention). The sample size is inflated to 900 per group to account for potential loss to follow-up and to ensure adequate power for a multi-center design.

Participant recruitment will be conducted using a multistage sampling approach across eastern, central, and western China. In each of three provinces, one urban area and one or two rural area will be selected. Participants meeting the inclusion criteria will be invited to participate and provide informed consent. For those aged 15 to 17, both parental or legal guardian consent and adolescent assent will be obtained. A randomization method will be employed to ensure balanced allocation across two groups, and the randomization sequence will be generated by an independent statistician to ensure a 1:1 allocation ratio. Participants and on-site intervention staff cannot be blinded; personnel responsible for blinded outcome assessment and statistical analysis will remain unaware of allocation.

Participants in the intervention group will be invited to use the HPV vaccine chatbot as well as a routine HPV educational pamphlet . The chatbot powered by a large language model (LLM), offers a scalable, accessible, and interactive platform around the HPV vaccine information. At the recruitment site, participants will access the chatbot through WeChat or a web browser and will be asked to use it for around five minutes. A WeChat group will be established, and the chatbot access link will be sent at one week, half a month, one month, one and half month, two months following the enrollment. The control group will not get access to the vaccine chatbot until the end of the trial and will receive standard care by a routine educational pamphlet about HPV infection and vaccination. Following the intervention, all participants will complete a questionnaire survey on their confidence, literacy, and hesitancy on the HPV vaccine. And their vaccination status will be obtained from official vaccination records at the end of trial.

The intention-to-treat (ITT) analysis will be used to evaluate the effectiveness of the intervention. Between-group comparisons will use t-tests for continuous variables and chi-square tests for categorical variables. HPV vaccine uptake as the primary outcome will be analyzed using log-binomial regression with study center included as a fixed effect and prespecified covariates included for adjustment. Risk ratios and adjusted absolute risk differences with 95% confidence intervals will be reported. If the log-binomial model does not converge, modified Poisson regression with robust sandwich variance estimation will be used. Timing of vaccination will be evaluated using Kaplan-Meier survival curves and log-rank tests, with Cox proportional hazards models to estimate intervention effects on time-to-vaccination. Binary secondary outcomes such as vaccine hesitancy and perceived disease risk will be analyzed using log-binomial or modified Poisson regression, consistent with the primary-outcome analysis; and continuous secondary outcomes such as vaccine confidence and literacy will be analyzed using linear regression models that account for study-center effects and adjust for prespecified baseline covariates. Prespecified subgroup analyses will assess heterogeneity of intervention effects across populations stratified by baseline characteristics. In addition, a per-protocol analysis will be conducted on the primary outcome, and within the intervention group, the effect of intervention intensity measured by chatbot engagement levels on primary outcome will also be evaluated.

Inclusion Criteria:

• Female, aged 15 to 45 years, inclusive.
• Not previously vaccinated against HPV.
• Reports no contraindications to HPV vaccination.
• Has no mental, visual, or reading impairments that would preclude cooperation with study activities.
• Is willing and able to provide informed consent.

Exclusion Criteria:

• Male.
• Age under 15 or over 45 years.
• Has a history of prior HPV vaccination.
• Has a known contraindication to HPV vaccination.
• Unable to comply with study procedures.
• Is unwilling or unable to provide informed consent.