Unable to recruit enough sample size for the RCT

The approval and distribution of COVID-19 vaccines is an important milestone in the fight against the pandemic. However, although vaccines are widely recognized as a key public health measure to control infectious disease, there are still people who remain hesitant to get vaccinated. In Canada, 83% of Canadian adults have received or are willing to receive the vaccine. However, nearly 20% of adults and a larger proportion of parents of younger children remain concerned about receiving vaccines. In the context of a project to enable family physicians to identify vaccine-hesitant patients and deliver a tailored COVID-19 vaccination outreach campaign, the Canadian Practice Information Network, a tool for primary care providers to communicate with their patients, will be used to engage and address vaccination hesitancy in Canada. The study will identify information needs about COVID-19 vaccination among community-dwelling patients in three Canadian provinces and will assess the effectiveness of a tailored COVID-19 vaccine digital health communication strategy for primary care practices to address vaccination hesitancy and improve COVID-19 vaccination uptake.

A two-arm cluster randomized trial design will be conducted in primary care practices in three Canadian provinces: Ontario, British Columbia, and New Brunswick. The study will use a convenience sample of primary care practice in these provinces. Recruited participants will be randomly allocated to the intervention and control groups. Patients assigned to the control group will receive health messages unrelated to the COVID-19.

Participants in the intervention group will receive a series of tailored messages on COVID-19 vaccination targeting the main factors for hesitancy to address their concerns and persuade them to receive the COVID-19 vaccine. Aggregated survey responses across practices will be used to create segments of unvaccinated patients reflecting age, language, education level, rurality, sex, gender, ethnicity, and their attitudes or reasons for vaccine hesitancy or access barriers. Then, tailored messages will be created in a way that is meaningful to the recipients in the different segments.

The primary outcome is the proportion of vaccine-hesitant individuals who receive a COVID-19 vaccine during the intervention period. The secondary outcome is the overall COVID-19 vaccine uptake.

Background. The approval and distribution of COVID-19 vaccines is a key milestone in the fight against the pandemic. As many countries around have rapidly implemented complex vaccination programs, there are still people who remain hesitant to get vaccinated. In Canada, 83% of Canadian adults have received at least one dose of a COVID-19 vaccine, but nearly 20% of adults remain concerned about receiving vaccines. Canadian primary care providers have not had a large role to date in the administration of COVID-19 vaccines, despite their traditional role in vaccinating infants and seniors. Nonetheless, primary care providers can play a key role in decreasing vaccine hesitancy, as they are one of the most trusted emissaries of people who are vaccine-hesitant. The Canadian Practice Information Network (CPIN), a primary care practice tool, can be used by primary care providers to send messages to patients across small independent primary care practices to improve uptake of COVID-19 vaccines in Canada. Because of their trusted relationship, messages sent by the patient's primary care provider are read and 60% of CPIN surveys completed,9 making this an efficient tool for patient engagement.

An experimental study will be conducted using CPIN as a tool for primary care providers across Canada to communicate and engage with their patients regarding COVID-19 vaccines to identify and address vaccination hesitancy in their own patient populations. A survey will be conducted to explore the reasons for vaccine hesitancy among the unvaccinated population of each participating provider as well as their unmet information needs about COVID-19 vaccination. This will then enable family physicians to implement a targeted and tailored COVID-19 vaccine digital health communication strategy reaching patients by e-mail or text to address vaccination hesitancy and improve COVID-19 vaccination uptake. The effectiveness of this communication strategy in persuading patients to get vaccinated against COVID-19 will be evaluated.

Trial design. A two-arm, pragmatic cluster randomized trial design will be used.

Setting and sample. The study will be conducted in primary care practices across Canada. The study will occur in a convenience sample of any model of comprehensive primary care practices. All primary care providers who provide comprehensive care for a defined patient panel that represents at least 75% of their patient population and who have been with the practice for > 2 years are eligible to participate including those who work part-time. Providers are considered to be in the same practice if they share at least four of the following: patient electronic medical record, waiting room, receptionist, coverage after hours, telephone number, and address.

Through CPIN, 300 providers across 100 Canadian practices will be used for connecting patients and physicians. It is assumed that each provider will care for on average 800 adult patients and that 20% are not yet vaccinated. CPIN will be used to send a questionnaire to at least 180,000 adult patients with email or cell number on record, to recruit participants. The questionnaire will ask patients about COVID-19 vaccination status and intention and, if not vaccinated, their reasons for vaccine hesitancy based on the Health Belief Model, or barriers to access. The survey responses will be aggregated across practices and use machine learning to classify the unvaccinated patients into distinct segments reflecting age, language, education level, rurality, sex, gender, ethnicity, reasons for vaccine hesitancy, and barriers to access.

Recruitment and randomization. Participants will be recruited among non-vaccinated patients from participating primary care (PC) practices in the three provinces. Participating patients will be the unit of randomization. Inclusion criteria are patients registered with a participating provider, aged 18 years or older, of either sex, in a participating primary care practice, who speak one of Canada's official languages (English and French), who have not received a COVID-19 vaccine and agree to participate in the study. A returned completed questionnaire will be interpreted as voluntary participation in the study. Recruited participants will be randomly allocated to the intervention and control groups. Patients assigned to the control group will receive health messages unrelated to the COVID-19. Rolling recruitment of patients from each practice will be spread over a four-month period.

Primary outcome. The principal outcome is the proportion of vaccine-hesitant individuals who receive a COVID-19 vaccine during the intervention period. The COVID-19 vaccination status will be patient-reported on the questionnaires sent by CPIN. Secondary outcomes will be a measure of patient willingness to receive a COVID-19 vaccine. The mean patient ratings will be used to measure the performance of different messages. All outcomes will be reported by practice population segments and shared with participating practices.

Power and Sample size. Based on a conservative estimate informed by previous studies, it is expected our intervention to lead to at least a 4% relative increase in the vaccination rate within the experimental group. Considering a 3:1 allocation ratio to intervention and control groups, an intracluster correlation (ICC)=0.05 to adjust for clustering at the level of the provider and practice, a minimum sample size of 4,320 individuals would be required to detect at least 4% difference between the intervention and control groups with an alpha=0.05 and power=90% (using n4props function of the CRTSize R package).

Qualitative data collection. Applying a mixed-method approach, online card sorting and focus groups will be used to assist in designing and evaluating our tailored health messages. Message usefulness, visual engagement, and persuasiveness will be assessed through the card sorting method and the use of personas.

Patient focus groups will be conducted following the initial survey and at the end of the intervention. The main purpose of the focus groups is to provide qualitative feedback on the usefulness of the health messages designed and the meaningful use of various visual designs from the patients' perspective. In so doing, a user-centered design approach will be adopted. This approach aims at involving our participants in the design process. A total of approximately 20 patients will be divided into focus groups of 4-5 participants at each of the initial and final time points for a total of 40 participants. Participant feedback will be elicited in a rating question after each message sent, and this will be considered in the content and visual design of each subsequent message to strengthen the message tailoring to the needs of the targeted patient population segment who will receive it.

Study intervention. Adult patients from practices recruited across the country will be sent an initial message introducing the CPIN service. A survey link will seek patients' informed consent and if given, patients will immediately be asked about their vaccination status, attitudes, and intentions around the COVID-19 vaccines, minority group membership, education level, and preferred language of health communication. The survey responses will be aggregated across practices and create up to ten separate segments of unvaccinated patients reflecting age, language, education level, rurality, sex, gender, ethnicity, and most importantly, their attitudes or reasons for vaccine hesitancy or access barriers. Then message content will be created in a way that is meaningful to the recipients in the different segments.

The Eastern Ontario Health Unit (EOHU) will work with the Communications Lab at the University of Ottawa to identify major controversies emerging in public discourse that might feed vaccine hesitancy and advise the EOHU to tailor our messages accordingly. Then, evidence-based messages will be produced, tailored to specific population segments based on demographic and behavioral or belief variables and adapt the way the content is presented based on feedback from patients in those segments. The data feedback across practices will be also aggregated to evaluate the performance of our tailored communication strategy in near real-time and adjust it as needed.

Statistical analysis. Once all participants have been randomized, descriptive statistics will be used to compare the two groups at baseline. All analyses will be on an intention-to-treat basis and will use individual participants as the unit of analysis. Descriptive analyses will be performed for selected patient demographic characteristics (age, sex, education), as well as by population segments. Chi-square tests will be used to examine whether individual characteristics differ between the intervention and control arms.

The proportion of individuals who were vaccinated within each intervention arm will be compared, overall and by population segment. Similarly, the proportion of individuals with strong, moderate, and low hesitancy to be vaccinated within each intervention arm will be compared, overall and by population segment and sociodemographic characteristics. Analysis of differences in the proportions of the main endpoint, vaccination status against COVID-19, as well as willingness to receive the vaccine, between the two arms will be performed using 2-sided chi-square tests. Asymptotic confidence limits on the differences and relative rate ratios will be calculated to assess the magnitude of the effect on the changes in the hesitancy and vaccination rates pre and post-intervention.

To determine which patient and practice characteristics are related to the likelihood for hesitancy while accounting for the clustered nature of the data, two-level generalized linear mixed modeling will be conducted using the willingness to be vaccinated (hesitant versus not hesitant) as the binary outcome variable. The Level 1 independent variables will be age, ethnic group, income level, and rurality of residence. The Level 2 independent variables will be the pre-intervention vaccination rate calculated to account for sociodemographic characteristics and intervention arm (Intervention versus Control).

To examine the variations of the results at individual and practice levels a multilevel logistic regression model will be fit to assess the individual and practice level changes for the hesitancy status and the attitudes and reasons for hesitancy. Subgroup analyses will be conducted to determine whether there is any significant heterogeneity in treatment effects occurring between population segments defined by different gender, age group, education level, and ethnic group.

Qualitative analysis. The overall goal of the analytical process of the focus group is a) to gathering information and explanation regarding the usefulness, persuasiveness, and visual engagement of the health messages, and b) to identify key requirements for designing tailored health messages. The data will be analyzed in two steps. First, a thematic analysis of the transcripts of the focus group will be done via NVivo software. The thematic analysis will be performed in four stages: data immersion, coding, identifying emerging sub-themes, and data reduction. Themes, sub-themes, and quotes will be tabulated, summarized, and shared between the team of researchers for validation during coordination meetings (once a month). To ensure the quality and validity of the thematic analysis, the initial analysis will be carried out by the researcher who had done the focus groups and will be cross-checked for achieving intercoder reliability. Secondly, the findings from the thematic analysis will be imported into XmindPro8 software and interactively, organized to identify conceptual groupings and similarities and differences.

Inclusion Criteria:

• Non-vaccinated adults
• Enrolled in a participating CPIN primary care practice

Exclusion Criteria:

• Patients who don't speak one of Canada's official languages (English and French)