This study will test the effectiveness of two interventions (parent-targeted text messages and a brief clinician-targeted training) at increasing HPV vaccination among 11- to 12-year-olds living in Florida.

Random Assignment: The study will test the hypotheses with a doubly randomized, two-level nested design for testing two nested implementation factors. Randomization will occur at two levels: the clinic and patient/parent. A group randomized trial forms the basis of randomization.

The study will randomly assign a fixed number of 30 clinics (and the providers within each clinic) equally to one of three provider trainings: (1) bundled training (n=10 clinics), (2) benefits training (n=10 clinics), and (3) no training (n=10 clinics). All providers in each clinic will be invited to participate in the assigned training. The investigators anticipate an average of 2 providers per clinic. The study will stratify by baseline vaccination rates to ensure balance across the three groups. The investigators chose to assign provider trainings at the clinic level because it is more feasible to conduct one training at each clinic and it limits the potential for contamination between providers.

The second level of randomization will occur among patients/parents. The investigators anticipate between 70-100 adolescents per clinic. The study will randomly assign parents (nested within clinic) equally to one of three text message groups: (1) bundled (n=30), (2) benefits (n=30), and (3) no messages (n=30). Inclusion criteria are: 10 , 11 , and 12-year-olds, attended the clinic in the past two years, and do not have records of changing providers or receiving the adolescent vaccines. Randomization will include 10-year-olds because text messages will be sent just prior to the adolescents' 11th birthday. The study will stratify text message assignment by patient biologic sex because of large differences in HPV vaccination and possible differential effectiveness of text messages. To ensure auditability, separate SAS programs will be written for the two levels of randomization.

Delivery of interventions: At each clinic randomized to training, providers will be invited to participate in a group-based training occurring over Zoom. Each training will last one-hour and providers will receive continuing medical education (CME) credit for participating. Trainings will focus on increasing the providers' intention to recommend the vaccine. Following best practices, each provider training will have a combination of activities (presentation, concrete skills development, role-play, and feedback), will be learner-centered, and engaging. Training will include: (A) presentation of HPV vaccination background; (B) the approach explanation, script, and a video-recorded example of a provider talking to a parent; (C) practicing prepared role play scenarios in small groups; and (D) an explanation of the follow-up, audio-recorded trainings. For part B, the bundled training will include existing nationally endorsed materials.

The study will audio-record thee visits with each provider to give individual feedback, to offer boosters (e.g., script review or barrier discussions) to providers who are lower than 80% adherent, and to evaluate provider adherence to assigned approaches. To increase the trialability, providers will be encouraged to use a standardized form to record their vaccine recommendations with their next ten patients and compare the HPV vaccination rates to their preceding ten patients.

For feasibility of scale-up and dissemination, text messages will be sent from clinics to parents by integrating our message content and timing into the clinic's existing appointment reminder system. The study will restrict recruitment to clinics with appointment reminder systems. If integration is not feasible or preferable to a clinic, study staff will facilitate texting.

Parents will be sent the interactive text messages as part of the clinics' quality improvement efforts. Each clinic will send the text messages from their appointment reminder phone number so parents are aware the messages are from the clinic. Parents who have opted-out of clinic text messages or who did not provide the clinic with a text-enabled phone number will be sent an email or automated voicemail message. Text messages will include an opt-out option.

At the beginning of the study, text messages will be sent to the parents of all 11- to 12-year-olds who have attended each clinic within the past two years and do not have records of changing providers or receiving the adolescent vaccines. Throughout the study year, we will send a text message two weeks prior to the child's 11th birthday. Additional text messages will be sent to parents of 11- to 12-year-olds who have a clinic appointment during the study period at the same time as the texted appointment reminder. Parents who respond to the text message will receive an appointment scheduling call within two business days.

Statistical analysis accounting for clustered design. Throughout the analysis, the statistical team will be blinded to study arm assignments. Initially, descriptive statistics (rates and means with 95% confidence intervals (CIs) will be compared between implementation strategies (brief provider training and interactive text messages) and recommendation approaches (bundled versus benefits).

To achieve all four study aims and evaluate the process measures, the study will account for the clustered data with generalized linear mixed models (GLMMs; e.g., SAS PROC GLIMMIX) rather than generalized estimating equations (GEE) given the relatively small number of clusters (30 clinics) and the two levels of randomization. In all four aims, the primary outcomes are HPV vaccination (initiation and up-to-date): binary variables at the patient level. The GLMMs will assume a binomial distribution and use a log link, modeling risk (i.e., rates) of initiation and up-to-date and estimating rate ratios (RRs) between implementation strategies and recommendation approaches. Staying within the framework of RR will aid analysis of additive interactions (Aim 3). If convergence failure is an issue after exploring avoidance methods, we will use the logit link and make interpretations around odds of initiation and up-to-date.

Aims 1 and 2: Main Effects of Implementation Strategies on Outcomes. The study will test the main effects of our implementation strategies on HPV vaccination rates with the inference of the fixed effects. For inference on the fixed effects, we will use the between-within DDFM. To account for the nested design, we will include random effects for provider and clinic. Initially, the study will include a multiplicative interaction between provider training and parent text messages to examine if one implementation strategy modifies the effect of the other. If not significant (P>=0.05), as hypothesized, the investigators will remove the multiplicative interaction and focus inference on the main effects (Aim 2) and additive interactions (Aim 3). Given the stratified randomization, the investigators will adjust for baseline HPV vaccination rates and patient sex. The investigators will also examine patient sex and race/ethnicity as possible effect modifiers.

The secondary outcomes of interest are provider recommendation frequency (Aim 1) and patient clinic visits (Aim 2). Clinic visits will be analyzed as binary (yes/no) with similar modeling approaches described above. The study will also model number of clinic visits per patient, most likely using a zero-inflated Poisson model (accounting for the clustered data via random effects). The investigators will carefully examine the distribution of the provider recommendation frequency, which is measured as a percentage of patient encounters and will resemble continuous outcomes, to determine the appropriate distribution/link function of the GLMM. The unit of analysis is provider, and therefore, only a random effect of clinic is needed. For Aim 1, the investigators will also perform sensitivity analyses to adjust for provider adherence score.

Aim 3: Additive Interaction of the Implementation Strategies. The investigators will calculate the synergy index (S), which measures the extent to which the RR for concordant provider and parent recommendation approaches together is greater than the sum of the provider approach RR and parent approach RR (i.e., additive interaction). An S > 1 indicates synergy and S < 1 indicates antagonism between the two approaches. Standard error calculations for S will be estimated using standard software/modules based on the delta method. S is a commonly-used measure of additive interaction of treatments/interventions. The investigators will calculate this synergy to test the hypothesis associated with Aim 3.1 (testing the combined effectiveness of any provider and parent intervention), and the investigators will calculate it separately for Aim 3.2 (testing the combined effectiveness of concordant interventions).

Aim 4: Clinical Practice-Level Moderators of Implementation. To determine if the five specified clinical practice factors modify the implementation strategies' effectiveness, we will consider fixed effects for clinical practice factors on HPV vaccination, and assess the specified interactions between clinical practice factors and the implementation strategies.

Process Evaluation Analysis Plan. The study will evaluate the effectiveness of the implementation strategies on the process measures. For the provider training, the study will evaluate changes in the primary targeted construct, provider intent to vaccinate, before, immediately after, and a year after training. Mean levels of provider intent to vaccinate (a composite of Likert-scaled questions) will be estimated and compared across the three time points and across provider training arms via GLMMs as described above. For the interactive text messages, the primary targeted construct of parent salience is measured in a one-time survey of parents in all study arms. Parent salience rates will be estimated (with 95% CIs) and compared across the parent text message arms using a similar modeling approach (with a binary variable/logit link).

Inclusion Criteria:

Adolescents

• Age 10- to 12-years-old
• Visited participating clinic in past two years
• No records of the adolescent vaccines Parents
• Parent of adolescent meeting the above criteria
• Parent of 10- to 12-year-old Providers
• Provide 11 to 12-year-old patients primary care
• Practice at a participating clinic

Exclusion Criteria:

Adolescents

• Out of the age range for participation
• Previously opted-out of text messages
• No cell phone numbers on file at clinic

Parents/Providers

• Unwilling to consent to participation

Due to the possibility of identifying adolescents, providers, or clinic staff, and the study investigator's access to the immunization data only under data sharing agreements, data will be available to users only under a data-sharing agreement. Through shared data agreements, de-identified data and documentation will be available to certified researchers once the main findings from the final dataset are released. Investigators will be charged a nonrefundable fee to cover administrative handling chargers and user support. The data-sharing agreements will include a commitment to: (1) use the data only for research; (2) not attempt to identify individuals; (3) secure the data using appropriate computer technology; (4) destroying the data after analyses are completed; and (5) acknowledge the source of the data.

Once findings from the main study have been released.

Investigators will complete a data-sharing agreement and be charged a nonrefundable fee to cover administrative handling chargers and user support.