In this evaluation, two versions of emails will be sent to Geisinger employees who have not been vaccinated against COVID-19. A delayed-contact control group will be emailed after a delay of at least two days. The researchers hypothesize that either of the emails - which use content informed by behavioral nudge theory - will lead to more COVID-19 vaccinations than the delayed-contact control group.

The researchers aim to evaluate the effectiveness of email reminders in promoting COVID-19 vaccinations among Geisinger employees. The project will compare two emails, which will vary in the subject line and email content. Both emails will test different approaches that draw from principles in behavioral science: social proof, reframing, and scarcity. Employees receiving these emails will be compared with employees in a delayed-contact group, who will not receive emails. Employees in the control group will later receive one of the emails.The responses to the emails will help determine the communications later sent to the control group.

All emails will ask recipients if they want to receive a COVID-19 vaccination. If they click on a hyperlink indicating yes, they will be forwarded to an online registration and scheduling homepage. If they click on a hyperlink indicating no, they will be forwarded to an online questionnaire surveying people about their main reason for declining the vaccine at this time. The questionnaire will automatically present information intended to assuage the specific concern that people endorse, followed by a hyperlink to the scheduling site.

The data will be analyzed with logistic regression models with the control group as the reference group, to compare the two email conditions versus the control group. This set of analyses will only be conducted for scheduling a vaccination appointment as opposed to email engagement outcomes (e.g., number of emails opened), which will not be applicable for the control group. A second set of logistic regression models predicting scheduling a vaccination appointment and email engagement will also be run, comparing the two email conditions against each other. The data will be examined two days later (to help decide which email to send to the delayed-control group) and four days later (when the policy for employee vaccination will change, making the content of the email irrelevant).

Update: The researchers looked at 3 days of data - as the emails to the delayed contact group were only sent at that time - and 4 days of data when the policy for employee vaccinations changed.

Inclusion Criteria:

• Employees at Geisinger Health System who have not gotten a COVID-19 vaccination

Exclusion Criteria:

• None

Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.

The data will become available after publication of study results in a scientific journal and will be available as long as the Open Science Framework hosts the data.

The data on the Open Science Framework will be open to anyone requesting that information.

The baseline outcome measure was not applicable, since registration for vaccination was the outcome, and all participants were selected for not being registered.

No adverse events were evaluated.

We only had access to email engagement data and registration data, which was kept separate from clinical data.

In the original ClinicalTrials record, the researchers preregistered that the study would examine primary and secondary outcomes after two days. This was extended to about three days, which was the time when the delayed control group was emailed. This altered timing was dictated by when the organization's marketing department was ready to send emails to the delayed control group.

Participants will not know what group they are in. The relevant care provider (either their primary care provider or provider administering the vaccine) will not know about this study or assignment into groups.