This study will test the relative efficacy of high-risk messages in increasing flu shot rates in patients at moderately high risk for flu and complications (those in the top 11-20% of risk). It will also examine whether informing patients that their high-risk status was determined by analyzing their medical records or by an artificial intelligence (AI) / machine-learning (ML) algorithm analyzing their medical records will affect the likelihood of receiving a flu vaccine.

Almost everyone age 6 months or older can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death by reducing the likelihood of contracting influenza. Flu shots are particularly important for patients at high risk of experiencing severe outcomes.

In the 2020-21 and 2021-22 flu seasons, the study team sent messages to Geisinger patients in the top 10% of risk for flu and complications according to an artificial intelligence algorithm. Messages that disclosed patients' risk status significantly increased flu vaccination rates. Additionally, messages that included risk information were most effective in patients at relatively lower risk (those in the top 4-10%) compared with those at the highest risk (top 3%).

The present work will test the effectiveness of high-risk messages in patients who are in the top 11-20% of risk, at high risk but lower than previous studies. These communications will inform patients they are at high risk with either (a) no additional explanation, (b) an explanation that this determination comes from an analysis of their medical records, or (c) the additional explanation that an AI or ML algorithm made this determination.

Inclusion Criteria:

• Included on a list of active Geisinger patients (all patients on this list attended at least one primary care appointment at Geisinger between 10/1/2008 and 4/13/2022, and either had a Geisinger primary care provider assigned as of April 2022, or were in the Electronic Health Record [EHR] since at least September 2021 and had at least one encounter in 2020-2022)
• Aged 18 or older
• In the top 11-20% of risk for flu and flu complications, according to Medial's flu complications machine learning algorithm (which operates on coded EHR data)
• Has a Geisinger PCP assigned as of August 2022
• Has had an encounter in the last 2 years as of August 2022

Exclusion criteria:

- Cannot be contacted via any of the communication modalities (e.g., letter, patient portal, SMS) being used in the study, either due to insufficient/missing contact information in the EHR or because they opted out of all modalities

Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.

The data will become available after publication of study results in a scientific journal and will be available as long as the Open Science Framework hosts the data.

The data on the Open Science Framework will be open to anyone requesting that information.

Providers who prescribe vaccination and diagnose conditions will not be randomized to study arms or informed of patient assignment. Although patients will not be explicitly informed which arm they have been randomized to, they will be aware of the messages they receive.