Not provided
| ID | Type | Description | Link |
|---|---|---|---|
| P30AG034532 | U.S. NIH Grant/Contract | View source |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Massachusetts Institute of Technology | OTHER |
| National Institute on Aging (NIA) | NIH |
| National Bureau of Economic Research, Inc. | OTHER |
Not provided
Not provided
Not provided
Not provided
The purpose of this study is to assess, prospectively, the effect on flu vaccination rates of salient alerts in the electronic health record that indicate a patient's high risk for flu and its complications. The investigators hypothesize that the salient alerts will lead to increased flu vaccination compared with a standard flu alert.
The CDC (Centers for Disease Control) recommends a flu vaccination to everyone aged 6+ months, with rare exception; almost anyone can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death. One barrier to vaccination is a lack of "cues to action," and, in particular, the lack of direct recommendation from medical personnel; this barrier is arguably the most effectively overcome by a simple nudge of clinicians, compared with barriers such as negative attitudes toward vaccination, low perceived utility of vaccination, and less experience with having received the vaccine.
Geisinger partnered with Medial EarlySign (Medial) to develop a machine learning (ML) algorithm to help identify people at risk for serious flu-associated complications based on existing electronic health record data. Eligible at-risk patients will be randomized to an active control group (clinician will be shown a standard flu alert) or one of two experimental groups (clinician will be shown an alert indicating patient's high risk, with or without describing the patient's factors contributing to that risk).
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Standard Alert | Active Comparator | Standard flu alert |
|
| High-risk Alert | Experimental | Flu alert that indicates patient is at high risk for flu and its complications |
|
| High-risk Alert with Risk Factors | Experimental | Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk |
|
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Alert | Behavioral | Non-interruptive best practice alert in the electronic health record |
|
| Measure | Description | Time Frame |
|---|---|---|
| Flu Vaccination | Patient received a flu vaccine (yes/no) | At the 1 day visit |
Not provided
| Measure | Description | Time Frame |
|---|---|---|
| High Confidence Flu Diagnosis | Patient received a flu diagnosis via a positive polymerase chain reaction [PCR]/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Up to 8 months |
| "Likely Flu" Diagnosis |
Patient Inclusion Criteria:
Clinician Inclusion Criteria:
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Affiliation | Role |
|---|---|---|
| Christopher F Chabris, PhD | Geisinger Clinic | Principal Investigator |
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| Geisinger Clinic | Danville | Pennsylvania | 17822 | United States |
Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.
By the paper's online publication date. Data will remain available for as long as the Open Science Framework hosts it.
The data on the Open Science Framework will be open to anyone requesting that information.
Not provided
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | Standard Alert | Standard flu alert Alert: Non-interruptive best practice alert in the electronic health record |
| FG001 | High-risk Alert | Flu alert that indicates patient is at high risk for flu and its complications Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) |
| FG002 | High-risk Alert With Risk Factors | Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) Risk factors: Alert body indicates the top 3 factors contributing to the high risk |
| FG003 | Clinicians | Geisinger clinician who saw patient-participants in our study for an appointment where their flu shot alert fires |
| Title | Milestones | Reasons Not Completed | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall Study |
|
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | Standard Alert | Standard flu alert Alert: Non-interruptive best practice alert in the electronic health record |
| BG001 | High-risk Alert | Flu alert that indicates patient is at high risk for flu and its complications Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes |
|---|---|---|---|---|---|---|---|---|---|
| Age, Customized | Count of Participants |
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Flu Vaccination | Patient received a flu vaccine (yes/no) | As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study. | Posted | Count of Participants | Participants | At the 1 day visit |
|
Adverse events were not monitored. See description.
This is a minimal risk study, and has received expedited IRB review in keeping with 45 CFR § 46.110. EHR and claims data will report inputs (i.e., the subject's assigned arm) and outcomes (i.e., flu vaccination, flu diagnosis, flu-like symptoms). We therefore did not monitor the data for AE/SAEs.
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | Standard Alert | Standard flu alert Alert: Non-interruptive best practice alert in the electronic health record |
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Gail Rosenbaum | Geisinger health | 570-243-1199 | grosenbaum@geisinger.edu |
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot | Yes | No | No | Study Protocol | Mar 15, 2023 | Mar 26, 2024 | Prot_001.pdf |
| SAP | No | Yes | No | Statistical Analysis Plan | Mar 28, 2023 | Mar 29, 2023 | SAP_000.pdf |
Not provided
| ID | Term |
|---|---|
| D007251 | Influenza, Human |
| D015438 | Health Behavior |
| D001519 | Behavior |
| ID | Term |
|---|---|
| D012141 | Respiratory Tract Infections |
| D007239 | Infections |
| D009976 | Orthomyxoviridae Infections |
| D012327 | RNA Virus Infections |
Not provided
Not provided
| ID | Term |
|---|---|
| D002110 | Caffeine |
| D012307 | Risk Factors |
| ID | Term |
|---|---|
| D014970 | Xanthines |
| D000470 | Alkaloids |
| D006571 | Heterocyclic Compounds |
| D011688 | Purinones |
Not provided
Not provided
Not provided
Not provided
Not provided
Patient-participants will not be explicitly told about the different arms, although clinician-participants will see the different arms and may notice these differences.
| Salient alert features | Behavioral | Larger alert header and body font size, use of different font colors and boldface |
|
| High-risk Text | Behavioral | Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) |
|
| Risk factors | Behavioral | Alert body indicates the top 3 factors contributing to the high risk |
|
Received a "high confidence flu" diagnosis (with positive polymerase chain reaction [PCR]/antigen/molecular test) and/or "likely flu" diagnosis (as assessed via International Classification of Disease [ICD] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) Note that "likely flu" is a superset of the "high confidence flu" diagnoses. |
| Up to 8 months |
| Flu Complications | Diagnosed with flu-related complications (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Up to 8 months |
| ER Visits | Number of ER visits from the patient's message appointment date through July 31, 2023 | Up to 11 months |
| Hospitalizations | Number of hospitalizations from the patient's message appointment date through July 31, 2023 | Up to 11 months |
| COVID-19 Vaccination Rates | Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Up to 8 months |
| Flu Vaccination During the 2022-2023 Season | Patient receives a flu vaccine (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Up to 8 months |
| Flu Vaccination by Sex | Patient received a flu vaccine (yes/no) | At the 1 day visit |
| Flu Vaccination by Race | Patient received a flu vaccine (yes/no) | At the 1 day visit |
| Flu Vaccination by Ethnicity | Patient received a flu vaccine (yes/no) | At the 1 day visit |
| BG002 | High-risk Alert With Risk Factors | Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) Risk factors: Alert body indicates the top 3 factors contributing to the high risk |
| BG003 | Clinicians | Geisinger clinician who saw patient-participants in our study for an appointment where their flu shot alert fires |
| BG004 | Total | Total of all reporting groups |
| Participants |
|
| Sex/Gender, Customized | Count of Participants | Participants |
|
| Ethnicity (NIH/OMB) | Count of Participants | Participants |
|
| Race/Ethnicity, Customized | Note: Clinician race/ethnicity data were combined. Clinicians whose race was reported as "Hispanic or Latino" were marked as "Unknown or Not Reported" in the race baseline measure. | Count of Participants | Participants |
|
| Region of Enrollment | Number | participants |
|
| OG002 | High-risk Alert With Risk Factors | Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) Risk factors: Alert body indicates the top 3 factors contributing to the high risk |
|
|
|
| Other Pre-specified | High Confidence Flu Diagnosis | Patient received a flu diagnosis via a positive polymerase chain reaction [PCR]/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Not Posted | Up to 8 months | Participants |
| Other Pre-specified | "Likely Flu" Diagnosis | Received a "high confidence flu" diagnosis (with positive polymerase chain reaction [PCR]/antigen/molecular test) and/or "likely flu" diagnosis (as assessed via International Classification of Disease [ICD] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) Note that "likely flu" is a superset of the "high confidence flu" diagnoses. | Not Posted | Up to 8 months | Participants |
| Other Pre-specified | Flu Complications | Diagnosed with flu-related complications (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Not Posted | Up to 8 months | Participants |
| Other Pre-specified | ER Visits | Number of ER visits from the patient's message appointment date through July 31, 2023 | Not Posted | Up to 11 months | Participants |
| Other Pre-specified | Hospitalizations | Number of hospitalizations from the patient's message appointment date through July 31, 2023 | Not Posted | Up to 11 months | Participants |
| Other Pre-specified | COVID-19 Vaccination Rates | Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Not Posted | Up to 8 months | Participants |
| Other Pre-specified | Flu Vaccination During the 2022-2023 Season | Patient receives a flu vaccine (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) | Not Posted | Up to 8 months | Participants |
| Other Pre-specified | Flu Vaccination by Sex | Patient received a flu vaccine (yes/no) | As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study. | Posted | Count of Participants | Participants | At the 1 day visit |
|
|
|
| Other Pre-specified | Flu Vaccination by Race | Patient received a flu vaccine (yes/no) | As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study. | Posted | Count of Participants | Participants | At the 1 day visit |
|
|
|
| Other Pre-specified | Flu Vaccination by Ethnicity | Patient received a flu vaccine (yes/no) | As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study. | Posted | Count of Participants | Participants | At the 1 day visit |
|
|
|
| 0 |
| 0 |
| 0 |
| 0 |
| 0 |
| 0 |
| EG001 | High-risk Alert | Flu alert that indicates patient is at high risk for flu and its complications Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) | 0 | 0 | 0 | 0 | 0 | 0 |
| EG002 | High-risk Alert With Risk Factors | Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk Alert: Non-interruptive best practice alert in the electronic health record Salient alert features: Larger alert header and body font size, use of different font colors and boldface High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) Risk factors: Alert body indicates the top 3 factors contributing to the high risk | 0 | 0 | 0 | 0 | 0 | 0 |
Not provided
Not provided
| D014777 | Virus Diseases |
| D012140 | Respiratory Tract Diseases |
| D011687 |
| Purines |
| D006574 | Heterocyclic Compounds, 2-Ring |
| D000072471 | Heterocyclic Compounds, Fused-Ring |
| D012306 | Risk |
| D011336 | Probability |
| D013223 | Statistics as Topic |
| D004812 | Epidemiologic Methods |
| D008919 | Investigative Techniques |
| D015984 | Causality |
| D015981 | Epidemiologic Factors |
| D011787 | Quality of Health Care |
| D017530 | Health Care Quality, Access, and Evaluation |
| D017531 | Health Care Evaluation Mechanisms |
| D011634 | Public Health |
| D004778 | Environment and Public Health |
| Male |
|
|
| Asian |
|
|
| Native Hawaiian or Other Pacific Islander |
|
|
| Black or African American |
|
|
| White |
|
|
| Two or More |
|
|
| Other |
|
|
| Unknown or Not Reported |
|
|
| Not Hispanic or Latino |
|
|
| Unknown or Not Reported |
|
|