Not provided
| ID | Type | Description | Link |
|---|---|---|---|
| R01LM013606 | U.S. NIH Grant/Contract | View source |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Mayo Clinic | OTHER |
| University of Pittsburgh Medical Center | OTHER |
| MedStar Health | OTHER |
| National Library of Medicine (NLM) |
Not provided
Not provided
Not provided
Not provided
This study aims to test the efficacy of an information visualization dashboard on decision-making using a randomized controlled trial with crossover.
This study aims to test the efficacy of using an information visualization dashboard on ICU providers' decision-making, efficiency, and performance compared to their institutional EHR through a randomized control trial with crossover.
Electronic Health Records (EHRs) are a major source of data for ICU clinicians. Synthesizing complex, electronic patient data is key to effective care delivery. EHRs contain both a record of past medical data and present continuous flows of new clinical data from various sources such as physiological inputs, laboratory results, imaging studies, and clinician notes. This complex, continuous stream of patient data can contribute to information overload, which can create barriers to the key cognitive tasks of data identification, extraction, and interpretation. Intensive Care Unit (ICU) providers must quickly synthesize data from more than 200 variables during critical care rounds, with critically ill patients generating a median of 1348 individual data points per day. Information overload has been identified as a key factor in the misinterpretation of data, leading to medical errors such as misdiagnoses. Improving our understanding of information overload-and investigations into new efforts to minimize it-can improve clinician workflow and productivity as well as patient safety. The objective of this study is to explore the impact of current data representation in the EHR on ICU clinicians' cognitive workload, performance, and satisfaction. The research design uses a mixed methods approach, including eye-tracking assessment and surveys, to assess the efficacy of current EHR interfaces for ICU clinicians in live and simulated environments.
The investigators will randomize consented participants into two groups: the control (EHR) group and the intervention (AWARE) group. During the Randomized Controlled Trial (RCT) crossover, providers in each group will review the same patient records and will perform the same tasks, and complete the same survey instruments. Providers in the control groups will review two patient cases in their institution EHR (Epic or Cerner) first and then, two new cases in AWARE, and providers in the intervention group will review two patient cases in AWARE and then two new cases in the EHR. Cases will be randomized to eliminate order bias and selection bias.
For the cases in the EHR, participants will navigate through the EHR as per their usual routine in the ICU, with no added training sessions before the study. For the cases in AWARE, participants will receive a short training presentation by the study team, explaining the functionality and design of AWARE. Also, a day before each session, the RA will send an email to the provider with a short demonstration video of AWARE to become familiar with the tool.
The study will be conducted in simulation or Biobehavioral labs at each site. The PI, or research assistant, will explain the study procedure and obtain consent; participants will be asked to use Tobii Pro Fusion during the session, participants will not need to wear anything, the eye-tracker will be mounted to the monitor. During the session, participants will review 2 patient cases in their institutional EHR and two in AWARE. After the participant completes the patient record review, the research assistant (RA) will ask the participant a series of decision-based Q&A activities requiring verbal responses or task completion in the EHR or AWARE. The provider may use the EHR or AWARE to complete the Q&A activity. Cases will be in random order for each participant to avoid selection or order bias. After usability testing, the investigators will ask the participant to fill out the NASA-TLX survey and the System Usability Scale (SUS). The NASA-TLX measures the perceived workload of using the EHR, and the SUS measures the level of satisfaction as a result of using the EHR. All sessions will be recorded.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| EHR (Control), then AWARE | Experimental | In this hour-long task, participants in this arm will first review and complete tasks using the institutional EHR. Then after an approximate 5 minute rest, participants will then review and complete tasks using AWARE. |
|
| AWARE Intervention, then EHR (Control) | Experimental | In this hour-long task, participants in this arm will first review and complete tasks using the AWARE intervention. Then after an approximate 5 minute rest, participants will then review and complete tasks using the institutional EHR. |
|
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| AWARE | Device | AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. |
| Measure | Description | Time Frame |
|---|---|---|
| Time to Task Completion | To assess the amount of time required to make a decision (i.e., complete a task), participants will be asked to find the answers to clinical questions and the average amount of time needed to answer will be reported. | Duration of Single Usability Session, up to 25 minutes |
| Overall Mean Task Scores | To measure performance, participants will be scored on the overall average responses to 8 questions using the following: 0 points for an incorrect response, 0.5 points for a partially correct response, and 1 point for correct responses. Overall mean score for the 8 questions combined is reported. | Duration of Single Usability Session, up to 25 minutes |
| Time to Case Completion | To measure efficiency, participants' time to complete a patient case (chart review and tasks) will be recorded and reported in minutes. | Duration of Single Usability Session, up to 25 minutes |
| Number of Clicks Per Case | To measure efficiency, participants' number of mouse clicks to complete case review and tasks will be recorded. | Duration of Single Usability Session, up to 25 minutes |
| Mean Number of Fatigue Instances Per Case | Physiologic fatigue will be measured using an eye tracking device to capture changes in pupil size (which represents cognitive fatigue). A fatigue instance was recorded whenever the participant's pupil diameter dropped below the subject-specific fatigue threshold. For every provider-interface combination, the investigators calculated the mean and standard deviation of these one-second pupil diameter values and defined the subject-specific fatigue threshold as the individual mean minus 1.5 standard deviations. | Duration of Single Usability Session, up to 25 minutes |
| Measure | Description | Time Frame |
|---|---|---|
| Perceived Workload Score | The NASA-Task Load Index (TLX) measures perceived workload using 6 subscale questions. Scores are reported as an overall total ranging from a minimum of 0 to a maximum of 100. Higher scores indicate higher workload (worse). It is expected to take 1-2 minutes to complete this index. | Immediately following each intervention, within approximately 1-2 minutes |
Not provided
Inclusion Criteria:
ICU physicians and advanced practice providers (APPs),
Residents
Combined Exclusion Criteria:
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Affiliation | Role |
|---|---|---|
| Saif Khairat, PhD | University of North Carolina, Chapel Hill | Principal Investigator |
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| University of North Carolina at Chapel Hill | Chapel Hill | North Carolina | 27514 | United States |
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 40925597 | Derived | Khairat S, Morelli J, Boynton MH, Bice T, Gold JA, Carson SS. Investigating Information Visualization to Combat Information Overload in Electronic Health Records: Protocol for a Randomized Controlled Trial. JMIR Res Protoc. 2025 Sep 9;14:e74247. doi: 10.2196/74247. |
Not provided
Not provided
Deidentified individual data that supports the results will be shared beginning 12 to 24 months following publication provided the investigator who proposes to use the data has a methodologically sound proposal, approval from an Institutional Review Board (IRB), Independent Ethics Committee (IEC), or Research Ethics Board (REB), as applicable, and executes a data use/sharing agreement with UNC.
Not provided
beginning 12 months after and continuing for 24 months following publication
Investigator has IRB, IEC, or REB approved methodologically sound proposal and an executed data use/sharing agreement with UNC.
Not provided
A total of 113 participants signed informed consent; however, 1 participant who signed consent was not randomized
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | EHR (Control), Then AWARE | In this hour-long task, participants in this arm will first review and complete tasks using the institutional EHR. Then after an approximate 5 minute rest, participants will then review and complete tasks using AWARE. AWARE: AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. Electronic Health Record (EHR): An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. |
| Title | Milestones | Reasons Not Completed | |||||
|---|---|---|---|---|---|---|---|
| First Intervention Session (25 Minutes ) |
|
Not provided
Not provided
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot_SAP | Yes | Yes | No | Study Protocol and Statistical Analysis Plan | Mar 21, 2026 |
Not provided
Not provided
| NIH |
Participants will be randomly assigned to a control or an intervention and then conduct a crossover stage. For example, a participant who is in the control group (EHR) will review two patient cases in the EHR. Then, the participant will review two other patient cases in AWARE (intervention), i.e., the crossover stage. Similarly, a participant who is in the intervention group (AWARE) will review two patient cases in AWARE. Then, the participant will review two other patient cases in the EHR (control), i.e., the crossover stage.
Not provided
Not provided
Not provided
| Electronic Health Record (EHR) | Device | An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. |
|
| System Usability Scale Survey Score | Perceived satisfaction measured using the System Usability Scale survey. Values range from a minimum of 0 to a maximum of 10. Higher scores indicate higher satisfaction (better). The overall score is reported and ranges from 0 to 100. It is expected to take 2-3 minutes to complete this survey. | Immediately following each intervention, within approximately 2-3 minutes |
| Number of Eye Fixations on Screen | Information seeking load measures the amount of information being processed by participants using an eye-tracking device to measure fixation points (which represents thought-processing). | Duration of Single Usability Session, up to 25 minutes |
| FG001 | AWARE Intervention, Then EHR (Control) | In this hour-long task, participants in this arm will first review and complete tasks using the AWARE intervention. Then after an approximate 5 minute rest, participants will then review and complete tasks using the institutional EHR. AWARE: AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. Electronic Health Record (EHR): An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. |
| COMPLETED |
|
| NOT COMPLETED |
|
| Time Between Interventions (5 Minutes) |
|
| Second Intervention Session (25 Minutes) |
|
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | EHR (Control), Then AWARE | In this hour-long task, participants in this arm will first review and complete tasks using the institutional EHR. Then after an approximate 5 minute rest, participants will then review and complete tasks using AWARE. AWARE: AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. Electronic Health Record (EHR): An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. |
| BG001 | AWARE Intervention, Then EHR (Control) | In this hour-long task, participants in this arm will first review and complete tasks using the AWARE intervention. Then after an approximate 5 minute rest, participants will then review and complete tasks using the institutional EHR. AWARE: AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. Electronic Health Record (EHR): An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. |
| BG002 | Total | Total of all reporting groups |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Age, Continuous | Mean | Standard Deviation | years |
| |||||||||||||||
| Sex: Female, Male | Count of Participants | Participants |
| ||||||||||||||||
| Race and Ethnicity Not Collected | Race and Ethnicity were not collected from any participant. | Count of Participants | Participants |
| |||||||||||||||
| Region of Enrollment | Count of Participants | Participants |
| ||||||||||||||||
| Stress Score | Participants will be asked to rate their current level of stress based on pictorial images on a numbered scale ranging from 0 (None) to 10 (Worst possible). Higher scores indicate a greater amount of stress. | Mean | Standard Deviation | scores on a scale |
| ||||||||||||||
| Sleepiness Score | Participants will be asked to rate their current level of sleepiness based on pictorial images on a numbered scale ranging from 0 (Wide awake, alert) to 10 (Sleep onset soon, having dreamlike thoughts). Higher scores reflect greater sleepiness. | Mean | Standard Deviation | scores on a scale |
|
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Time to Task Completion | To assess the amount of time required to make a decision (i.e., complete a task), participants will be asked to find the answers to clinical questions and the average amount of time needed to answer will be reported. | Due to computer file corruption for 2 participants in the EHR Arm/Group and 1 participant in the AWARE Arm/Group, data were lost. | Posted | Mean | Standard Deviation | minutes | Duration of Single Usability Session, up to 25 minutes |
|
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Primary | Overall Mean Task Scores | To measure performance, participants will be scored on the overall average responses to 8 questions using the following: 0 points for an incorrect response, 0.5 points for a partially correct response, and 1 point for correct responses. Overall mean score for the 8 questions combined is reported. | Due to computer file corruption for 1 participant in the EHR Arm/Group, data were lost. | Posted | Mean | Standard Deviation | score on a scale | Duration of Single Usability Session, up to 25 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Primary | Time to Case Completion | To measure efficiency, participants' time to complete a patient case (chart review and tasks) will be recorded and reported in minutes. | Due to computer file corruption for 2 participants in the EHR Arm/Group and 1 participant in the AWARE Arm/Group, data were lost. | Posted | Mean | Standard Deviation | minutes | Duration of Single Usability Session, up to 25 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Primary | Number of Clicks Per Case | To measure efficiency, participants' number of mouse clicks to complete case review and tasks will be recorded. | Due to computer file corruption for 2 participants in the EHR Arm/Group and 1 participant in the AWARE Arm/Group, data were lost. | Posted | Mean | Standard Deviation | clicks/case | Duration of Single Usability Session, up to 25 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Primary | Mean Number of Fatigue Instances Per Case | Physiologic fatigue will be measured using an eye tracking device to capture changes in pupil size (which represents cognitive fatigue). A fatigue instance was recorded whenever the participant's pupil diameter dropped below the subject-specific fatigue threshold. For every provider-interface combination, the investigators calculated the mean and standard deviation of these one-second pupil diameter values and defined the subject-specific fatigue threshold as the individual mean minus 1.5 standard deviations. | Due to computer file corruption for 3 participants in the EHR Arm/Group and 1 participant in the AWARE Arm/Group, data were lost. | Posted | Mean | Standard Deviation | fatigue events/case | Duration of Single Usability Session, up to 25 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Primary | System Usability Scale Survey Score | Perceived satisfaction measured using the System Usability Scale survey. Values range from a minimum of 0 to a maximum of 10. Higher scores indicate higher satisfaction (better). The overall score is reported and ranges from 0 to 100. It is expected to take 2-3 minutes to complete this survey. | Surveys were administered on paper. Due to staff error, surveys were not distributed to 8 participants in the EHR Arm/Group and 7 participants in the AWARE Arm/Group. | Posted | Mean | Standard Deviation | score on a scale | Immediately following each intervention, within approximately 2-3 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Secondary | Perceived Workload Score | The NASA-Task Load Index (TLX) measures perceived workload using 6 subscale questions. Scores are reported as an overall total ranging from a minimum of 0 to a maximum of 100. Higher scores indicate higher workload (worse). It is expected to take 1-2 minutes to complete this index. | Surveys were administered on paper. Due to staff error, surveys were not distributed to 7 participants in the EHR Arm/Group and 6 participants in the AWARE Arm/Group. | Posted | Mean | Standard Deviation | score on a scale | Immediately following each intervention, within approximately 1-2 minutes |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Secondary | Number of Eye Fixations on Screen | Information seeking load measures the amount of information being processed by participants using an eye-tracking device to measure fixation points (which represents thought-processing). | Due to computer file corruption for 3 participants in the EHR Arm/Group and 1 participant in the AWARE Arm/Group, data were lost. | Posted | Mean | Standard Deviation | Eye Fixations | Duration of Single Usability Session, up to 25 minutes |
|
From the time of signing informed consent through study completion, approximately 1 hour.
Not provided
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | EHR (Control) | Participants in this arm reviewed and completed tasks using the institutional EHR. Electronic Health Record (EHR): An Electronic Health Record (EHR) is an electronic version of a patients medical history, that is maintained by the provider over time, and may include all of the key administrative clinical data relevant to that persons care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports. | 0 | 112 | 0 | 112 | 0 | 112 |
| EG001 | AWARE Intervention | Participants in this arm reviewed and completed tasks using the AWARE intervention. AWARE: AWARE is a platform with a visualization dashboard that sits on top of pre-existing, comprehensive EHR systems. AWARE provides clinical decision support tools for the ICU, Emergency Departments and other critical care areas right at the patient's bedside. | 0 | 112 | 0 | 112 | 0 | 112 |
Not provided
Not provided
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Saif Khairat, PhD, MPH, FAMIA | University of North Carolina at Chapel Hill | 919-543-5413 | Saif@unc.edu |
| Mar 31, 2026 |
| Prot_SAP_001.pdf |
| ICF | No | No | Yes | Informed Consent Form | Jun 24, 2023 | Jun 6, 2025 | ICF_000.pdf |
Not provided
| ID | Term |
|---|---|
| D057226 | Information Seeking Behavior |
| ID | Term |
|---|---|
| D003142 | Communication |
| D001519 | Behavior |
Not provided
Not provided
| ID | Term |
|---|---|
| D057286 | Electronic Health Records |
| ID | Term |
|---|---|
| D016347 | Medical Records Systems, Computerized |
| D008499 | Medical Records |
| D011996 | Records |
| D003625 | Data Collection |
| D004812 | Epidemiologic Methods |
| D008919 | Investigative Techniques |
| D009934 | Organization and Administration |
| D006298 | Health Services Administration |
| D017531 | Health Care Evaluation Mechanisms |
| D011787 | Quality of Health Care |
| D017530 | Health Care Quality, Access, and Evaluation |
| D011634 | Public Health |
| D004778 | Environment and Public Health |
Not provided
Not provided
|
|
|
|
|
|
|
|
| Units | Counts |
|---|---|
| Participants |
|
|
|
| Units | Counts |
|---|---|
| Participants |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|