There are patients who die or have a bad outcome in the hospital and this could be prevented. Data in the nurses' notes could be used by computers to tell the rest of the care team that a patient is not doing well and that they should act more quickly. This project will build and evaluate a computer system that makes it easier for the care team to see and understand that data and act quickly to save patients. The aims of this study is to answer the questions, what is the level of provider use of the CONCERN CDS notification system (called CONCERN SMARTapp) and resulting impact on selected patient outcomes? Specifically, the study has 1) validated desired thresholds for the CONCERN CDS system and 2) integrated the CONCERN CDS system for early warning of risky patient states within CDS tools.

In this portion of the study (aim 3), the investigator will implement and evaluate the CONCERN CDS system on primary outcomes of in-hospital mortality and length of stay and secondary outcomes of cardiac arrest, unanticipated transfers to the intensive care unit, and 30-day hospital readmission rates.

Annually, more than 200,000 patients die in U.S. hospitals from cardiac arrest and over 130,000 patients inpatients deaths are attributed to sepsis. These deaths are preventable if patients who are at risk are detected earlier. Prior work found that nursing documentation within electronic health records (EHRs) contains information that could contribute to early detection and treatment, but these data are not being analyzed and exposed by EHRs to clinicians to initiate interventions quickly enough to save patients. A new source of predictive data is defined by analyzing the frequency and types of nursing documentation that indicated nurses' increased surveillance and level of concern for a patient. These data documented in the 48 hours preceding a cardiac arrest and hospital mortality were predictive of the event. While clinicians strive to provide the best care, there is a systematic problem within hospital settings of non-optimal communication between nurses and doctors leading to delays in care for patient at risk. Well-designed and tested EHRs are able to trend data and support communication and decision making, but too often fall short of these goals and actually increase clinician cognitive load through fragmented information displays, "note bloat", and information overload. Substitutable Medical Applications & Reusable Technologies (SMARTapps) using Fast Health Interoperability Resource (FHIR) standard allow for open sharing and use of innovations across EHR systems. The aim of this project is to design and evaluate a SMARTapp on FHIR used across two large academic medical centers that exposes to physicians and nurses our new predictive data source from nursing documentation to increase care team situational awareness of at risk patients to decrease preventable adverse outcomes.

Communicating Narrative Concerns Entered by RNs (CONCERN) Clinical Decision Support (CDS) system is the application being designed and evaluated. CONCERN Intervention Trial Design will be a multiple time-series intervention. Baseline data will be collected at all study sites. Silent release mode (no SMARTapp notification) will be used in non-equivalent control units and as a post-intervention unit control to evaluate if notifying clinicians can decrease rates of length of stay on non-ICU units and rates of 30-day hospital readmissions. Different versions of the CDS system (SMARTapp) will be incorporated for dynamic, adaptive functionality and determine if the pattern of nursing documentation has changed. A "burn-in" phase is built in to evaluate adoption and adaptation to the algorithm and phases for deployment of the silent release mode within the multiple time-series intervention trial for a total of 18 months of data collection, including pre-intervention data collection and silent release modes.

Study inclusion criteria for patients:

• Inpatients with a stay of at least 24 hours on one of our study units

Study exclusion criteria for patients:

• Inpatients with less than 24 hours on one of our study units
• Patients less than 18 years of age
• Hospice patients
• Did not have a hospital encounter, patients not on one of our study units.

Definition of Study Units:

A clinical unit is considered a CONCERN Study unit if it meets the following criteria:

• A general medical or surgical acute care or critical care unit

The following clinical units are NOT considered CONCERN Study units:

• Pediatric or Neonatal units
• Hospice units
• Emergency Department
• Oncology units
• Obstetrician (OB)/labor and delivery units
• Behavioral/psych units
• Observational units
• Operating room
• Pre-op
• Post-op/Post Anesthesia Care Unit (PACU)
• Same day surgical units
• Plastics units
• Virtual departments in EHR database.

This study will develop an open source SmartApp using interoperability standards. The investigators will make all source code available to any researchers upon request. This approach will ensure open access to all project aims, methods, resources, and deliverables by anyone via a study web-page. To aid in replication this web-page will include methods as well as other project processes such as stakeholder engagement, technical development, governance and lessons learned.

24 months