Investigating Trends in Compliance With Quality Assurance Metrics
Investigating Trends in Compliance With Quality Assurance Metrics
Over 40 million major operative procedures are performed in the US annually and comprise about 40% of healthcare expenditures. Despite decades of research, perioperative mortality and morbidity remain a major healthcare system cost and detriment to long-term quality of life. More than ten percent of patients experience a significant event such as surgical site infection, reoperation, myocardial infarction, pulmonary embolus, or death. Nearly 100,000 patients die after surgery each year. National data demonstrate a 3-fold variation in risk adjusted surgical morbidity and mortality, suggesting many opportunities for improvement in perioperative care.
Anesthesiology care demonstrates wide variation in practice. Sometimes, this variation is appropriate because the anesthesiologist is responding to patient comorbidities or procedure specific events. However, even after controlling for patient specific factors, there is a substantial amount of unexplained variation in fundamental elements of anesthesiology care. The same procedure and patient can be performed using completely different anesthetic techniques, hemodynamic management strategies, and medications. This variation in care can lead to a variation in outcome.
The use of electronic health records (EHR) with detailed preoperative and intraoperative data allows an automated system to be developed to notify clinicians their compliance to both process of care metrics and outcome metrics. The Multicenter Perioperative Outcomes Group (MPOG) quality improvement arm is known as Anesthesiology Performance Improvement Reporting Exchange (ASPIRE). Like other Collaborative Quality Initiatives, the primary goal of ASPIRE is to provide hospitals with risk-adjusted feedback on outcome and process of care variation. In addition, ASPIRE creates an active best-practice sharing environment to enable data to spur action. Recent literature has demonstrated that hospital-level feedback may not be adequate to improve performance and clinical outcomes. In addition to hospital level data and feedback, ASPIRE can disseminate provider-specific electronic feedback that may decrease variation in care known to impact complications and cost.
The primary aim for this research study on ASPIRE's QI program is to determine if the investigators can change behavior as measured by a provider's compliance to specific performance metrics. The investigators believe that the start of individual provider performance feedback reports to ASPIRE members presents a unique opportunity to research the efficacy of these novel tools. The investigators propose to test the hypothesis that monthly provider specific feedback emails on ASPIRE quality metrics over a period of 9 months improves provider compliance as measured by a either a 10% improvement in the Total Performance Score or by moving from below to above the 90% performance threshold in the Total Performance Score Index.
Each provider type (faculty, CRNA, resident/fellow) within a hospital participating in ASPIRE will be individually randomized to either receiving the electronic performance improvement email or not for a total of nine months. No individual at the participating site will see the individualized email compliance reports except for the specific provider. Only an aggregate of the compliance across the entire hospital will be supplied to the chairperson and the quality assurance directors. After the completion of the nine month randomization period, all providers will receive monthly ASPIRE performance improvement emails.
The University of Michigan is the coordinating center but also participating in this research on QI project. De-identified patient data will be pulled in aggregate for each provider using the MPOG database. The provider performance for each measure will then be sent from ASPIRE to the randomized care provider via an email. The chairperson and quality assurance directors will only see aggregate data on compliance rates and can NOT identify individual compliance rates. Each participating site will obtain their own institutional IRB to participate in this study.
Anesthesiology care demonstrates wide variation in practice. Sometimes, this variation is appropriate because the anesthesiologist is responding to patient comorbidities or procedure specific events. However, even after controlling for patient specific factors, there is a substantial amount of unexplained variation in fundamental elements of anesthesiology care. The same procedure and patient can be performed using completely different anesthetic techniques, hemodynamic management strategies, and medications. This variation in care can lead to a variation in outcome.
The use of electronic health records (EHR) with detailed preoperative and intraoperative data allows an automated system to be developed to notify clinicians their compliance to both process of care metrics and outcome metrics. The primary goal of ASPIRE is to provide hospitals with confidential risk-adjusted feedback on outcome and process of care variation. In addition, ASPIRE creates an active best-practice sharing environment to enable data to spur action.
Recent literature has demonstrated that hospital-level feedback may not be adequate to improve performance and clinical outcomes. In addition to hospital level data and feedback, ASPIRE can disseminate provider-specific feedback that may decrease variation in care known to impact complications and cost. ASPIRE uses the underlying EHR data integration foundation of the Multicenter Perioperative Outcomes Group to aggregate and analyze process of care and outcome data.
To date, there is no anesthesia standard for quality improvement practice regarding provider-specific feedback. The primary aim of this research study on ASPIRE's QI program is to determine whether provider-specific feedback affects quality improvement performance metrics. The investigators believe that the start of individual provider performance feedback reports to ASPIRE members presents a unique opportunity to evaluate the efficacy of these tools.
The investigators propose to test the hypothesis that monthly provider specific feedback emails on ASPIRE quality metrics over a period of 9 months improves provider compliance as measured by a either a 10% improvement in the Total Performance Score or by moving from below to above the 90% performance threshold in the Total Performance Score Index.
Pre-defined process of care and outcome metrics will be calculated using the Multicenter Perioperative Outcomes Group (MPOG) database. De-identified patient data will be extracted from the MPOG database in aggregate for the anesthesia providers to determine their overall compliance to the process of care and outcome metrics. The compliance metrics for each provider will be stored in the MPOG/ASPIRE database. Any measure implemented in production in between July 1, 2015 and July 1, 2016 will be incorporated into the analysis. The quality improvement system generates a monthly email to the provider stating their performance compared against the performance of their peers for each measure. Each measure is then hyperlinked back into ASPIRE web-based dashboard where the provider can review the cases that they failed on each measure. The visualization removes protected health information but is the representation of the physiologic monitoring, medication and fluids administered, laboratory values, and time-based events of the procedure. Provider attribution for each measure will follow existing ASPIRE specifications (available at https://www.aspirecqi.org/aspire-measures ). Each provider type (faculty, CRNA, resident/fellow) within a hospital participating in ASPIRE will be individually randomized to either receiving the electronic performance improvement email or not for a total of nine months. After the completion of the nine month randomization period, all providers will receive monthly ASPIRE performance improvement emails.
At the conclusion of the project, clinical outcomes of interest will be extracted from the MPOG database via an honest broker using ICD-9 and ICD-10 codes and all-cause 30 day mortality.
ASPIRE Quality Measures: Measure Description BP 01: Avoiding intraoperative hypotension (mean arterial pressure less than 55 mmHg) BP 02: Avoiding gaps in systolic or mean arterial pressure measurement longer than 10 minutes GLU 01: Percentage of cases with intraoperative glucose > 200 (between anesthesia start and anesthesia end) with administration of an insulin bolus or infusion or glucose test recheck GLU 02: Percentage of cases with glucose < 60 (between anesthesia start and anesthesia end with a glucose test recheck or administration of dextrose containing solution (between anesthesia start and anesthesia end + 2 hours) NMB 01: Percentage of cases receiving a non-depolarizing neuromuscular blockade medication that have a Train of Four (TOF) count documented NMB 02: Percentage of cases receiving a non-depolarizing neuromuscular blockade medication with administration of reversal agent PUL 01: Percentage of cases with median tidal volumes less than 10 ml/kg of ideal body weight (IBW) TRAN 01: Hemoglobin or hematrocrit measurements before each transfusion for patients receiving discretionary intraoperative red blood cell transfusions TRAN 02: Avoiding post transfusion hematocrit greater than 30%
Statistical Analysis:
All statistical analysis will be completed using a de-identified dataset for which the analyst will have no link back to each of the individual's unique names. The investigators will combine several key process of care measures into a process of care bundle for the analysis. The bundle for each participating site will include the process of care quality measures included on month 1 of the email feedback program. If there were more than one instance that a provider could pass or fail for a specific case, if the provider passed or failed at least once it would pass or fail for the entire case. The primary outcome will be the proportion of providers that achieve improvement in performance from start to end of the study period. The investigators will exclude providers that already met all the metric thresholds for the primary analysis.
Our primary analysis will include all providers. Secondary analysis will include performers not meeting threshold measures. Improvement in performance will be determined by the following method:
The threshold for what will be considered a performer not meeting threshold measures will be determined by examining the distribution of the bundle compliance. Randomization is pre-determined at the start of the project and is NOT based on if the provider is classified as a performer meeting threshold measures or not. Each provider's baseline compliance rate will be determined by the performance from the first month's feedback email. In addition, the investigators will do a secondary analysis investigating if the providers that met all the threshold metrics prior to the project further improved during the study period. A sub-group analysis excluding the coordinating center is planned.
The primary analysis will only include sites where all provider types are randomized (attendings/residents/CRNAs). Sites where one provider type (ie attending or CRNA) was randomized and the other provider type was not will be included in a secondary analysis.
To assess the investigators primary hypothesis of the impact of provider-specific emails on overall compliance during the nine-month study period, a repeated measures generalized linear model (GLM) will be used. Between-subjects (randomization to email) and within-subjects (compliance) analyses will be reported. The primary analysis will be stratified by provider type (faculty, fellow, resident, and CRNA). A sub-group analysis excluding the coordinating center is planned.
For the investigators secondary analysis to determine if a provider who already met the threshold performance metric prior to the study further increased in their compliance a linear regression will be used.
Outcome analyses will be performed only using data related to inpatient / admit day of procedure operations. First, to determine if providers receiving an email about the participants specific bundled compliance affects a patient's overall combined morbidity and mortality a binary logistic regression model will be used. The dependent variable will be combined morbidity and mortality as a Boolean concept. The independent variables entered into the model will be: primary provider AND primary attending both received an email, primary provider did NOT receive an email but the primary attending did receive an email, primary provider did receive an email but the primary attending did NOT receive an email, elixhauser comorbidity score of 2 or more, ASA (binary concept as 1,2 versus 3,4), age (binned by decade of life), male gender, BMI (defined by the World Health Organization Classifications). The investigators can then determine whether providers receiving feedback emails have a risk adjusted improved combined morbidity and mortality rate. If the investigators find this to be true, a mediation analysis at the provider level will be performed to determine if our independent variable (email received) influences the mediator variable (bundle compliance rate) which thereby influences the dependent variable (outcome of interest). The investigators will report the direct effect for receiving the email on the provider's outcome rate as well as the indirect effect of the email that passes through the bundle compliance rate to affect the outcome rate. These values will be reported as the proportion of total effect that is mediated by compliance. Sensitivity analyses for residents/fellow and CRNAs will also be performed. A sub-group analysis excluding the coordinating center is planned.
Inclusion Criteria:
Exclusion Criteria: