TITLE A randomized controlled trial to assess the best method for evaluating avoidable mortality in hospitals using medical record review.

DESIGN Randomised controlled trial This trial will be conducted in concordance with the CONSORT guidelines for randomised controlled trials.

AIMS Compare explicit versus implicit case review methods for detecting avoidable mortality in patients who have died in hospital Assess the accuracy and efficiency of physicians versus nurses in detecting avoidable mortality in patients who have died in hospital

OUTCOME MEASURES Rate of adverse event detection between explicit and implicit review methods Type of adverse event detection between explicit and implicit review methods

Kappa score of agreement for inter-rater reliability for the above outcomes:

• Between individual reviewers within using the same review method
• Between different reviewer types.

PARTICIPANTS Physician reviewers and nurses No patient participation. Case records of patients who had inpatient hospital mortality will be reviewed.

ELIGIBILITY Over 18, Able to consent, English Language Speaking, Able to participate in case note review training, appropriate level of clinical experience.

DURATION 2 years with extension if required and reviewed.

BACKGROUND

What is avoidable mortality?

As emphasized by the recent Francis Report into care at the Mid-Staffordshire hospital, avoidable mortality is an important measure of safety and quality of healthcare. Measures of risk-adjusted mortality (SHMI and HSMR) based on population mortality statistics have previously been criticized for overestimating harm. It is difficult to determine within these measures the degree to which deaths could have been prevented with adequate medical care. To assess the quality of healthcare, it is more useful to focus on deaths that have objective evidence of being related to adverse events or complications.

Adverse events are defined as injuries or complications that occur as a result of health care management and not as a result of the patient's pathology, causing prolonged hospital stay, morbidity and mortality. Avoidable mortality can be defined as a death where an adverse event or events has occurred that contributed to or precipitated this death.

What methods are there for measuring avoidable mortality?

The avoidable nature of an inpatient death has traditionally been assessed through retrospective case record review. The best means of performing such a review is unclear. The ideal review system would maximize the adverse event detection rate but be efficient, timely and pragmatic.

There are two types of case note review: implicit (or holistic) and explicit. The implicit review involves a trained expert reading a case note in full and accessing any further supplementary information. Current methods of using experienced reviewers to provide holistic assessment of case notes are intuitively sound. However the process of recruitment, training and reviewing notes is prolonged. Implicit review is subjective and can result in lower levels of inter-rater reliability than explicit methods. . It may be insensitive to detecting differences in care quality at a hospital level . Hogan and colleagues recently used holistic retrospective record review to assess the incidence of hospital avoidable mortality in England finding a rate of 5.2% of mortality cases.

Explicit systems, such as the Global Trigger Tool, have also been used to detect adverse events. The Institute for Healthcare Improvement (IHI) developed a Global Trigger Tool for Measuring Adverse Events (GTT), which has been validated for use in the United Kingdom (UK) as the Acute Trigger Tool. The GTT is a reliable and valid method for detecting adverse events in hospitals with the highest sensitivity for harm detection of any review method and with a good level of inter-rater agreement and specificity It has been used to track the rate of adverse events over time and compare hospital event rates. The trigger tool has been used in combination with more in depth or "deep dive" questions to investigate preventable hospital mortality. Kaiser Permanente (KP) developed a quantitative method where nurses with physician guidance can more efficiently review notes to determine the avoidable nature of death.

Explicit and implicit case review methods have been compared before in order to assess quality of care. This study is novel in that it will compare sensitivity and specificity for determining preventability of mortality.

Who should act as an avoidable mortality reviewer?

Given the limited number of available expert reviewers, it would be useful to demonstrate that other reviewers have similar reliability in assessing avoidable death. Trained nurses are a potential resource for case note review. They have basic clinical skills and knowledge. However a more protocol driven method may safeguard against errors due to decreased appreciation of clinical events. Therefore explicit criteria for review maybe more likely to demonstrate greater inter-rater reliability.

What is the purpose of this study?

This study aims to assess what the most efficient and reliable case note review method is for detecting avoidable death, so that it could be applied on a regular basis to investigate excess mortality in hospitals.

It will also evaluate if appropriately trained nurses can reliably detect avoidable deaths and whether the specificity and sensitivity of their review is comparable to experienced physician reviewers.

INTERVENTION: EXPLICIT CASE REVIEW METHOD

A modified Acute Trigger Tool (ATT); the UK version of IHI Global Trigger Tool will be used. (Appendix 1) Each reviewer will scan the records and look for specific triggers. Once identified the pertinent portions of the record will be analysed using the "deep-dive" questions that have been validated by KP. (Appendix 2) These questions aim to determine whether preventable harm occurred leading to death. If not found the reviewer should ignore the trigger and search for further triggers. Positive triggers do not indicate adverse events.

If adverse events are identified without triggers, these should also be included.

As a result of adverse events detected, the reviewers will then judge the preventability of the death using a 6-point Likert scale.

Data will be extracted to a pre-formatted data sheet.

For nursing review instances of harm will be reviewed and validated by a physician on the team, if the nurse review is in doubt

INTERVENTION: IMPLICIT CASE REVIEW METHOD

Implicit review will be a two-stage process. The reviewers will read the notes and judge whether any adverse events occurred or whether there were any errors in the care pathway that are deemed to be preventable.

Reviewers will be asked to assess all aspects of the patients' care and the entire record including nursing notes and drug charts. Demographic information will be collected as well as co-morbidities and functional impairments For each case where an adverse event lead to death reviewers would then judge the preventability of the death using the 6-point Likert scale.

Data will be extracted using a structured data collection form.

STANDARDISED DETERMINATION OF AN ADVERSE EVENT

Adverse event implies unintended harm to a patient. These should also include the following:

• Event where the perspective of the patient would be that harm occurred to them (NB this does not include psychological harm)
• Event was not the natural progression of the disease process; i.e. as part of the medical treatment
• Events which are present on admission, provided that it was harm related to medical care, e.g. delayed diagnosis in outpatients

For both methods the reviewers will have opportunity to discuss questions with the expert reviewer and research team.

For all adverse events leading to death the timing, and contributory factors as assessed by the reviewers will be logged.

STANDARDISED SEVERITY SCORING OF ADVERSE EVENT (NCC MERP)

Category E: Temporary harm to the patient and required intervention Category F: Temporary harm to the patient and required initial or prolonged hospitalization Category G: Permanent patient harm Category H: Intervention required to sustain life Category I: Patient death

STANDARDISED TRAINING

All nurses and physician reviewers involved in the study shall complete the ATT training, which will comprise web-based seminars and practice notes. They will be assessed using example case notes to ensure a satisfactory level of detection is achieved.

Training in implicit case note review will also be employed using an iterative process with example notes.

Once an acceptable standard has been reached for all reviewers the review process will commence.

BLINDING

• Reviewers will be blinded to the results of previous assessments so as not to bias the result.
• The research team will be blinded as to who reviewed the notes and by which method when collecting the results.
• All reviewers will be given a randomly generated identification number.

RANDOMISATION

Randomisation of reviewers and notes will be performed using a random number generator. All notes and reviewers will be assigned a random number and then allocated to groups.

CONTROL GROUP

Each reviewing group acts a control for the other in this study. The case notes will be assessed by both teams using different methods and in order to not introduce bias due to increased learning the teams will change review methods also when the notes are crossed over.

Inclusion Criteria:

FOR REVIEWERS

• Over 18, with capacity to consent and fluent in English language
• Able to commit to training and reviewing process

Nurses • Senior nurses, either senior band 5 or band 6 and above, preferably with at least 5 years general ward experience.

Physicians

• At least five years clinical practice in either medical/surgical/primary care/emergency care/critical care/anaesthesia as a consultant.
• If retired, have been out of medical practice no less than 5 years.

FOR CASE NOTES/SAMPLING STRATEGY

All case notes where the patient has died during admission from Imperial College Healthcare Trust, Northwest London Hospitals National Health Service (NHS) Trust and The Hillingdon Hospitals NHS Foundation Trust will be listed and then randomly sampled. They should fulfil the following criteria:

• Deceased patients with closed records (all coding completed)
• Length of stay at least 24 hours
• Over 18.
• Admitted between 2010 and 2013

Exclusion Criteria:

FOR REVIEWERS

• Cannot participate in this review if part of the clinical team caring for the patient whose notes are being reviewed.

FOR CASE NOTES

• Incomplete records.
• Paediatric records/ maternity records/ inpatient Psychiatric records.