Does an Indian Version of the International Classification of Disease Injury Severity Score Predict Mortality in Four Public Hospitals in Urban India?
Does an Indian Version of the International Classification of Disease Injury Severity Score Predict Mortality in Four Public Hospitals in Urban India?
In this project, we derive survival risk ratios (SRR) based on International Classification of Disease version 10 (ICD-10) injury codes to validate the ICD Injury Severity Score (ICISS) in data from four public university hospitals in India.
Introduction
In 2013 trauma was estimated to cause 4,8 million deaths, which is more than HIV/AIDS, tuberculosis, malaria and maternal conditions combined (1). Ninety per cent of these deaths occur in lower-middle income countries (LMIC) and an estimated two million lives could be saved annually by improved quality of care (2,3). India is considered a lower-middle income country with over 1 million annual trauma deaths (1). In 2020 trauma is estimated to be the third leading cause of death in the country (4). Hence, efforts to strengthen trauma care in India are urgently needed.
Trauma patients constitute a heterogeneous population, making trauma research and outcome comparison over time and between contexts difficult but important (5). Accounting for factors such as selection bias, difference in care and case mix is crucial for correct conclusions (6,7).To enable this several tools or scores have been developed including the Injury Severity Score (ISS) and the Trauma and Injury Severity Score (TRISS) (8,9). The use of these scores as part of quality improvement programmes has been associated with improved trauma care (10).
In ISS and TRISS the severity assigned to each injury is based on expert consensus. In contrast, the international classification of disease (ICD) injury severity score (ICISS) was developed using a more data driven approach (9,11). This score is based on survival risk ratios assigned to ICD injury codes to estimate an individual patient's probability of survival. According to a recent systematic review ICISS outperforms ISS derived methods (12), but so far almost all research on ICISS is from high income countries. Therefore, our research question is, does an Indian version of ICISS predict mortality in four public hospitals in urban India?
Study design
We will conduct a retrospective registry based study to derive and temporally validate a new version of ICISS.
Setting
We will use data from an ongoing prospective cohort study called Towards Improved Trauma Care Outcomes (TITCO) in India that started in four public university hospitals across urban India. The four centres are Lokmanya Tilak Municipal General Hospital in Mumbai, King Edward Memorial Hospital in Mumbai, Jai Prakash Narayan Apex Trauma Center in Delhi, and the Institute of Post-Graduate Medical Education and Research and Seth Sukhlal Karnani Memorial Hospital in Kolkata. The data used in this study was collected between October 2013 and January 2015.
Trained project officers conducted all data collection. The project officers had a health master's degree or higher education. They worked eight hours a day and rotated between day, evening and night shifts. There was one project officer for each hospital. The project officers where continuously supervised and trained by project management. Patients were followed up until discharge, death or 30 days, whichever came first.
Source and method of participant selection
Project officers included all consecutive patients that fitted the eligibility criteria. Data for patients admitted during the project officers' shifts were collected using a combination of direct observation and extraction from patient records. Data for patients admitted outside of their shifts was collected retrospectively from patient records within days of patient arrival. All patients discharged before 30 days where considered alive at 30 days.
Data sources/measurements
Data on covariates were extracted from patient records or from the patients or their accompanying relatives. Injuries were also extracted from patient records, including imaging reports and operation notes and were then coded using ICD-10. We will calculate the SRR for to each unique ICD-10 code using SRR=A/(A+B), where A denotes the number of surviving patients with a specific ICD-code and B is the number of non-surviving patients with the same specific ICD-code. The calculated SRR gets a value between zero and one. One represents 100% survival and zero represents 0% survival. We will calculate the final ICISS score for each patient as the product of all individual SRRs. Hence, ICISS also ranges from 0 to 1 and should be interpreted as the patient's probability of survival. This method is commonly referred to as the conventional ICISS.
Bias
The personal collecting the data were observers and did not take part in the actual care. During the conversion from injuries in free text to ICD-10 codes the coders will be blinded to patient demographics and outcomes. ICD-10 coding will be done after completing World Health Organization (WHO) ICD-10 online training module and after achieving over 80% agreement in several samples of 50 injuries compared to an external coder.
Study size
We will use all available data from TITCO and create a temporal split sample, using the earlier data for derivation and the most recent data for validation. These two samples are henceforth referred to as the derivation sample and validation sample respectively. We will first estimate the required sample size of the validation sample to include the most recent 200 consecutive events, i.e. patients who died within 24 hours, and all non-events enrolled during the same time period.
We use mortality within 24 hours for our sample size calculation as we want the study to be powered for secondary outcomes also. This effective sample size will allow us to detect a significant difference in discrimination and calibration of ICISS between derivation and calibration samples at 80% power and a 5% significance level. We will include all remaining patients in the derivation sample.
Quantitative variables
We will analyse all quantitative variables as continuous.
Statistical methods and analyses
The derivation and validation of ICISS will be conducted as two separate steps, described below. We will use R for all statistical analyses. We will assess predictive performance in terms of discrimination and calibration. Discrimination will be assessed by calculating the area under the receiver operating characteristics curve (AUROCC) and calibration will be assessed by comparing observed and predicted outcomes visually in a calibration plot and statistically by calculating the calibration slope. Confidence intervals for predictive performance measures will be estimated using a bootstrap approach.
We interpret overlapping confidence intervals as evidence of lack of a significant difference. Parametric and non-parametric exact tests will be used as appropriate, with 95% confidence intervals and a 5% significance level. Our main analysis will be a complete case analysis, in which we exclude observations with missing values in any of the following variables: age, sex, mechanism of injury, transfer status, and outcome. Observations with no injuries reported will be assigned an ICISS of 1 and for each observation the final ICISS will be calculated based only on SRR for ICD-codes that occurred in at least 10 observations in the derivation sample.
Derivation
We will derive SRR in the derivation sample for each of the outcomes and used them to calculate ICISS for each patient. In other words, we will calculate one set of SRR for mortality within 30 days, henceforth referred to as SRR-30D, and one set of SRR for mortality within 24 hours, henceforth referred to as SRR-24H. We will then calculate two ICISS for each patient. We will use similar denotation to refer to these ICISS, i.e. ICISS-30D and ICISS-24H. Finally, we will assess the performance of ICISS-30D in predicting mortality within 30 days and within 24 hours, and repeated this analysis for ICISS-24H.
Validation
We will use the SRR-30D and SRR-24H that we derive in the derivation sample to calculate ICISS-30D and ICISS-24H in the validation sample. We will then assess the performance of ICISS-30D in predicting mortality within 30 days and within 24 hours, and the performance of ICISS-24H in predicting mortality within 30 days and within 24 hours. Finally, the performance of each model in the validation sample will be compared with the same model's performance in the derivation sample.
Sensitivity analyses
We will conduct four sensitivity analyses. In the first sensitivity analysis we will include observations with missing values in covariates but with complete outcome data. In the second sensitivity analysis we exclude observations without any reported injury. In the third sensitivity analysis we will calculate ICISS based on all available SRR, regardless of how frequently the corresponding ICD-10 codes occurr in the dataset. Finally, we will calculate ICISS for each patient based only on unique ICD-10 codes, in other words, each ICD-10 code will only be allowed to contribute one SRR to ICISS even if it occurrs more than once in the same patient.
Inclusion criteria
Exclusion criteria