Modification and Validation of the RIPASA Score for Diagnosis of Acute Appendicitis Across Different Ethnic Populations (MODRIS): An International Multicentre Study
Modification and Validation of the RIPASA Score for Diagnosis of Acute Appendicitis Across Different Ethnic Populations (MODRIS): An International Multicentre Study
Purpose: The diagnostic approach of patients with acute appendicitis (AA) remains debated. A number of clinical prediction rules (CPRs) exist for diagnosis of AA with variable sensitivity, specificity, and diagnostic accuracy, in different ethnic populations. Among these, the Raja Isteri Pengiran Anak Saleha Appendicitis (RIPASA) score was shown to have a high sensitivity though with a poor specificity. The aim of this study is to modify and validate the RIPASA score in 4 distinct ethnic groups, and to compare the performance of the new score to the original RIPASA and Alvarado scores.
Methods: This multicenter, international prospective observational study will be conducted in 6 countries, and will include all eligible patients referred to surgical specialists with acute right iliac fossa pain or suspected AA in the participating centres. Patients will be categorized into 4 ethnic groups based on the country of origin. The modified RIPASA score will be developed in one ethnic group, and along with the original RIPASA and Alvarado scores, will be externally validated in the other 3 ethnic groups. Management of patients will be prospectively evaluated in a standardized manner. The focus of the analysis will be on the performance of the 3 CPRs in different ethnic and gender groups using receiver operating characteristic curve analyses.
Discussion: We expect this study to develop a CPR that can assist surgeons and surgical residents to early identify patients with AA in the busy clinical and low-resource settings, and to optimize the diagnostic value of the RIPASA and Alvarado scores in different ethnic and gender groups.
Introduction Acute appendicitis (AA) is the most common abdominal emergency requiring emergency surgery, and its preoperative clinical diagnosis remains a vexing challenge. Several clinical prediction rules have been developed to facilitate diagnosis of AA, and to predict severity of the disease, nonetheless there has been a controversy regarding the benefit of these CPRs in reducing negative appendicectomy rates and costs and improving surgical outcomes. Among these, the Alvarado score is the most commonly validated CPR, followed by Kalan's modified Alvarado score. The latter CPR is more user friendly and almost equally sensitive though less accurate than the original Alvarado score. Both scores are however more useful when used as "rule out" rather than "rule in" diagnostic tools, particularly in female patients of child-bearing age. Moreover, while numerous reports have demonstrated that these scores are reliable Clinical predictors for AA in western countries, parallel studies failed to replicate these results in different ethnic populations. This was partially ascribed to variations in the characteristics of the evaluated patients, and variable interpretation of variables by different clinicians in different settings.
The Raja Isteri Pengiran Anak Saleha Appendicitis (RIPASA) score, which is based on 15 clinical and laboratory parameters, has been recently validated by Erdem and colleagues, and compared to other three CPRs including the Alvarado score. The RIPASA score presented the highest sensitivity (100%, cut-off value of 7.5), whereas it had a poor specificity (28%). The diagnostic accuracies of the scoring systems however were almost comparable. In a recent meta-analysis conducted by Frountzas et al, culling data from 12 studies, the RIPASA score showed superior sensitivity (94% vs 69%) and area under curve (AUC) value (0.94 vs 0.79), and a lower specificity (55% vs 77%) compared to the Alvarado score. More importantly, ten out of the 12 studies in this analysis were carried out in Asian, Middle Eastern, and Latin American patients and evidenced that the RIPASA score consistently outperformed the Alvarado score in terms of diagnostic accuracy and sensitivity. On the other hand, none of the studies included African patients, and only two studies were conducted in western populations, showing conflicting results.
A recent study has shown that the most reliable laboratory parameters in the diagnosis of AA are primarily "neutrophilia" and secondarily "leukocytosis". Nonetheless, different cut-off values for these predictors have been suggested in the literature. In a well-designed prospective blinded trial, Dueholm et al. demonstrated that the interquartile range (25%-75%) of the total white blood cell (WBC) count (x 109/ l) for "No appendicitis" and "Appendicitis" was 7.1-11 vs 11-16.3, respectively. The corresponding figures for neutrophils were 60-80% and 75-88%, respectively. The authors assessed the diagnostic value of tests combinations, and indicated that a diagnostic model using WBC > 11 (x 109/ l) as a cut-off value has the highest efficiency, with a negative predictive value of 98%. In contrast, Iyrik et al. showed that WBC count of 12 (x 109/ l) and neutrophils of 73% were the optimal cut-off values to discriminate between "Appendicitis" and "No appendicitis". Of note, the cut-off value for leukocytosis in the RIPASA score is not clearly defined, and the parameter "neutrophilia" is not included. Therefore and in light of the current evidence, we would like to introduce some modifications to the RIPASA score in order to improve the specificity and to make it simpler, reproducible, and applicable to busy clinical settings.
Methods Hypothesis and objectives The hypothesis is that a better understanding of the performance of CPRs in relation to ethnic differences may improve their diagnostic value and may have a role in avoiding unnecessary operations, reducing the negative appendicectomy rate and improving patient outcomes.
The primary objective of this study is to develop a modification of the RIPASA score in a Latin American population, and to externally validate the modified RIPASA (MODRIS) score in 3 other ethnic populations (European, African, and South Asian). For comparison, the performance of the original RIPASA and Alvarado scores will also be analyzed.
Secondary objectives are:
Study setting and participants For this prospective multicentre cohort study, recruitment of the study subjects will take place in the participating centres in 6 countries during the study period November 1st 2019 to May 31th 2020. Patients will be stratified according to the country of origin into four ethnic groups: "African" originating from Nigeria and Sudan, "Latin American" originating from Mexico, "South Asian" originating from India and Pakistan, and "European" originating from Spain.
Ethics and informed consent This study is conducted in accordance with the declaration of Helsinki. Participation in the study does not oppose a risk greater or different from the general surgical treatment of patients with RIF pain. To ensure data anonymity, no patient identifiable information will be collected by the main data manager. The study protocol was approved by the medical ethics committee of the trial coordinating centre (Soba University Hospital, Sudan, ID: 201903311). In other countries, local lead investigators were responsible for arranging research ethics committee or institutional approval locally, as appropriate. The study has been registered in the ClinicalTrials.gov database (ID: NCT04207645).
Patient management and follow-up During the study period, the decision to operate on patients will depend on the surgeon's judgement, imaging findings, or both, and not any appendicitis scoring results. With the exception of the main investigators, all researchers who are involved in the data collection will be blinded to the three CPRs under evaluation. This is to ensure that predictors are assessed blinded to outcome. The clinical data of patients will prospectively be collected during the initial evaluation using standardized case report forms. These include demographics, clinical findings, laboratory investigations, imaging studies, operative details, and postoperative course. All patients, including those who are managed conservatively, will be followed up for 1 month to assess for postoperative adverse events or whether they required a subsequent operation.
The patients will be dichotomized into two groups: "No appendicitis" group comprises patients who are managed conservatively, underwent negative appendicectomy, or have diagnoses other than AA, and "Appendicitis" group comprises patients who have histologically confirmed AA.
Brief centre survey:
At each participating centre, a consultant surgeon has completed a brief survey regarding the service characteristics and the clinical pathway of management of patients with RIF pain in their hospital.
Definitions:
Predictors change:
Sample size calculation The study population comprises a development and internal validation cohort (Latin American group), and 3 external validation cohorts (European, South Asian, and African groups). In each of the 4 groups, the sample size calculation will be based on a planned interim analysis that will be carried out at the start of the trial, and will involve the first 200 patients. The patients of this internal pilot study will be included in the validation cohorts. In the Latin American group, the number of final candidate predictors will be determined based on calculation of the number of events, and this is to verify an events-per-variable rate greater than 20. In each of the external validation cohort, an algorithm based on a binary logistic regression model, recently described by Palazon-Bru et al., will be used to estimate the required sample size for external validation of the MODRIS score. This method uses smooth curves rather than risk categorization, thereby providing greater validity to the results. Based on the results of the survey, it is anticipated that centres in each geographical region (5-11 centres per region) will capture a total of 60-200 patients with RIF pain per month. Recruitment will continue in each region until the estimated sample size is reached.
Statistical analysis Statistical analysis will be performed using SPSS version 22.0 (IBM corp., Armonk, NY, USA) and R 3.5.1 (Foundation for Statistical Computing, Vienna, Austria). Normal distribution of data will be assessed using visual inspection of histograms. Normally distributed data will be presented as mean with standard deviation (SD) and non-normally distributed data will be presented as median with interquartile range (IQR). Categorical variables will be presented as counts and proportions. Comparisons between groups will be performed using both parametric and non-parametric tests as appropriate. A two-sided significance level of P < 0.05 will be considered statistically significant. Missing data will be explored and if the percentage exceeds 10 % a multivariate imputation with chained equations will be performed.
Development and internal validation of the MODRIS score: four continuous predictors (age, temperature, WBC count, and neutrophil percentage) will be transformed into basic cubic splines to determine the functional form of these predictors regarding the outcome risk. All candidate predictors, including all splines transformations of the continuous predictors, will be included in a logistic regression model and then a backward elimination method based on the likelihood ratio test (P < 0.25) will be applied to select the final predictors. In addition, gender will be included as a variable and interaction term for all symptoms and signs. The model will be adapted to a scoring system using the Framingham Heart Study methodology. The MODRIS score will be internally validated through a bootstrap technique with 1000 random samples. Discrimination will be assessed by calculation of area under the receiver-operator characteristic curve, whilst calibration will be obtained through the construction of smooth curves.
External validation: the MODRIS score will be externally validated in 3 ethnic groups. Simultaneously, discrimination of the RIPASA and Alvarado scores will be determined, and the DeLong test will be used to compare the difference between the AUCs of the 3 CPRs. The ROC analyses will be used to calculate the optimal cut-off value for the 3 CPRs in each ethnic group. The specificity, sensitivity, and negative predictive value will also be calculated. Based on the optimal cut-off value, patients will be stratified into "low risk" and "high risk" groups. An ideal CPR would achieve the highest specificity while maintaining a negative predictive value of ≥ 95% in the low-risk group. The postoperative outcomes of "low risk" groups will be compared between CPRs. In addition, gender-specific analysis will be carried out to assess the performance of the CPRs in each gender group.
Discussion The current study aims to externally validate the MODRIS score, and to compare it to the original RIPASA and the widely used Alvarado score in 3 distinct ethnic groups, using a large, prospective international cohort. Although several previous studies have validated the latter two CPRs, most were either underpowered or had a retrospective design.
In a recent multicentre study in Europe validating 12 CPRs for AA, the Adult Appendicitis and Appendicitis Inflammatory Response (AIR) scores were found to be the best performers. Nevertheless, both scores require C-reactive protein (CRP) measurement for their calculation. We did not include CRP level in the MODRIS score which makes it particularly useful in low-resource settings with limited access to diagnostic investigations. Although it correlates well with the severity of the disease and postoperative complications, a single CRP reading has only a modest positive predictive ability for AA, and its incorporation as an adjunctive test in the modified Alvarado score did not improve the diagnostic accuracy of the scoring system. Moreover, the MODRIS score maintains the advantages of the RIPASA score such as it requires calculation only at a single point in time, does not require special clinical experience, and includes the gender factor, thereby ensuring a lower false-positive rate of AA in female patients compared to Alvarado score.
Implications for clinical practice Previous reports have indicated that AA in developing countries with limited resources has a different disease profile compared to that described in the developed world. It is associated with prolonged delays to definitive surgical care, significant morbidity, and poor outcome, which are partially ascribed to a much wider range of differential diagnoses of RIF pain than in the developed world, resulting in failure to early diagnose the condition. In another aspect, Scarborough et al. compared the postoperative outcomes of African and White American patients undergoing appendicectomy for AA. The analysis showed that the black race was independently associated with increased incidence of serious and overall postoperative complications. The authors suggested that blacks may suffer a delay in either diagnosis or management of appendicitis relative to whites. Clinical prediction rules have been advocated by several authors as one of the suggested strategies to improve outcomes of AA by facilitating clinical decision-making among junior doctors. Optimizing the diagnostic value of CPRs in different ethnic groups may help to improve the quality and safety of care and effectiveness of the management of patients suspected for AA, particularly in African patients and in resource-constraint environments.
Limitations By the very nature of the multicentre design of the study and the variable clinical and laboratory assessment of patients in different centres, inter-observer variability is expected to present in the dataset and this may impact the internal validity of the study. However, efforts were made to minimise this inherent bias by recruiting centres from a limited number of countries, which are likely to follow a similar clinical pathway of management of AA. Data collection will be carried out using standardized forms designed to be completed at the patient's bedside, to reduce measurement and recall bias. Another limitation of the study is that the follow-up is limited to 30 days after the index hospital admission. It is possible that a proportion of patients who are managed conservatively may subsequently undergo appendicectomy either at other hospitals or beyond the 30-day follow-up. Furthermore, although we are trying to achieve high levels of case ascertainment, it is possible that a small number of eligible patients may be missed during the study period.
Inclusion Criteria:
Exclusion Criteria:
Menorca, Spain