Predicting the Risk of Venous Thromboembolism in Critically Ill Patients

Predicting the Risk of Venous Thromboembolism in Critically Ill Patients: Validation of Pre-existing Risk Prediction Models and Development of a Prediction Model

Introduction: Venous thrombosis (VTE), including both deep vein thrombosis (DVT) and pulmonary embolism (PE) remains a frequent complication in patients admitted to the Intensive Care Unit (ICU). Multiple prediction models for estimating the risk of VTE have been developed. However, many models have not been externally validated. The aim of this study is to perform a comprehensive external validation of pre-existing prediction models for predicting the risk of in-hospital VTE in critically ill patients. In case current risk assessment models fail, the investigators aim to additionally develop and internally validate a new risk prediction model.

Methods: During the first phase of the study the investigators will perform external validation of existing prediction models. The performance, discrimination, calibration and clinical usefulness of the models will be evaluated. In the second phase of the study, in case performance of current risk assessment models is deemed insufficient for clinical application, the investigators will develop a model for predicting the risk of in-hospital VTE in critically ill patients. A multivariable prediction model will be constructed using a combination of predefined candidate predictors. This model will be internally validated and performance will be compared with performance of existing VTE risk prediction models.

Dissemination: This protocol will be published online. This study will be reported according to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement and this study will be submitted to a peer-reviewed journal for publication.

Objective

1. To perform external validation of existing prediction models (phase 1)
2. To develop and internally validate a model for predicting the risk of in-hospital VTE in critically ill patients (phase 2)

Study design:

The objectives will require a large number of outcome events (see sample size calculation). To fulfill this requirement and to increase generalizability of the results, the intention is to collect data from several sources, which will include several types of study designs. The investigators currently have access to data from the Simple Intensive Care studies' (SICS-I and SICS-II; clinicalTrials.gov identifier:NCT02912624 and NCT03577405). In addition, an unpublished database that contains registry data from a community-hospital in the Netherlands will be used. Third, data from the prospective AFIB-ICU cohort study will be used. To further increase the sample size, the investigators are exploring additional collaboration with colleagues aiming to include more data.

In the final manuscript a table with the following descriptives of each data source will be included: study design, selection criteria, time period of conduct; data collection (prospective, retrospective); and types and definitions of included predictor and outcome variables.

This study will be reported according to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.

Study participants:

The target population are critically ill patients who are admitted to the ICU regardless of underlying disease. Patients who were admitted after planned surgery or other procedures will be excluded. The investigators foresee that there will be differences between the selection criteria of each cohort. As stated above, the investigators will reflect on these differences in the final manuscript.

Candidate predictors:

Candidate predictors were selected based on the following criteria:

1. established or suggested association with VTE (based on literature)
2. or incorporation in another VTE risk assessment model;
3. and readily available and easy to obtain in daily clinical practice.

The following candidate predictors will be explored: active cancer, acute infection, acute renal failure, cardiovascular failure, central venous access, elderly age, estrogen therapy, sex, major surgery, mechanical ventilation, multiple trauma, obesity, previous VTE, reduced mobility, respiratory failure, stroke, thrombophilic disorder, vasopressor use. When available the investigators also aim to include C-reactive protein (CRP), hemoglobin (Hb), leukocytes and thrombocytes in the model. A complete list of all candidate predictors including their definitions and units of measurement, is displayed in table 1*.

Outcome:

The primary outcome will be in-hospital VTE. VTE will be defined as any objectively proven event occurring during initial hospital admission. No screening protocol will be used. DVT will include acute thrombosis of lower-extremity veins (iliac, femoral or popliteal), confirmed by compression ultrasonography, venography, CT, MRI, or autopsy. Pulmonary embolism will be defined as acute thrombosis within the pulmonary vasculature as shown by ventilation-perfusion scan, CT angiography, or autopsy. Upper extremity DVT will be included in the model but venous thrombosis in any other site (e.g. portal vein thrombosis) will be excluded as these may represent a different entity.

Data management:

Data from multiple cohorts will be collected and stored in a secure location in any applicable format in compliance with institutional, national and international applicable regulatory laws.

Identifying and inclusion of previous risk assessment models:

A recently published systematic review will be used to select the prediction models for external validation6. Additionally, the investigators will contact experts in the field and conduct a non-structured literature search to identify any additional and/or more recently published prediction models for in-hospital VTE. The following online libraries will be searched: Embase, Medline and Google Scholar. No time restrictions will be used. Models will be selected for inclusion if all predictor variables are available.

Sample size:

For the two phases of this study: external validation of existing prediction models and, second, development and internal validation of a new model for predicting the risk of in-hospital VTE in critically ill patients, the required sample size will be calculated in two different ways.

Phase 1: external validation of existing prediction models The investigators will assess whether the collected data are sufficient for external validation by calculating the sample size for external validation according to a recently published study. For each risk assessment model, a sample size calculation will be performed.

Phase 2: development and internal validation of a model for predicting the risk of in-hospital VTE in critically ill patient In phase 2, the sample size needed to develop and internally validate the model was calculated according to a recently published method for sample size calculation for prediction models. Sample size calculation was based on a binary prognostic outcome, the estimated area under the curve (AUC), the estimated amount of candidate prognostic variables and the estimated outcome proportion. An AUC of 0.75 was used as baseline. Assuming an outcome prevalence of 3% in the study sample implicates that apporoximately 7600 patients need to be included to register 230 events for the evaluation of 22 candidate predictor variables.

Statistical analysis:

All analyses will be conducted after data collection has been completed according to, and after publishing of, this protocol. Deviations from this protocol will be reported accordingly in the final manuscript. Patient characteristics will be presented as means (with standard deviations; SD) or medians (with interquartile ranges; IQR) depending on distributions. Categorical data will be presented as proportions. Where appropriate, the investigators will account for clustering between cohorts from different sources independently using meta-analytic techniques.

External validation of existing prediction models:

For external validation, the overall model predictive performance, calibration, discrimination and clinical usefulness will be tested. Overall predictive performance will be tested using Nagelkerke's R2. Discrimination, which is the ability to distinguish patients with and without VTE, will be quantified using the concordance (C) statistic, identical to the area under the curve in a receiver operating characteristic curve. Calibration, which is the agreement between predicted and observed frequency, will be evaluated by modeling a regression line with intercept (α) and slope (β). Decision curve analysis will be performed to evaluate clinical usefulness of prediction models compared to alternative strategies (such as 'treat all patients' or 'treat no patients').

Development and internal validation of a prediction model:

This second objective will only be pursued if external validation of existing models appears insufficient for clinical use and if the predefined effective sample size has been reached. A multivariable prediction model for estimating VTE risk will be constructed. Subsequently, the score will be internally validated to correct for optimism. Normality of the data will be assessed using P-P plots and histograms. Linearity will be assessed using scatter plots. Differences between continuous variables will be assessed using Student's t-tests or Mann-Whitney-U test where appropriate.

The model will be constructed using the following steps:

1. Candidate predictor selection criteria were described above. Definitions are displayed in table 1*.
2. Missing variables (<25%) will be imputed using multiple imputations. Missing variables (>25%) will be excluded. Multiple imputations for missing outcome data will not be performed and patients with missing VTE data will be excluded from all analyses.
3. A binary logistic regression model will be constructed using in-hospital VTE as dependent outcome and the candidate predictors as independent variables. Continuous variables will not be converted to categorical variables. The aim is to include as few variables as reasonably possible to increase simplicity and enhance clinical applicability. Variables will be included based on previous knowledge, clinical reasoning, and observed strengths of associations. The investigators will not use a prespecified significance threshold for elimination. Results will be presented as adjusted Odds ratios (OR) with 95% confidence intervals (CI) and regression coefficients (β-values).
4. The logistic model will be converted to a clinically usable risk assessment model using methods previously described in the Framingham Heart Study. Additionally, the full model will be presented.
5. Internal validation will be performed using bootstrapping
6. Several tests for evaluation of model performance will be used. Overall predictive performance will be tested using Nagelkerke's R2. Discrimination, which is the ability to distinguish patients with and without VTE, will be quantified using the concordance statistic (C). Calibration is by definition expected to be good in the development cohort, and this will be assessed by modeling a regression line with intercept (α) and slope (β). The apparent and optimism-corrected model performance estimates will both be presented.

Data management and analysis will be conducted using STATA version 14.0 or newer (StataCorp, College Station, TX) or R software.

Ethics:

Patients gave informed consent for participation in the SICS-I and SICS-II studies (METc M15.168207). For the registry database a waiver for informed consent was provided (METc M11.104639 and M16.193856). For the AFIB ICU study informed consent from patients or surrogates will be obtained if needed as per national laws.

*Tables 1 is available upon request, please refer to the primary investigator.

Inclusion Criteria:

1. Emergency admission

   Exclusion Criteria:

2. Age < 18 years

3. Planned admission either after surgery or for other reasons