An Observational Cohort Study to Validate SEARCH, a Novel Hierarchical Algorithm to Define Long-term Outcomes After Pulmonary Embolism
An Observational Cohort Study to Validate SEARCH, a Novel Hierarchical Algorithm to Define Long-term Outcomes After Pulmonary Embolism
Potential outcomes after PE occur on a spectrum: complete recovery, exercise intolerance from deconditioning/anxiety, dyspnea from concomitant cardiopulmonary conditions, dyspnea from residual pulmonary vascular occlusion, chronic thromboembolic disease and chronic thromboembolic pulmonary hypertension. Although a battery of advanced diagnostic tests could distinguish each of those conditions, the yield of individual tests among all post- PE patients is low enough that routine testing of all PE patients is not typically performed. Although the various possible post-PE outcomes have enormous implications for patient care, they are rarely distinguished clinically. Perhaps for this reason, chronic conditions after PE are rarely (if ever) used as endpoints in randomized clinical trials of acute PE treatment.
The proposed project will validate a clinical decision tree to distinguish among the various discrete outcomes cost-effectively through a hierarchical series of tests with the acronym SEARCH (for symptom screen, exercise function, arterial perfusion, resting heart function, confirmatory imaging and hemodynamics). Each step of the algorithm sorts a subset of patients into a diagnostic category unequivocally in a cost-effective manner. The categories are mutually exclusive and collectively exhaustive, so that each case falls into one, and only one, category.
Each individual test used in the algorithm has been clinically validated in pulmonary embolism patients, including the cardiopulmonary exercise test (CPET) technique that the investigators developed and validated. However, the decision tree approach to deploying the tests has not yet been validated.
Aim 1 will determine whether the SEARCH algorithm will yield concordant post-PE diagnoses when multiple reviewers independently evaluate multiple cases (reliability).
Aim 2 will determine whether the post-PE diagnoses are stable, according to the SEARCH algorithm, between the first evaluation and the subsequent one six months later (validity).
Study Design Patients with acute pulmonary embolism (PE) receive follow-up care in participating University of California Alliance on Pulmonary Embolism (UCAPE) pulmonary embolism clinics according to the standards of care previously agreed to by the physicians within UCAPE network. At least three months after the onset of acute PE, the patient's physician presents a summary of the case during an on-line meeting without disclosing protected health information or other identifying information. Six designated evaluators independently categorize each patient into a diagnostic endpoint node according to the SEARCH criteria.
SEARCH criteria (positive = at least one criteria met)
Symptoms
Insufficient data: At the first (post-3-month) evaluation, if breathing comfort and exercise tolerance compared to the condition prior to the acute pulmonary embolism could not be ascertained, then the "S" parameter is marked "insufficient data." The patient is excluded from the first primary outcome and the secondary outcome analyses. At the second (6 months subsequent) evaluation, if breathing comfort and exercise tolerance compared to the condition prior to the acute pulmonary embolism could not be ascertained, then the "S" parameter is marked "insufficient data." The patient is be excluded from the second primary outcome and the secondary outcome analyses. If the patient died from pulmonary embolism or pulmonary vascular disease between the first and second evaluations, the patient is to be excluded from the second primary outcome analysis but included in the secondary outcome analyses.If the patient died between the first and second evaluations from a cause other than pulmonary embolism or pulmonary vascular disease, the patient is to be excluded from the second primary outcome and the secondary outcome analyses.
Exercise
Insufficient data: At the first evaluation, if S criteria were positive and the patient did not have a subsequent interpretable CPET, then the E parameter is marked "insufficient data." The patient is excluded from the first primary and secondary outcome analyses. At the second evaluation, if S criteria were positive and the patient did not have a subsequent interpretable CPET, then (1) if the S criteria had not worsened since the first time point (mMRC score had not increased), then the results of CPET from the first evaluation point is accepted as true for the second time point; or (2) if the S criteria had worsened (mMRC score increased) and a CPET was not subsequently performed, then the patient is excluded from the second primary outcome and the secondary analysis.
Arterial perfusion
Insufficient data: At the first evaluation, if S and E criteria were positive and the patient did not have a subsequent interpretable perfusion scan, then the A parameter is marked "insufficient data." The patient is excluded from the first primary outcome and the secondary outcome analyses. At the second evaluation, if S and E criteria were positive and the patient did not have a subsequent interpretable perfusion scan, then (1) if the S criteria had not worsened since the first time point (mMRC score had not increased), then the results of the perfusion scan from the first evaluation point is accepted as true for the second time point; or (2) if S criteria had worsened (mMRC score increased) and a perfusion scan was not subsequently performed, then the A parameter is marked "insufficient data." The patient is excluded from the second primary outcome and the secondary analyses.
Resting echocardiography
Insufficient data: At the first evaluation, if S, E and A criteria were positive and the patient did not have a subsequent interpretable echocardiogram, then the R parameter is marked "insufficient data." The patient is excluded from the first primary outcome and the secondary outcome analyses. At the second evaluation, if S, E and A criteria were positive and the patient did not have a subsequent interpretable echocardiogram, then (1) if S criteria had not worsened since the first time point (mMRC score had not increased), then the results of the echocardiogram from the first evaluation point are accepted as true for the second time point; or (2) if S criteria had worsened (mMRC score increased) and an echocardiogram was not subsequently performed, then the R parameter is marked "insufficient data." The patient is excluded from the second primary outcome and the secondary outcome analyses.
Confirmatory imaging
Insufficient data: At the first or the second evaluations, absence of interpretable confirmatory imaging (CT or pulmonary angiogram) does not cause the patient to be excluded from the final analysis.
Hemodynamics
Insufficient data: At the first or the second evaluations, absence of right heart catheterization does not cause the patient to be excluded from the final analysis.
X criteria
Insufficient data: At the first or the second evaluations, absence of right heart catheterization during exercise does not cause the patient to be excluded from the final analysis.
Six months after the first evaluation, the patient's physician presents the case again, along with any updated data, to the UCAPE reader group. Without knowledge of the consensus first evaluation score or the clinically assigned second evaluation score, each member of the UCAPE reader group in the meeting again independently categorizes the patient with a diagnostic endpoint node, using the same on-line scoring tool used for Aim 1. Aim 1 is further described in the Statistics Section under the first and second primary outcomes.
In the instance in which a patient dies after the first evaluation but before the second evaluation could be performed, the patient's physician reviews the case to determine if the death was more likely than not to be related to (1) PE or other pulmonary vascular disease; or (2) an alternative diagnosis.
Aim 2 will determine whether the post-PE diagnoses are stable, according to the SEARCH algorithm, between the first evaluation and the subsequent one (six additional months later). After the scoring procedure described in Aim 1, the mode of the score from the UCAPE readers will be recorded as the consensus score. Aim 2 is further described in the Statistics Section under the secondary outcome.
Quality Assurance
Data Validation
Study facilitators review presentations to ensure lack of protected health information. The presenters validate accuracy of information in the case presentations.
Data checks
A core group performs quality assurance (QA) reviews of test results for completeness, accuracy, uniformity and clarity of (deidentified) data necessary to categorize patients.
Source data verification
Presenters ensure that study data reflects the source data. The presenters will not disclose the patients' identities or any other protected health information to the study staff.
Data dictionary
Pulmonary Embolism - Medical Subject Heading (MeSH) definition Venous Thrombosis - MeSH definition Stroke volume augmentation (SVA): increase in ventricular stroke volume during exercise, compared to stroke volume at rest Physiological dead space proportion (VD/VT): proportion of the ventilatory air in which gas exchange does not occur Chronic thromboembolic disease (CTED): pulmonary hypertension only during exercise due to intravascular pulmonary arterial scars after acute pulmonary embolism Chronic thromboembolic pulmonary hypertension (CTEPH): pulmonary hypertension due to intravascular pulmonary arterial scars after acute pulmonary embolism Symptomatic residual pulmonary vascular occlusion (RPVO): dyspnea or exercise intolerance with objective evidence of segmental or larger mismatched perfusion defects Symptomatic residual pulmonary vascular occlusion with increased VD/VT (RPVO_VD/VT): RPVO with higher-than-normal VD/VT Symptomatic residual pulmonary vascular occlusion with decreased SVA (RPVO_SVA): RPVO with lower-than-normal SVA Symptomatic residual pulmonary vascular occlusion with increased VD and decreased SVA (RPVO_VD+SVA): RPVO with higher-than-normal VD/VT and lower-than-normal SVA Symptomatic residual pulmonary vascular occlusion not otherwise specified (RPVONOS): RPVO with unspecified physiological effect
Standard Operating Procedure Outline
Presenters review all cases in their UCAPE pulmonary embolism follow-up clinics who meet the inclusion criteria. The presenters and a quorum of at least four evaluators independently grade cases.
Missing data
Data that are unavailable, non-reported, uninterpretable, or considered missing because of data inconsistency or out-of-range results will be marked as "missing."
Statistical analysis and sample size estimates
Primary outcome sample size
Fleiss' kappa statistic will measure agreement among the multiple reviewers about the clinical group designation (1-5) for the first evaluation point. The agreement among interpretations will be graded with commonly accepted criteria for kappa values.
Up to six readers review and score cases. Potential outcomes are grouped into five clinically related groups with the following expected probabilities of occurrence:
Symptomatic recovery (59%)
Dyspnea without CPET defects or dyspnea from alternative diagnoses (19%)
Not distinguished among RPVO vs CTED vs CTEPH among RPVO vs CTED vs CTEPH (14%)
RPVO or CTED (13%)
CTEPH (4%).
The expected kappa will be 0.75. With these assumptions, a validation sample set of n = 150 would produce a lower 95% confidence interval (CI) of 0.7 with an alpha of 0.05.
Secondary outcome sample size
For the secondary outcome, we will group the outcomes into six clinically related groups (pi-x), with the sixth group representing death attributed to PE or pulmonary vascular disease:
Symptomatic recovery
Dyspnea without CPET defects or dyspnea from alternative diagnoses or death from alternative diagnoses
Not distinguished among RPVO vs CTED vs CTEPH
RPVO or CTED
CTEPH
Death from PE, CTEPH or other pulmonary vascular disease.
The investigators expect that more than 90% of patients will remain in the same diagnostic group between the first evaluation and the second evaluation. The investigators anticipate that a study size of 150 would disclose 90% agreement and exclude with 95% confidence agreement in fewer than 15% of patients.
Inclusion Criteria:
Exclusion Criteria:
● Cases in which the clinical presentations contain insufficient interpretable data to permit evaluation of SEARCH criteria, as described in the Study Design section, will be excluded from the analysis.