Evaluation of the Impact of AI-Based Cardiac CT Interpretation Tool on Emergency Physicians' Decision-Making
Evaluation of the Impact of AI-Based Cardiac CT Interpretation Tool on Emergency Physicians' Decision-Making
" This prospective, pragmatic, randomized controlled trial is designed to evaluate the impact of an artificial intelligence (AI)-based coronary computed tomography angiography (CCTA) interpretation tool (Angiomics) on emergency physicians' diagnostic performance and clinical decision-making in patients presenting with acute chest pain.
CCTA is a critical diagnostic modality for suspected acute coronary syndrome (ACS) in the emergency department (ED). Accurate interpretation often requires experienced radiologists, who may not always be available, particularly during off-hours. The introduction of AI-based interpretation tools into clinical workflow has the potential to enhance diagnostic accuracy, increase physician confidence, reduce delays in decision-making, and improve efficiency of resource utilization. However, evidence regarding the real-world effectiveness of such AI tools in the ED setting remains limited.
Eligible participants will include adults aged 18 years or older presenting to the ED with chest pain and classified as intermediate risk (HEART score 4-6). Participants will be randomized into two groups: (1) AI-assisted CCTA interpretation, in which emergency physicians interpret scans with access to AI results; and (2) standard interpretation, in which emergency physicians interpret CCTA without AI support. In both groups, physicians will document the presence of stenosis in the four major coronary arteries (LM, LAD, LCX, RCA) and report diagnostic confidence on a 5-point Likert scale.
The primary outcome is the negative predictive value (NPV) of CCTA interpretation at the patient level, comparing AI-assisted versus standard interpretations against the reference standard of blinded consensus readings by board-certified radiologists. Secondary outcomes include sensitivity, specificity, positive predictive value (PPV), accuracy, diagnostic confidence, vessel-level diagnostic performance, and agreement with radiologist consensus using Cohen's Kappa.
The study aims to enroll approximately 530 participants (276 in the control arm and 254 in the intervention arm, accounting for an expected 10% dropout). Enrollment and follow-up will be conducted at Severance Hospital and Gangnam Severance Hospital over a 24-month period following IRB approval. The results are expected to provide evidence for the clinical utility and effectiveness of AI-based CCTA interpretation in the ED and to guide integration of AI into emergency care in order to optimize patient outcomes and healthcare efficiency.
"This study is a prospective, pragmatic, randomized controlled trial designed to evaluate the impact of an artificial intelligence (AI)-based coronary computed tomography angiography (CCTA) interpretation tool (Angiomics) on the diagnostic accuracy, confidence, and decision-making of emergency physicians in patients presenting with acute chest pain.
Background and Rationale CCTA is widely used in the emergency department (ED) for patients presenting with acute chest pain or suspected acute coronary syndrome (ACS). It provides rapid and precise visualization of the coronary arteries and assists emergency physicians in making time-sensitive clinical decisions. However, interpretation of CCTA requires specialized training and experience, which may not always be available, particularly during nights or in resource-limited settings. This limitation can delay diagnosis and treatment, potentially leading to adverse patient outcomes.
Recent advances in AI technology have enabled the development of automated tools capable of analyzing CCTA images and identifying clinically significant findings such as coronary artery stenosis and myocardial ischemia. These tools have the potential to support emergency physicians by improving diagnostic accuracy, reducing interpretation time, enhancing physician confidence, and optimizing the use of healthcare resources. Despite these advantages, the influence of AI-based interpretation on real-world ED clinical workflows and physician decision-making remains insufficiently studied.
The present trial is designed to address this knowledge gap by evaluating whether the integration of an AI-based CCTA interpretation tool improves emergency physician diagnostic performance and clinical confidence, and whether such improvements translate into more reliable decision-making in the ED.
Study Design
This study will employ a prospective, pragmatic, randomized controlled design. Eligible participants will be adults aged 18 years or older presenting to the ED with chest pain and classified as intermediate risk according to the HEART score (4-6 points). After obtaining informed consent, participants will be randomized into two groups:
All physicians will evaluate stenosis in the four major coronary arteries (left main, left anterior descending [LAD], left circumflex [LCX], and right coronary artery [RCA]) and record their findings using a standardized format (Yes/No/Non-diagnostic/Other findings). In addition, physicians will rate their diagnostic confidence on a 5-point Likert scale ranging from 1 (no confidence) to 5 (very high confidence).
Reference Standard The reference standard for diagnostic performance will be the blinded consensus interpretation of board-certified radiologists specialized in thoracic imaging. Previous studies have demonstrated concordance of 98-99% between expert radiologist consensus and invasive coronary angiography, supporting the validity of this approach.
Outcomes The primary outcome is the negative predictive value (NPV) of CCTA interpretation at the patient level, comparing AI-assisted versus standard interpretations. Patient-level outcomes will be derived by aggregating findings across the four major coronary arteries.
Secondary outcomes include:
Sample Size and Statistical Analysis Sample size estimation was performed using PASS software based on published data and local prevalence. Accounting for a prevalence of 18%, the calculated sample size required is 181 patients with negative diagnoses in each group. This corresponds to 248 participants in the control arm and 228 in the intervention arm. Assuming a 10% dropout rate, the final enrollment target is 530 participants (276 in the control arm and 254 in the intervention arm).
An interim analysis will be conducted after 50% enrollment, with alpha-spending adjusted using the O'Brien-Fleming method. Final analysis will be performed with a two-sided alpha of 0.049.
Diagnostic performance (sensitivity, specificity, PPV, NPV, accuracy) will be compared using chi-square or Fisher's exact tests as appropriate. Agreement between physician and radiologist consensus readings will be assessed using Cohen's Kappa. All statistical tests will be two-sided, and significance will be set at p < 0.05.
Data Collection and Confidentiality Data will be collected within the routine clinical workflow and stored in a secure, de-identified format according to institutional policies and national privacy regulations. Only authorized investigators will have access to the data. Data will be stored for three years after study completion before secure disposal.
Safety and Monitoring This trial poses minimal risk to participants, as it involves no procedures outside of routine clinical care. No additional radiation, medications, or invasive interventions are introduced by study participation. The study principal investigator will monitor protocol adherence and data quality on an annual basis.
Significance This study will provide evidence regarding the clinical utility of an AI-based CCTA interpretation tool in the ED. Findings are expected to clarify whether AI support improves diagnostic accuracy and physician confidence, and whether it can be effectively integrated into real-world emergency care. The results may inform best practices for the implementation of AI tools to optimize patient outcomes and resource utilization in acute care settings.
Inclusion Criteria:
Exclusion Criteria:
aromchoi@yuhs.ac+82-2-2228-2465