The Influence of Explainability and Integrability of AI-C... | NCT07401979

NCT07401979

Status Module

Description Module

Brief Summary

The goal of this observational experimental study is to determine how system-level features of artificial intelligence clinical decision support systems (AI-CDSS)-specifically explainability and integrability-affect usage behavior among primary care physicians in China. The study focuses on licensed primary care physicians, regardless of gender, age, years of clinical experience, or prior AI exposure.

The main questions it aims to answer are:

Do specific AI features (e.g., feature attribution, chain-of-thought explanation, seamless workflow integration, automated data input) independently influence physicians' adoption intention, diagnostic accuracy, and their perceptions of the system's usefulness and ease of use?
Do pairwise combinations of these AI features produce significant interaction effects-either synergistic or antagonistic-on these outcomes? Researchers will compare 32 distinct AI interface configurations generated from a 2⁶-¹ fractional factorial design (Resolution VI), each representing a unique combination of six binary AI features: (A) gradient-based feature importance (0 = absent, 1 = present), (B) chain-of-thought reasoning (0/1), (C) workflow integration (0 = multiple pop-up alerts, 1 = unified sidebar display), (D) automated data extraction (0 = manual entry, 1 = auto-populated from case text), (E) recommendation scope adapted to primary care settings (0 = restricted to essential options, 1 = full range of recommendations), and (F) model confidence display (0 = absent, 1 = present). This design enables unbiased estimation of all six main effects and all 15 two-way interactions.

Participants will:

Complete three standardized clinical case scenarios involving common respiratory infections via a web-based simulation platform; First provide an initial diagnosis and treatment plan without any AI input; Then review an AI-generated recommendation embedded with a randomly assigned combination of the six AI features; Revise their final diagnosis and prescription based on the AI suggestion; Rate their adoption intention, perceived usefulness, and perceived ease of use using validated 7-point Likert-scale items after each case.

Design Module

Interventional Study Design

Biospecimen

No data available

No data is available for this block.

Arms/Interventions Module

Arm Groups

Label	Type	Description	Intervention Names
AI-CDSS Basic Configuration	No Intervention	Participants interact with an AI Clinical Decision Support System with the following feature settings: Sidebar Display (disabled), Chain-of-Thought Reasoning (disabled), Confidence Display (disabled), Feature Importance (disabled), Stepwise Medication (disabled), Auto-extraction (disabled). Participants complete 3 clinical case evaluations for respiratory tract infections using this AI configuration.
AI-CDSS Configuration 2	Experimental	Participants interact with an AI Clinical Decision Support System with the following feature settings: Sidebar Display (disabled), Chain-of-Thought Reasoning (disabled), Confidence Display (disabled), Feature Importance (disabled), Stepwise Medication (enabled), Auto-extraction (enabled). Participants complete 3 clinical case evaluations for respiratory tract infections using this AI configuration.	Behavioral: AI-CDSS with Stepwise Medication and Auto-extraction
AI-CDSS Configuration 3	Experimental	Participants interact with an AI Clinical Decision Support System with the following feature settings: Sidebar Display (disabled), Chain-of-Thought Reasoning (disabled), Confidence Display (disabled), Feature Importance (enabled), Stepwise Medication (disabled), Auto-extraction (enabled). Participants complete 3 clinical case evaluations for respiratory tract infections using this AI configuration.	Behavioral: AI-CDSS with Feature Importance and Auto-extraction
AI-CDSS Configuration 4	Experimental	Participants interact with an AI Clinical Decision Support System with the following feature settings: Sidebar Display (disabled), Chain-of-Thought Reasoning (disabled), Confidence Display (disabled), Feature Importance (enabled), Stepwise Medication (enabled), Auto-extraction (disabled). Participants complete 3 clinical case evaluations for respiratory tract infections using this AI configuration.

Interventions

Name	Type	Description	Arm Group Labels	Other Names
AI-CDSS with Stepwise Medication and Auto-extraction	Behavioral	An artificial intelligence-based Clinical Decision Support System for respiratory tract infections. Active features: Stepwise Medication; Auto-extraction. Inactive features: Sidebar Display; Chain-of-Thought Reasoning; Confidence Display; Feature Importance. The system provides diagnosis suggestions and antibiotic recommendations for primary care physicians.

Outcomes Module

Primary Outcomes

Measure	Description	Time Frame
Correction Event Rate	Proportion of cases where the physician's initial decision was incorrect, but was corrected to the right decision after viewing AI recommendations. Calculated as: (Number of cases where initial decision was wrong AND final decision was correct) / (Total number of cases where initial decision was wrong). This measures the positive corrective influence of AI-CDSS on physician decision-making. Continuous variable ranging from 0 to 1.	Immediately after intervention
Misleading Event Rate	Proportion of cases where the physician's initial decision was correct, but was changed to an incorrect decision after viewing AI recommendations. Calculated as: (Number of cases where initial decision was correct AND final decision was wrong) / (Total number of cases where initial decision was correct). This measures the potential negative influence of AI-CDSS on physician decision-making. Continuous variable ranging from 0 to 1.	Immediately after intervention
Diagnostic Accuracy (Top-1)	Proportion of cases where the physician's final primary diagnosis (first-ranked diagnosis) matches the correct diagnosis (gold standard). Measured separately for initial diagnosis (before AI) and final diagnosis (after AI) to assess AI impact on diagnostic accuracy. Continuous variable ranging from 0 to 1.	Immediately after intervention
Diagnostic Accuracy (Top-3)	Proportion of cases where the correct diagnosis (gold standard) appears within the physician's top 3 differential diagnoses. Measured separately for initial diagnosis (before AI) and final diagnosis (after AI). This captures whether the correct diagnosis was considered even if not ranked first. Continuous variable ranging from 0 to 1.	Immediately after intervention
Appropriateness of Antibiotic Use Decision

Secondary Outcomes

Measure	Description	Time Frame
Decision Certainty	Physician's self-reported confidence level in their final clinical decision after viewing AI recommendations. Measured using a 5-point Likert scale: 1 = Very Uncertain, 2 = Uncertain, 3 = Neutral, 4 = Certain, 5 = Very Certain. This reflects the cognitive impact of AI-CDSS on decision confidence.	Immediately after intervention

Eligibility Module

ID	Term
D007239	Infections
D012140	Respiratory Tract Diseases