Due to an error in the implementation of the project, CDS was provided to all providers, rather than to the intended random sample of providers.

The goal of the study is to determine whether clinical decision support (CDS) affects the number, type, or appropriateness of targeted high-cost radiology images (i.e. MR and CT) ordered. The CDS will be delivered in Epic through ACRSelect software, which is a leading decision support tool based on the American College of Radiology (ACR) Appropriateness Criteria (see http://www.acr.org/Quality-Safety/Appropriateness-Criteria), and presents the ACR appropriateness scores for each image on a scale of 1-9 with 1-3 labelled as 'usually not appropriate', 4-6 'May be appropriate', and 7-9 'usually appropriate'.

With healthcare spending accounting for almost one-fifth of the U.S. economy and an even larger share of public sector budgets, there is substantial interest in innovations in healthcare delivery that can reduce the "over use" of resources that have no or low value to patients. As a result, there is a key need for rigorous evidence on scalable interventions aimed at improving the efficiency of the U.S. healthcare sector in general, and in the public sector in particular, which accounts for $1.25 trillion in annual healthcare spending (Centers for Medicare & Medicaid Services, 2013).

In particular, there is widespread concern in both the medical profession (Callaghan et al., 2014; Sherman, 2012) and the public sector (U.S. Government Accountability Office, 2008) of the cost and health risks of "over-scanning". Estimates suggest that as many as 30% of imaging in the U.S. are unnecessary (Consumer Reports, 2015; Dehn et al., 2000; Georgiou et al., 2011). Medicare direct spending on "high-cost" scans (e.g. MRs and CTs) was about $10 billion in 2012, or about 2% of total Medicare costs (Medicare Payment Advisory Commission, 2014); the indirect costs are likely considerably greater, since imaging often triggers additional follow up care (Sherman, 2012; Shreibati and Baker, 2011). It is also estimated that about 2 percent of cancers in the U.S. are due to CT use (Brenner and Hall, 2007).

Reflecting this concern, starting in 2017 Medicare will no longer reimburse for high-cost scans unless ordered using an "acceptable" Clinical Decision Support (CDS) system (Pitts, 2014). Despite this upcoming policy change, we know of no large-scale randomized trials on the impact of CDS for imaging.

The intervention in this study provides Clinical Decision Support (CDS) for targeted high-cost radiology orders, MR and CT scans, to healthcare providers treating patients in outpatient settings affiliated with Mount Sinai Hospital and Mount Sinai Queens hospital in New York City. CDS is a tool embedded in an order entry system that provides information and guidance to providers on whether their intended order is "appropriate" and whether there are more highly recommended alternatives. The randomization is at the provider level: one group will receive the CDS, while the remaining providers will serve as the control group.

The CDS will be delivered through the order-entry software, Epic, through ACRSelect software, which is a leading decision support tool based on the American College of Radiology (ACR) Appropriateness Criteria.

Recommendations that appear in the CDS tool are a computerized version of guidelines created by the American College of Radiology (ACR). The guidelines score the appropriateness of a scan order for a given health indication, where indications include common symptoms and diagnosis keywords, such as "acute headache." In particular, indication-scan pairs are assigned an "appropriateness rating" from 1-9. Scores 1-3 are 'usually not appropriate,' 4-6 are 'may be appropriate,' and 7-9 are 'usually appropriate.'

Inclusion Criteria:

• Outpatient medical provider with scan ordering permissions at Mount Sinai Hospital or Mount Sinai Queens
• Has an active ID in the Epic electronic medical record ordering system

Exclusion Criteria:

• Opted out of the study prior to October 13, 2015