National mandates for increasing accountability in behavioral healthcare have created an urgent need for effective quality assurance (QA) procedures featuring pragmatic measures of treatment implementation quality in usual care (UC). The most practical method for assessing treatment quality in UC is therapist self-report; unfortunately, therapist-report measures of fidelity to evidence-based practices (EBPs) have demonstrated weak validity to date. This study addresses this need for effective QA procedures by first developing treatment quality assurance procedures designed to increase the adoption and quality of empirically supported family-based services (FBS) for adolescent substance use (ASU) in usual care, and then testing two system-level implementation strategies for installing the new quality procedures in ASU treatment sites.

FBS have achieved the strongest evidence base for treating ASU and are a prime candidate for upgrading the quality of ASU services in various systems of care. FBS comprise both family participation in services, the systemic parameters wherein family members are included in assessment and treatment activities; and family therapy techniques, the specific interventions that clinicians use to directly target family members and family functioning for change. For FBS to fulfill their potential to enhance ASU treatment systems, FBS implementation must be supported by effective quality assurance procedures designed to ensure that FBS are delivered with fidelity. The proposed study will leverage a strong research-government partnership between the applicant organization and the New York single-state agency for SU services.

The study will first develop innovative quality procedures (Aim 1) that use existing FBS quality metrics to promote high-fidelity FBS: Measurement Training and Feedback System for Implementation (MTFS-I), a pragmatic evidence-based method for increasing FBS quality by providing monthly feedback on therapist-reported FBS delivery along with brief online FBS training modules. The study will then experimentally compare two system-level implementation strategies designed to foster MTFS-I utilization in usual care for ASU. Core Training Only will contain two 3-hour training sessions: Mapping existing FBS and identifying site goals for FBS improvement; and Installing and sustaining the MTFS-I. Core + Facilitation is an additive strategy that will begin with the Core Training sessions and then continue with monthly facilitation meetings for one year to promote MTFS-I use and progress toward FBS improvement.

The study will feature a three-group cluster randomized trial testing Core Training Only versus Core + Facilitation versus no-intervention Control in 9 ASU clinics across New York State. MTFS-I utilization data will be collected from Core Training and Core + Facilitation sites for one-year follow-up (after initial Core Training); FBS quality data on family participation and family therapy technique use will be collected from all sites over one-year follow-up; observational coding methods will verify therapist reports of FBS quality; and client outcome data for all sites will be retrieved from administrative data warehouses over one-year baseline and one-year follow-up. These data will enable between-condition comparisons of FBS delivery (Aim 2: MTFS-I utilization, family participation, family therapy technique use) and client outcomes (Aim 3: therapeutic goal achievement, substance use change). If study aims are achieved, investigators would be positioned to mount a fully powered RCT to test the effectiveness of these FBS quality improvement procedures at scale.

Discovering and disseminating effective methods to improve the quality of treatment services for adolescent substance use (ASU) within the national behavioral healthcare system is an urgent public health priority. Despite the strong legislative and policy focus on quality of care evident in the Children's Health Insurance Program Reauthorization Act of 2009 and the Affordable Care Act of 2010, recent comprehensive reports underscore the enduring "quality chasm" between behavioral treatments proven in controlled research versus those commonly practiced in usual care. ASU treatment quality in particular is considered mediocre to inadequate, due to a host of factors headlined by the absence or modest quality of evidence-based services, along with inadequate provider training, little quality monitoring, inattention to data-driven decision-making, and ineffective system-level policies for promoting existing quality mandates.

Family-based services (FBS) are a prime candidate for upgrading the quality of the ASU treatment system. FBS comprise both family participation in services, the systemic parameters wherein family members are included in assessment and treatment activities; and family therapy techniques, the specific interventions that clinicians use to directly target family members and family functioning for change. FBS have reached the highest levels of empirical validation for ASU, posting an exemplary record of success in comparison to alternative evidence-based treatments as well as usual care, and FBS produce the largest average effect sizes by a large margin. Due in large part to this extensive evidence base, FBS have long been strongly endorsed by federal agencies, national associations, and policy-making groups. There is also incentive from ASU clinical providers and payers to deliver FBS, which are now approved for treating ASU and disruptive behavior disorders by federal and private insurance plans and regulatory agencies that govern licensed treatment providers. FBS are also widely endorsed by therapists treating youth in routine care. Thus there is impetus from all corners to expand FBS implementation in usual care for ASU.

There are several well-documented barriers to dissemination of manualized FBS models, including the cost and complexity of delivering these models in everyday settings. In addition, for FBS to fulfill their potential to enhance the quality of ASU treatment in usual care, FBS implementation must be supported by effective quality assurance (QA) procedures designed to ensure that FBS are delivered with fidelity, that is, to the target population, by appropriately trained providers, and in accord with specified procedures. The time is propitious for developing such procedures in the current healthcare market, which is incentivized to establish reliable standards for quality care. First, there is growing demand for innovative quality indicators of behavioral treatment that assess appropriateness and potential effectiveness of care. Conventional quality indicators capture broad principles of behavioral care such as treatment assignment, retention and follow-up rates, referrals for ancillary care, and client safety. However, the emerging quality-of-care implementation framework advocates that fidelity to evidence-based treatments itself be considered a quality indicator. Second, in order to properly monitor treatment fidelity in usual care, there is urgent need to develop quality metrics that can reliably and pragmatically measure fidelity in front-line treatment settings. The implications of emerging scientific and policy mandates for improving treatment quality are clear with regard to FBS for ASU: Pragmatic QA procedures for ensuring high-fidelity FBS need to be developed, and these procedures need to be anchored by reliable FBS fidelity metrics.

This study will develop pragmatic QA procedures designed to promote FBS adoption and quality in ASU treatment systems using a measurement feedback system (MFS). MFS is a performance feedback loop in which a given quality indicator is continuously monitored by the clinician to gauge case progress and support clinical decision-making. MFS feedback loops usually take the form of easy-to-digest data reports that provide summary appraisals of individual client progress on selected indicators in comparison to a desired benchmark. To date MFS has been used in mental health care to enable monitoring of client outcomes primarily-for example, therapists tracking weekly client-report depression scores compared to age-adjusted norms on a depression inventory. With adults, utilizing MFS has led to impressive gains in outcomes with diverse samples: preventing early treatment failure, reversing symptom deterioration, and enhancing overall outcomes. MFS research on youth is in its early stages but growing rapidly, with strong enthusiasm about reaping comparable benefits. Importantly, clinicians trained in MFS develop positive attitudes toward it.

MFS successes for client outcomes have generated enthusiastic support for the value of developing complementary MFS procedures that provide measurement Training along with data-based Feedback on implementation: MTFS-I. Due to streamlined administration procedures and emerging evidence, MTFS-I has been labeled a promising QA strategy with broad dissemination potential for youth. MTFS-I has already been incorporated into manualized QA procedures to bolster fidelity to several varieties of manualized treatment; in a few of these instances, feedback reports contain data input from therapists on fidelity indicators. One study found that MTFS-I for clinician-reported session content during youth therapy increased the likelihood and rapidity of addressing that same content in future sessions. In keeping with primary goals of the R34 mechanism, this study will test a pragmatic MTFS-I designed to improve the quality of FBS implementation in ASU treatment sites. The MTFS-I will draw on technology from existing feedback systems, but also, be the first to target system-level FBS delivery for ASU. The four MTFS-I components, described in C4, are designed to increase the extensiveness (i.e., amount and frequency of) FBS implementation.

Inclusion Criteria:

• Therapists treating adolescents aged 12-21 at partnering treatment sites

Exclusion Criteria:

-