The Efficacy of Technology-Based Tools in Applied Behavior Analysis Treatment Planning for Autism Spectrum Disorder
The Efficacy of Technology-Based Tools in Applied Behavior Analysis Treatment Planning for Autism Spectrum Disorder
Technology enhancement in Applied Behavior Analysis (ABA) treatment planning may increase confidence, efficiency, consistency, and satisfaction for Board Certified Behavior Analysts (BCBAs) which, in turn, can provide for better clinical outcomes for patients on the autism spectrum. To this end, the investigators will examine the use of >3 technology-based tools that will be implemented in the BCBAs' clinical workflow to aid with treatment planning. The study will initially involve two aims that are non-interventional (these processes will occur in the background and will have no impact on any cohorts), followed by an interventional aim that includes two arms (i.e., two BCBA cohorts). BCBAs within both arms will observe and practice the standard of care for ABA, and thus patient care will not be impacted. The outcome measures are primarily focused on the BCBAs as follows: Arm 1: An experimental group (BCBA Tech cohort) will receive the full tech package (TREAAT2+) from the start. Arm 2: The control group (BCBA non-Tech cohort) will not have access to any tools from the tech package for the first 6 months. In the subsequent 18 months, they will receive one tool every 6 months until gaining access to the entire tech package.
Autism spectrum disorder (ASD) is a complex, heterogeneous neurodevelopmental disorder that accounts for >$250 billion in direct and indirect annual expenditures and is estimated to occur in 1 out of 36 children <8 years of age in the US. Validated ASD treatments, such as Applied Behavior Analysis (ABA), rely upon Board Certified Behavior Analysts (BCBAs) to develop highly individualized treatment plans via a complex and labor-intensive process. Integrating technology-based approaches for treatment planning (largely unexplored in the context of ABA), such as data-driven clinical decision support (CDS) systems and assistive technology, can modernize and optimize the BCBA workflow, which, in turn, can enhance patient care. There has been a high demand for BCBAs in recent years, which has led to shortages straining care providers, with about 72% of BCBAs experiencing significant burnout, increasing BCBA turnover. One contributing factor to the BCBA burnout rate is the workload, which can fuel exhaustion and disengagement.The BCBA workflow includes significant time spent on developing effective individualized treatment plans via a tedious, non-automated, and heterogeneous process lacking standardized tools. Thus, there is a need to automate existing workflows to increase BCBA efficiency, confidence, consistency, and satisfaction, in order to mitigate burnout and consequently improve the patient management process, and by extension patient clinical outcomes.
In this SBIR project, the investigators propose to integrate a data-driven technological package (TREAAT2+) into the BCBA's workflow to assist with streamlined and consistent ABA treatment planning. TREAAT2+ consists of (1) a machine learning algorithm (MLA)-based CDS tool that analyzes data from electronic health records (EHRs) and recommends treatment dosage in terms of hours, where the MLA is integrated into a proprietary application ("app"); (2) a treatment planning software tool integrated with the proprietary app to facilitate highly accessible treatment oversight; and (3) individual patient progress reports pushed onto the proprietary app from Autism Analytica (AA). The predecessor of the app-integrated MLA, TREAAT, was validated and achieved excellent performance (AUROC of 0.895) for a binary treatment dosage recommendation (<20 or >20 hours/week). The investigators will enhance the capacity of the MLA for more granular treatment dosage recommendations, deploy a treatment planning software tool in the app for BCBA use, and provide pushed AA patient data assessments in the app. This will improve BCBA efficiency and confidence within their workflow, and thereby significantly reduce the burden related to the manual and subjective nature of the treatment planning process. TREAAT was validated with proprietary data from patients of Montera Health TX LLC ("Montera"), and the investigators will use a larger number of patients to fine-tune and validate the app-integrated MLA to improve generalizability. The investigators expect that the MLA will perform as well as or better than the original TREAAT in this expanded patient population and that the MLA, in conjunction with the treatment planning software tool and the pushed AA data, will significantly improve the BCBA workflow. The lack of current workflow automation coexists with significant BCBA burnout rates, and TREAAT2+ provides the solution of modernizing time-consuming tasks within treatment planning. By bridging the technological gap in the BCBA workflow, TREAAT2+ will mitigate BCBA burnout, and by extension improve patient care.
Study Aim 1: Retraining and upgrading the MLA of TREAAT2+. The investigators will use a larger set of retrospective and prospective Montera patient data than was employed for the original TREAAT training, and will design the MLA output as increments of treatment dosage recommendation (in 10 hr increments). The investigators will additionally integrate the MLA into our proprietary app. The investigators expect MLA performance metrics to be comparable to or better than retrospective benchmarks from the pilot study (AUROC: 0.895; 95% CI: 0.811 - 0.962).
Study Aim 2: Test the robustness of the MLA of TREAAT2+ in treatment plan development. The investigators will conduct a non-interventional prospective evaluation of the app-integrated MLA. The agreement between the incremental treatment dosage suggested by the MLA and the dosage prescribed by the BCBA will be assessed. MLA performance will be evaluated on demographic subpopulations to ensure bias minimization. The investigators expect that there will be substantial agreement between the treatment dosage prescribed by the BCBA and the dosage suggested by MLA, as measured by minimum inter-rater reliability (e.g., Cohen's Kappa) of greater than 0.6, which indicates substantial agreement according to the Landis and Koch's classification system.
Study Aim 3: Evaluate the impact of deploying TREAAT2+ within the BCBA workflow. The investigators will utilize prospective data from two BCBA cohorts. An experimental group (BCBA Tech cohort) will receive the full tech package (TREAAT2+) from the start. The control group (BCBA non-Tech cohort) will not have access to any tools from the tech package for the first 6 months. In the subsequent 18 months, they will receive one tool every 6 months until gaining access to the entire tech package. The investigators expect to demonstrate the efficacy of TREAAT2+ with statistically significant improvement (p < 0.05) over baseline in qualitative endpoints (efficiency, confidence, consistency, and satisfaction; as measured by BCBA self-reported Likert questionnaire scales) and quantitative endpoints (time allocated to treatment plan development).
Inclusion Criteria:
Exclusion Criteria:
qmao@fortahealth.com4158051725