Evaluating a Virtual Mentoring Program Plus Mental Health Videos for Teens

Digital Delivery of Evidence-Based Mental Health Content and Mentoring to Adolescents

This clinical trial evaluates Appa Health, a mental health smartphone and computer app for teenagers. Appa is built from two components. Component #1 shares short TikTok-style videos with teens to cover techniques from a science-backed mental health treatment, Cognitive Behavioral Therapy. These videos are designed by mental health experts and delivered by mental health influencers. Component #2 pairs teens with peer mentors with similar life experiences, who help guide teens through each week's video content. All peer mentors are trained and supervised by licensed mental health experts. The researchers believe Appa Health improves teens' mental health and well-being through teens' strengthened relationships and support from their mentor, as well as through the skills they gain through watching the videos. This study hopes to understand: (1) whether these Appa Health components improve teens' mental health and well-being, (2) how these components might improve teen outcomes, and (3) costs associated with each of these components. To answer these questions, the researchers will compare two versions of Appa support: Appa Complete and Appa Lite. Appa Complete involves pairing teens with mentors and sharing videos. Appa Lite involves only sharing videos with teens. First, the researchers will conduct a small version of this study to test procedures. Seventy-five teenagers (13-18 years old) with high depression or anxiety scores will be randomly given either: Appa Complete (mentoring + videos), Appa Lite (videos only), or a Waiting List Control. Based on the results of this smaller study, the researchers will change study procedures to improve teens' experiences and engagement in this research. After completing this smaller study, the researchers will run a larger version of the same study with 400 more teens. After teens are randomly given Appa Complete, Appa Lite, or a Waiting List Control, the researchers will test whether Appa's two treatment components improve teens' depression, anxiety, well-being, and other related outcomes. The researchers expect both Appa Complete and Appa Lite will improve teens' outcomes more than the Waiting List Control. The researchers also expect Appa Complete will improve teens' outcomes more than Appa Lite. The researchers will also measure costs associated with giving teens both treatment components and use this information to improve Appa's services for future teens.

Pilot Study (1 year). Optimize Study Procedures and Participant Retention (6 months). In an initial pilot study with N=75 youth, the researchers will examine the adapted Appa Health and research study content, offering data for optimization and retention incentive decisions prior to larger scale RCT through the following research questions (RQs): RQ 1.1: Are research and incentive protocols sufficient for recruiting and retaining adolescents in the study, and how can they be optimized? RQ 1.2. Do Appa's modified engagement strategies developed in preliminary studies increase use of CBT video content? This pilot study will enable the research team to identify and address any problems in recruitment, study procedures, and retention prior to the effectiveness trial.

Setting. The study will be carried out within the Appa Health remote platform, which comprises over 20 trained youth mentors serving, on average, 400 youth every 3 months. Clients come primarily from school-based partnerships. Appa has several existing school partners who refer students to Appa either through a paid partnership from the school or through access to parents of students.

Recruitment and Randomization. Participants will be recruited from the Appa Health client population using email outreach. As described in the Commercialization Plan, the Appa client base from individual paying clients, school systems, and B2B customers is expected to be sufficient to meet this goal within the study timeframe. Screening of youth using the PHQ-8 and GAD is standard protocol for Appa; those who score above 10 on either instrument will be invited to participate in the study. Due to the relatively small sample size of the pilot trial, the research team will use stratified random sampling to ensure approximate balance to the three study conditions on baseline covariates (gender, age, baseline severity on GAD & PHQ-8, race/ethnicity, and LGBTQ status), with stratification blocks created via nearest neighbor matching on all variables.

Interventions. Youth will be randomized to three conditions, which in the larger RCT will enable analyses of the main effect of the traditional Appa package as compared to a waiting list control group, as well as exploring the impact of a light touch version of Appa for commercial purposes. This will also permit examination of the impact of the two components on specific psychosocial mechanisms, as appropriate in an experimental therapeutics paradigm.

"Appa Lite": Unguided EBP videos and materials. Participants assigned to the Appa Lite condition will have unlimited access to the Appa website, including short form CBT-based videos and materials such as an emotions wheel pdf and thoughts, feelings, and behavior worksheets.

"Appa Complete": Weekly mentorship meetings & guided CBT videos. Participants assigned to Appa Complete will have unlimited access to the Appa website and materials described in Appa Lite, along with weekly mentorship meetings with a trained and supervised near-peer mentor with whom they have been matched.

"Waiting List Control": Participants assigned to the waitlist control will receive a small pdf packet of psychoeducational materials and will be permitted to use Appa at the end of the 12 weeks of data collection. Families assigned to this condition will not pay for Appa's services while on the waitlist.

Data Collection. The REDCap electronic data collection application will be used to collect data from mentors, supervisors, and caregivers at enrollment (baseline) and at midpoint (6 weeks) and end (12 weeks).REDCap will also be used to collect survey data from adolescents at enrollment (baseline) and at midpoint (6 weeks) and end (12 weeks). Adolescent data collection will include quantitative and qualitative surveys and assessments focused on therapeutic interactions (mentorship fidelity, frequency of video watching and mentor interactions, supportive accountability), mechanisms of change (e.g., use of CBT, confidence in relationships, reduced isolation), clinical outcomes (depression, anxiety, top problems, quality of life/flourishing, self-esteem), possible moderators (demographics, alliance, mentor-adolescent match), and study-related measures (research burden, open-ended comments). To control for compensatory mental health service use by participants in any condition, the prohibition of which in the study would be unethical, the research team will collect data on non-Appa services received elsewhere, including mentorship. Adolescents will be incentivized with $25 per survey timepoint and non-monetary, low burden incentives such as humorous messaging and GIFs. Mentor data collection will focus on mentor fidelity to the supportive accountability model (coded audio recordings of mentor sessions and text communication), moderators (surveys of alliance, mentor-client match), and time diaries for the cost analysis. Mentor supervisors will also provide time diary data. Caregivers will provide data on caregiver strain and the financial burden of adolescent care due to mental health, and caregivers will be incentivized with $5 per administration. Fidelity to mentorship will be measured by coding three randomly selected audio recordings of mentorship sessions and 10 randomly selected text exchanges for each client. Cost assessments. Activity-based costing will estimate the incremental cost of Appa Complete and Appa Lite. The research team will estimate the direct cost of video production and mentoring including web hosting, payments to video creators, mentor salary, training, and supervision. Data collection using time diaries will occur via online surveys of mentors and supervisors. The research team will identify activities related to mentoring and associated labor and non-labor inputs such as time, supplies, travel, overhead, and costs associated with meetings. Activities will be categorized to facilitate understanding of cost allocation. The price of goods and services will be available via budget and expense reports.

Effectiveness Trial: Study Appa Costs and Effectiveness (2.5 years). A large randomized controlled trial with N=400 youth will replicate the pilot study, excluding measures collected on research burden and incentive satisfaction. If the research team does not make substantive changes to the product after the initial pilot trial, the main effectiveness trial analysis will include all 75 pilot participants (total N=up to 475 youth). The three-arm design will enable Appa to determine the cost and effectiveness of each Appa component and enable evidence-based decision making on the commercial viability of scaling these components. Appa will learn the degree to which users engage in components, the cost of those components, and the impact of those components on important mechanisms and outcomes. The research team will test the following hypotheses: H1a-j: At 12 week follow-up, teens in the Appa Complete condition will have significantly greater improvement than both other conditions on primary outcomes of a) youth top problem severity, b) use of CBT skills, c) self-efficacy for CBT; as well as secondary outcomes of youth d) depression, e) anxiety, f) quality of life, g) self-esteem, h) existential isolation, and caregiver i) strain, and h) financial burden. H2a-h: Clients in the Appa Lite condition will have superior outcomes than the control group on a) use of CBT skills, b) depression, c) anxiety, d) top problem severity, e) quality of life, f) self-esteem, g) caregiver strain and h) caregiver financial burden; H3a-d: Near peer mentoring will increase engagement in digital tools and mechanisms such that those who receive mentoring (Appa Complete), when compared to Appa Lite, will a) watch more videos, b) have greater efficacy for CBT skills, c) report greater use of CBT skills, and d) report greater meaning and purpose. Exploratory research questions: ERQ1: Do hypothesized mechanisms (meaning and purpose, identification with mentor, self-efficacy for using CBT, fidelity to practice and supportive accountability, and use of CBT skills) mediate the impact of Appa on depression, anxiety, and top problem severity? ERQ2: Is the impact of Appa on top problem severity, depression, and anxiety moderated by client demographics, mentor alliance, frequency of meetings, degree of demographic match between client and mentor, or dosage of video use? The research team is particularly interested in whether there are differential outcomes by race and social identity.

Cost-effectiveness. H4: Appa Complete will be more expensive but more cost-effective than Appa Lite.

Participants. Inclusion and exclusion criteria will replicate the pilot study.

Recruitment. Recruitment methods will replicate the pilot study. The research team will recruit approximately 400 adolescents, roughly 133 in each condition. The research team assumes 20% attrition for a final complete sample of 320; however, these analytic methods permit inclusion of all available data including dropouts. If the research team does not make substantive changes to the product after the initial pilot trial, the main effectiveness trial analysis will include all 75 pilot participants (total N=up to 475 youth).

Randomization. Random assignment to condition will occur using urn randomization at the client level to ensure roughly equal numbers in each condition based on key hypothesized moderators including gender, age, baseline severity on GAD & PHQ-8, race/ethnicity, and LGBTQ status. Urn randomization is an adaptive biased-coin method that ensures balance on multiple covariates in rolling-enrollment designs but approaches complete randomization in larger sample sizes.

Data collection. Data collection will replicate the pilot study.

Inclusion Criteria (Youth):

• 13 to 18 years old
• Generalized Anxiety Disorder screen or Patient Health Questionnaire screen scores of 10 or higher
• Living in the United States
• Speak English
• Read/comprehend English at a 5th grade level or higher (treatment delivery is in English)
• Able to appropriately and independently engage with study intervention (i.e., without parental support)

Inclusion Criteria (Caregivers):

- Must be a caregiver (e.g., parent, grandparent, custodian) of a youth participant in the study

Inclusion Criteria (Mentors):

- Must be a mentor with a youth mentee in the study

Inclusion Criteria (Supervisors):

- Must supervise a mentor with a mentee in the study

Exclusion Criteria (Youth):

• Psychotic disorders
• Substance use disorders
• Active engagement in self-harm
• Active suicidal ideation (intent or plan)