Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| University of Pennsylvania | OTHER |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
The objective is to compare the accuracy, costs, and cost-effectiveness of three fidelity measurement methods to assess fidelity to cognitive-behavioral therapy for youth. The investigators will randomize 135 therapists, implementing cognitive-behavioral therapy, to 3 conditions: self-report, chart stimulated recall, and behavioral rehearsal (N = 45 for each group). To calculate the outcomes of interest, each condition will be compared to the gold-standard fidelity measurement method, direct observation.
To achieve the primary objective, the investigators plan to (1) identify the most accurate fidelity measurement method; (2) estimate the economic costs and cost-effectiveness of the fidelity measurement methods; and (3) compare stakeholders' motivation to use each method, as well as identify their perceived barriers and facilitators to use of each method. This study will have a significant positive impact in two ways. First, it will validate fidelity measurement methods that can be used for research. Second, it will produce tools that can be used by community mental health clinics to monitor therapist fidelity, an indicator of therapy quality.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Self-report | Experimental | Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated. |
|
| Chart Stimulated Recall | Experimental | Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used. |
|
| Behavioral Rehearsal | Experimental |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Self-report | Behavioral |
| ||
| Chart-Stimulated Recall |
| Measure | Description | Time Frame |
|---|---|---|
| Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores | Client sessions for all 3 conditions were scored for CBT adherence via direct observation using the Therapy Process Observational Coding System-Revised Strategies (TPOCS-RS) Scale (McLeod et al., 2015) on a 7-point Likert Scale (1='not at all,' 7='extensively'). We analyzed the TPOCS-RS' Maximum CBT score (highest coded intervention technique across all 12 possible interventions in a given session). We also measured CBT adherence during client sessions using the strategies from each study condition (described in 'Arms/Groups' below). All 3 conditions used parallel 7-point scales to the TPOCS-RS. Higher scores for the 3 study condition strategies indicate greater reported use, and higher scores on the TPOCS-RS indicate greater observed use of CBT interventions. Our goal was to compare the study condition methods of assessing CBT adherence to direct observation (TPOCS-RS), the gold standard. | through study completion, an average of 1 month |
Not provided
Not provided
Inclusion Criteria: Therapists
they provide mental health treatment services in community mental health agencies in the City of Philadelphia participating in this study and
Exclusion Criteria: Therapists
Inclusion Criteria: Supervisors
Exclusion Criteria: Supervisors
Inclusion Criteria: Youth and their Legal Guardians
Youth and their legal guardians will be eligible to participate in this study if the youth
Exclusion Criteria: Youth and their Legal Guardians
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Affiliation | Role |
|---|---|---|
| Rinad S Beidas, PhD | Northwestern University | Principal Investigator |
| Emily M Becker-Haimes, PhD | University of Pennsylvania | Principal Investigator |
Not provided
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 36229116 | Derived | Becker-Haimes EM, Marcus SC, Klein MR, Schoenwald SK, Fugo PB, McLeod BD, Dorsey S, Williams NJ, Mandell DS, Beidas RS. A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy. Behav Ther. 2022 Nov;53(6):1191-1204. doi: 10.1016/j.beth.2022.06.001. Epub 2022 Jun 10. | |
| 27633780 | Derived | Beidas RS, Maclean JC, Fishman J, Dorsey S, Schoenwald SK, Mandell DS, Shea JA, McLeod BD, French MT, Hogue A, Adams DR, Lieberman A, Becker-Haimes EM, Marcus SC. A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol. BMC Psychiatry. 2016 Sep 15;16(1):323. doi: 10.1186/s12888-016-1034-z. |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
We enrolled 126 therapists in the study; these participants consented and were randomized to treatment.
We assessed 304 therapists for enrollment in the study; of the therapists who were not enrolled, 86 did not meet inclusion criteria, 84 declined to participate, and 8 were excluded for other reasons.
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | Self-report | Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated. Self-report |
| FG001 | Chart Stimulated Recall | Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed cognitive behavioral therapy (CBT) strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used. Chart-Stimulated Recall |
| FG002 | Behavioral Rehearsal | Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria. Behavioral Rehearsal |
| Title | Milestones | Reasons Not Completed | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall Study |
|
|
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | Self-report | Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated. Self-report |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes |
|---|---|---|---|---|---|---|---|---|---|
| Age, Continuous | Number of participants analyzed for each group does not match overall number of baseline participants in that group because some participants did not disclose their age. |
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores | Client sessions for all 3 conditions were scored for CBT adherence via direct observation using the Therapy Process Observational Coding System-Revised Strategies (TPOCS-RS) Scale (McLeod et al., 2015) on a 7-point Likert Scale (1='not at all,' 7='extensively'). We analyzed the TPOCS-RS' Maximum CBT score (highest coded intervention technique across all 12 possible interventions in a given session). We also measured CBT adherence during client sessions using the strategies from each study condition (described in 'Arms/Groups' below). All 3 conditions used parallel 7-point scales to the TPOCS-RS. Higher scores for the 3 study condition strategies indicate greater reported use, and higher scores on the TPOCS-RS indicate greater observed use of CBT interventions. Our goal was to compare the study condition methods of assessing CBT adherence to direct observation (TPOCS-RS), the gold standard. | Participants (i.e., therapists) were assigned to one of 3 conditions: self-report, chart stimulated recall, or behavioral rehearsal. The unit of analysis for Maximum Mean condition score and Maximum Mean direct observation (TPOCS-RS) score was client session; therapists could include more than one client session for inclusion in analyses. | Posted | Mean | Standard Deviation | Maximum CBT score on a scale | through study completion, an average of 1 month | Client Sessions |
Adverse events were monitored from study initiation through study completion, an average of 6 weeks for each participant.
Not provided
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | Self-report | Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated. Self-report |
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Rinad Beidas, PhD | Northwestern University | 312-503-0546 | rinad.beidas@northwestern.edu |
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot_SAP | Yes | Yes | No | Study Protocol and Statistical Analysis Plan | May 11, 2020 | Nov 21, 2022 | Prot_SAP_000.pdf |
Not provided
| ID | Term |
|---|---|
| D057566 | Self Report |
| ID | Term |
|---|---|
| D011795 | Surveys and Questionnaires |
| D003625 | Data Collection |
| D004812 | Epidemiologic Methods |
| D008919 | Investigative Techniques |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria.
|
|
| Behavioral Rehearsal | Behavioral |
|
| Therapist unable to enroll clients |
|
| Clients declined, piloting condition, or only telehealth |
|
| BG001 | Chart Stimulated Recall | Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used. Chart-Stimulated Recall |
| BG002 | Behavioral Rehearsal | Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria. Behavioral Rehearsal |
| BG003 | Total | Total of all reporting groups |
| Mean |
| Standard Deviation |
| years |
|
| Sex: Female, Male | Number of participants analyzed for each group does not match overall number of baseline participants in that group because some participants did not disclose their sex. | Count of Participants | Participants |
|
| Ethnicity (NIH/OMB) | Count of Participants | Participants |
|
| Race (NIH/OMB) | Count of Participants | Participants |
|
| Region of Enrollment | Number | participants |
|
| Education | Number of participants analyzed for each group (for some group[s[) does not match overall number of baseline participants in that group (for some group[s]) because some participants did not disclose their level of education.](streamdown:incomplete-link) | Count of Participants | Participants |
|
| Years Experience | Number of participants analyzed for one or more group(s) does not match overall number of baseline participants in that group(s) because some participants did not disclose their number of years' experience. | Mean | Standard Deviation | years |
|
| Type of Employee | Count of Participants | Participants |
|
| Participated in city sponsored Evidence-Based Practice training initiative | Count of Participants | Participants |
|
| Number of client sessions recorded in the study | Mean | Standard Deviation | sessions |
|
| Client Sessions |
|
|
|
| 0 |
| 41 |
| 0 |
| 41 |
| 0 |
| 41 |
| EG001 | Chart Stimulated Recall | Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used. Chart-Stimulated Recall | 0 | 42 | 0 | 42 | 0 | 42 |
| EG002 | Behavioral Rehearsal | Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria. Behavioral Rehearsal | 0 | 43 | 0 | 43 | 0 | 43 |
Not provided
Not provided
| D017531 | Health Care Evaluation Mechanisms |
| D011787 | Quality of Health Care |
| D017530 | Health Care Quality, Access, and Evaluation |
| D011634 | Public Health |
| D004778 | Environment and Public Health |
| Male |
|
| Not Hispanic or Latino |
|
| Unknown or Not Reported |
|
| Asian |
|
| Native Hawaiian or Other Pacific Islander |
|
| Black or African American |
|
| White |
|
| More than one race |
|
| Unknown or Not Reported |
|
| Master's Degree |
|
| Doctorate |
|
| Fee for service |
|
| No response |
|