The More Than Words® Parent-Delivered Program for Autistic Children and Those with Social Communication Challenges: a Single Arm Pragmatic Feasibility Trial
The More Than Words® Parent-Delivered Program for Autistic Children and Those with Social Communication Challenges: a Single Arm Pragmatic Feasibility Trial
The Hanen Centre®'s More Than Words® (MTW) program is a globally used, parent coaching intervention delivered by speech-language pathologists (SLPs) to parents of children with social communication challenges, including those who have a confirmed or suspected diagnosis of autism. The training program is designed to provide parents with the tools, strategies, and support they need to help their children reach their full communication and social interaction potential. In this study, children and parents enrolled in the MTW program as a part of their current standard of care will be recruited by their SLPs from their community sites of the publicly-funded Ontario Preschool Speech and Language Program. The primary aim is to evaluate the feasibility and acceptability of study procedures in recruitment and data collection involving parents and children receiving MTW in the community. The investigators will also examine changes in child communication, parent-child social interactions, parent confidence, and parent use of social communication strategies prior to and following participation in the program. Findings from this study will inform the optimal design of a future pragmatic randomized control trial to examine the effectiveness of the MTW program for preschool children and their caregivers.
Background
Pediatric speech-language pathologists (SLPs) provide early intervention services that focus on supporting children to engage, interact, and communicate with the people around them in meaningful contexts. For young children, their parents are often their primary communication partners. Parent coaching interventions are commonly used by SLPs to support social communication development in young autistic children. Parent coaching interventions allow clinicians to capitalize on parents' expert knowledge of their child. Additionally, these interventions increase the potential for children to generalize skills learned in sessions to their daily life, thus increasing treatment dosage without increasing the cost. Furthermore, by reducing the frequency of direct contact hours with the therapist, public systems can distribute these interventions with fewer resources and to more parents.
One parent coaching intervention commonly used by SLPs is More Than Words® - The Hanen Program® for Parents of Autistic Children or Children Who May Benefit from Social Communication Support (MTW). In fact, a recent survey identified MTW as the preschool autism intervention program most widely used (61%) by SLPs in Ontario, Canada. The two existing RCTs on MTW involved university-based program delivery and report mixed findings that included positive effects in parents' responsiveness and aspects of social communication for some children. Non-randomized observational studies of the MTW program reported improved parent responsiveness, self-efficacy, and interaction style; and improved child language and social communication skills post-program. When asked about perceived outcomes, parents from community-delivered MTW programs highlighted the new strategies they learned, their child's growth in functional communication skills, overall improvement in their relationship with their child, and the social and professional support network they gained. Studies of parents' experiences participating in MTW suggests they value the program and the opportunity to connect with other parents, but seek improvements in navigating program content, the numerous forms associated with participation, parent-to-parent discussion, and the amount of individualized time with the SLP. Although these results are promising, as is the case for parent-delivered interventions more broadly, community-based RCTs of the MTW program are lacking, and little evidence exists for the mediators and moderators of effective MTW delivery in real-world practice. Pragmatic RCTs are needed to build this understanding.
For a pragmatic RCT of the MTW program (and other parent coaching interventions) to be useful, it is critical to ensure that the methods used to evaluate treatment effects and mediators/ moderators are not only sound but also ecologically valid and feasible for use in the community practice contexts in which the study is to be conducted. Given that real-world practice can be highly variable and unpredictable in ways that are outside of the control of the researcher, large-scale RCTs can be at risk for implementation failure without careful attention at the methodological planning stages. One solution is to first conduct a pragmatic methodological feasibility trial to identify and develop mitigation plans for possible issues in recruitment, retention, data collection, and analysis. A second associated solution is to employ principles of integrated knowledge translation, that is, ensure that co-construction and collaboration between researchers and community partners begins right from the planning stages. This can ensure that outcome measures selected are ideal from both a reliability, validity, and sensitivity perspective and practical for clinicians and families to use during real-world program delivery. In sum, for a full-scale pragmatic RCT to be successful and for results to be meaningful to end users, foundational work is needed that focuses on (a) systematic co-development of a comprehensive evaluation protocol and (b) pilot testing of the acceptability and feasibility of the evaluation protocol within real-world community settings.
With an overarching objective to conduct pragmatic RCTs of the MTW program, the investigators first needed a comprehensive evaluation protocol of the MTW program as implemented in community practice that would be both feasible for clinicians and families and sensitive enough to measure treatment effects and moderators and mediators of child and family outcomes. Therefore, the investigators partnered with program developers and expert SLPs from The Hanen Centre to co-construct a logic (causal) model of the MTW program that links specific program objectives with evaluation tools (outputs, outcomes) and procedures. In brief, key areas integrated during development of the logic model and evaluation plan included: (a) theories and research guiding MTW program development, (b) outcome measures and treatment mediators and moderators identified in previous research, (c) expert SLP views on MTW objectives and outcomes, and mediators and moderators affecting treatment responses, and (d) organizational structures, barriers, and facilitators for SLPs delivering the program. The result of this co-construction process was a logic model of the MTW program and an associated evaluation protocol with proposed outcome measures.
The current study addresses the second aim, which is to assess the feasibility of the selected outcome measures and proposed evaluation procedures. Aligned with the framework proposed by Gadke and colleagues, the investigators will assess the following dimensions of feasibility: (a) recruitment capability, (b) feasibility, acceptability, and practicality of data collection and analysis procedures, and (c) preliminary effectiveness based on the selected outcome measures.
Methods The investigators will use a pragmatic, single-arm, pre-post feasibility study design employed to evaluate the co-constructed evaluation protocol, which included demographic and outcome measures with associated data collection and analysis procedures.
MTW Virtual Program MTW is a 13-week parent-implemented program designed to enhance communication, play development, and parent-child interaction for children under 4 years of age who have autism or social communication challenges. MTW is delivered by SLPs certified by The Hanen Centre, a global not-for-profit organization based in Toronto that developed the program and supports training in its delivery. Both in-person and virtual versions are available; the version used for all MTW programs delivering during this trial will be virtual.
Participants In Ontario, Canada, parents who have concerns about their young child's speech-language development can self-refer for services through the Ontario Preschool Speech and Language (PSL) program, a publicly-funded provincial program supporting over 60,000 children from birth through to school entry annually. For children with diagnosed or suspected autism in the PSL program, the most used standard of care includes offering the MTW program to families. All SLPs and parent-child dyads participating in this feasibility study will do so during routine MTW program sessions in the Ontario PSL program.
SLPs The Hanen Centre will send an invitation to participate in this study to SLPs working in the Ontario PSL program who were known to be certified to deliver the MTW program and routinely offering these sessions to families on their caseloads. SLPs who express interest will then connect with the study team to assess eligibility. A member of the research team will attend the parent orientation session of the MTW program and introduced the opportunity to participate in the current study to all parents who were eligible to enroll in the SLP's upcoming MTW program. Parents who express interest will then be connected with a member of the study team who is also a registered SLP over Zoom for further information about the study and to complete an electronic consent process for their child and themselves to participate.
Parent-Child Dyads As per standard care in the Ontario PSL and in alignment with The Hanen Centre guidelines, parents are offered the opportunity to enroll in a MTW program group session by their SLP based on the following eligibility criteria: the child (a) is diagnosed with autism, suspected of having autism, or identified as having social communication challenges, (b) is ≤ 47 months of age, (c) demonstrates delays in social communication and play skills according to the SLP's clinical assessment, and the parent (d) is available to attend the 13-week MTW program and (e) is comfortable communicating in the language in which the MTW program was being delivered. Where more than one parent of the same child is participating in the MTW program, one parent for each child will be recruited to complete the parent-report measures pre- and post-program. As per recommendations for non-randomized feasibility studies, a formal sample size calculation is not required but the investigators aim to collect complete datasets from 12 parent-child dyads based both on Julious' recommendation of 12 participants per group in pilot feasibility trials, and what the investigators assess as practically possible within the study timeframe.
Measures In alignment with the intent to conduct measurements embedded within real-world practice, data will be collected during implementation of MTW program sessions delivered by community-based SLPs to families as a part of standard care.
Recruitment Capability The investigators will track the rates of study enrolment by SLPs and parents. The investigators will also record reasons for SLP non-enrolment when these were offered. During the post-study interviews with enrolled SLPs, the investigators will gather any known reasons for parents declining participation.
Feasibility, Acceptability, and Practicality of Measures and Procedures A primary means for assessing feasibility is tracking completion rates for the study measures, that is, the number and proportion of dyads for whom data were provided to the research team. This includes (a) demographic and outcome measures completed by parent and/or SLP then uploaded to REDCap by the SLP, (b) parent-child video play sessions recorded and uploaded to a shared university OneDrive folder by the SLP, and (c) outcome and evaluation forms completed by the parent online via REDCap. The investigators will also measure the time required for the research team to code videos using two outcome measures (described below) and the ability to attain sufficient coding reliability. Finally, upon study completion, participating SLPs will be asked in an interview to share their perceptions on the barriers and enablers to recruitment and data collection, the practicality and useability of the outcome measures and procedures, and suggested design changes for future RCTs.
Preliminary Effectiveness Profile of Preschool Communication (PPC). The PPC is an outcome monitoring data collection tool designed within the World Health Organization's International Classification of Functioning, Disability and Health framework (ICF) to collect information about preschool children's communicative participation and impairments, along with clinically relevant predictors of outcome and response to intervention. Originally developed for use within the Ontario PSL program, the PPC was intentionally designed with the goal of being feasible, usable, and useful for real-world practice settings and has demonstrated high interrater reliability for categorizing children's overall communication concerns. This measure will be used to collect demographic data including age and sex and to gather information that could be used in future studies to explore possible moderators of communication development and response to intervention (e.g., environmental risk factors, personal factors such as behavior difficulties or low birth weight). The PPC is completed by the SLP based on information obtained through their initial assessment and discussion with the parent.
Communication Function Classification System (CFCS). The CFCS is a valid and reliable tool used to classify a child's everyday communication abilities into one of five levels based on how they send and receive messages. The SLP assesses the child's level of communication based on function not chronological age or developmental stage using the CFCS flow chart, which includes five levels ranging from Level I (Effective Sender and Receiver with unfamiliar and familiar partners) to V (Seldom Effective Sender and Receiver with familiar partners). As such, a lower level is indicative of a more effective communicator. Each child's CFCS Level was collected pre-program by the SLP. This data will be collected because in future RCTs, it can be used both to collect baseline demographic variables and for moderator analyses to determine whether outcomes vary by pre-program CFCS Level.
Communication and Symbolic Behavior Scales - Developmental Profile (CSBS-DP) Caregiver Questionnaire (CQ). The CSBS-DP CQ contains 41 multiple-choice items that assess seven prelinguistic skills: emotion and eye gaze, communication (rate and communicative function), gestures, sounds, words, understanding, and object use (symbolic and constructive play). Three composite scores (social, speech, symbolic) and a total raw score are generated. Although designed for children aged 6 to 24 months, it can be used with older children who are functioning within this developmental range and thus is applicable to the target population of the MTW program. The investigators will record Social, Expressive Speech and Symbolic Communication composites and Total raw scores pre-post program.
Focus on the Outcomes of Communication Under Six (FOCUS-34). The FOCUS-34 is a reliable and validated criterion-referenced measure of real-world changes in children's communicative participation in response to speech-language intervention. The FOCUS-34 includes 34 items based on the Activities and Participation components of the ICF. It includes 23 items focused on the child's capacity for communication and 11 items on the child's ability to engage independently, both of which parents rate on a 7-point Likert scale. The total score on the FOCUS-34 ranges from 34-238 points, although the primary clinical outcome is the pre-post-program change scores. A change score of 11 points or more is considered a meaningful clinical change, 7-10 is possibly a meaningful clinical change, and 6 or less is not likely a meaningful clinical change. These ecologically sound cut-offs were derived from research showing that children waitlisted for SLP services had an average change score of 5.87 and change scores of 11 points or higher were unlikely to occur due to maturation alone. Therefore, the FOCUS-34 allows clinicians and researchers to determine whether change pre- to post-intervention can be attributed to the intervention even in the absence of baseline data or a control group. The FOCUS-34 is used in the Ontario PSL as a provincial outcome monitoring tool and thus is part of the existing standard of care for participants in this study. For this study, the investigators will record the FOCUS total raw scores pre- and post-program and classify children's change scores as to whether they were clinically meaningful.
MTW Social Communication Checklist (SCC). The SCC is a Hanen Centre-developed form that is completed by SLPs in collaboration with parents. It is used to identify the stage that best describes the child's current communication skills. The four stages are coined by The Hanen Centre as: Own Agenda, Requester, Early Communicator, and Partner. The SCC is completed pre-program as part of the standard delivery of the program, but for this study, it will also administered post-program to explore its utility in measuring change. The investigators developed a social communication stage classification procedure based on SCC responses whereby the child is assigned the stage at which they had the most skills selected on the form. The investigators will evaluate whether the child stayed at the same stage or changed stages post-program. For children who stay in the same stage post-program, the investigators will also explore whether they gained new skills within that stage by counting how many skills within the stage were checked pre- vs post-program.
Clinical Global Impressions - Severity (CGI-S) and Improvement (CGI-I). For the purposes of this study, the CGI-S will be coded pre-program in response to the prompt, "How impacted is the child by social communication challenges at this time?" on a 7-point severity scale (not assessed, normal, borderline, and mildly, moderately, markedly, severely, extremely impacted). It will also be coded post-program with respect to changes in social communication challenges relative to pre-program on a 7-point scale (very much improved, much improved, minimally improved, no change, minimally worse, much worse, very much worse). Descriptions for the CGI-S and the CGI-I ratings from Toolan et al. will be used to guide the coding process. The CGI-S rating will be made using the pre-program play session video recording and the CGI-I rating was made using the play recording from the final (third) video feedback session. Coding of the CGI items will be completed by a clinician researcher and a SLP graduate student. To establish reliability, 15% of the videos will be double coded in tandem; once agreement is reached, an additional 15% of videos will be independently rated by the two coders and reliability was calculated using weighted Cohen's Kappa.
Joint Engagement Rating Inventory (JERI). The JERI is an observational measure that captures features of joint engagement between young children and their parents during an unstructured play interaction within a naturalistic setting (i.e., play in the home context). It was designed for use with children who have diagnoses of developmental delays, inclusive of autism. The 2020 version of the JERI has 32 items, but as recommended in the manual, the investigators will use a subset of social engagement behaviors aligned with the research questions, namely, 11 items aligned with skills targeted by the MTW program. This includes five child behavioural outcomes (initiation of communication, responsiveness to partner's communication, expressive language level and use, quality of behaviour patterns, affect), four parent behavioural outcomes (affect, following in on the child's focus, symbol highlighting, scaffolding) and two dyadic interaction items (fluency and connectedness, routines and rituals). Each item is scored on a 7-point rating scale (1 - feature is minimally present, 7 - feature is highly present). Coding of JERI items will be completed by a clinician researcher and a SLP graduate student using the same pre and post program parent-child play videos used for the CGI coding. The procedure for establishing and calculating reliability will be identical to that previously described for the CGI. Note that coding for the JERI will occur prior to coding of the CGI to allow masking of the timing of the videos (pre- vs post-program).
Parent Self-Efficacy/Knowledge Questionnaire. This brief 4-item survey incorporates self-efficacy questions from the Early Intervention Parenting Self-Efficacy Scale and open-ended questions assessing parent MTW strategy knowledge. The two self-efficacy questions are: (a) "I feel confident that I can help my child to communicate their thoughts and ideas" and (b) "I feel confident that I can help my child develop their play skills". Each question is rated on a 7-point Likert scale (from strongly agree to strongly disagree). The two open-ended questions ask parents to list strategies or techniques they currently use to support their child's communication and play skills, respectively. The same four questions will be presented pre- and post-program.
MTW Parent Satisfaction Survey. This brief survey was developed by The Hanen Centre as a means for the SLP to collect feedback on the program from the parent. The investigators will use several questions from this survey to evaluate perceived effectiveness according to parents. Perceived effectiveness for the child is assessed with the question "My child benefited from the More Than Words program" rated on a 7-point Likert scale (ranging from strongly agree to strongly disagree). Perceived effectiveness for the parent is rated using the question "Overall, I think the MTW program was..." using a 5-point scale ranging from extremely helpful to not helpful at all and a follow-up checklist asking parents to select the program information they found to be most helpful. Perceived effectiveness for parent-child interaction is evaluated using by asking parents if they think they have changed the way in which they interact and communicate with their child as a result of taking the program, and follow-up open-ended questions for "yes" respondents asking them to list the most important ways they feel they have changed and for "no" respondents seeking details on why they felt they hadn't changed.
MTW Session and Video feedback Evaluation Forms. Evaluation forms developed by the Hanen Centre are collected as part of standard delivery of MTW at 4 time points, namely, after the 4th group session and each of the 3 individual video feedback sessions. The forms include 4-5 open ended questions exploring parent implementation of the MTW program and perceived helpfulness of the session and providing an opportunity for inquiry and suggestions to enhance implementation. Evaluation forms can be used in RCTs to measure fidelity and quality of implementation, clarify potential causal mechanisms, and support identification of contextual factors associated with variation in outcomes. As such, the investigators seek to evaluate the feasibility of collecting this information during the current study. Although the MTW evaluation forms are typically collected via paper or email by the SLP delivering the program, web-based forms on REDCap will be used to assess the feasibility of collecting this information directly from parents.
Data Collection Procedures As indicated and above, some of the measures are collected as part of the existing standard of care either because they formed part of the MTW program materials or are the existing standard in the Ontario PSL program. The participating SLPs will administer, collect, and/or complete the PPC, CFCS, CSBS-DP, FOCUS-34, and SCC as per the typical procedures they follow during virtual service delivery (e.g., via email, over Zoom). They will then upload anonymized copies of either the measures or standard of care datasheets summarizing their scores via REDCap for analysis by the research team. The SLPs will collect the videotaped parent-child interactions using secure laptop computers through their community sites and Open Broadcaster Software as per standard of care. They will then upload copies of the recordings to a secure university OneDrive folder for later JERI and CGI coding by the research team. To facilitate ease of completion and data collection, forms that do not require completion or scoring by the SLP (e.g., parent satisfaction survey, parent self-efficacy/knowledge questionnaires) will be delivered to parents via REDCap using the survey feature. This way, parents will be able to input their responses via mobile devices or computers to share directly with the research team.
Within one month after completion of the study, a 40-minute virtual interview will be conducted with each of the participating SLPs to explore their experiences participating in the study (i.e., perceptions of the feasibility and acceptability of the recruitment and data collection procedures) and to gather feedback and suggestions for development of a subsequent RCT. Interviews will be conducted by a trained researcher using a semi-structured interview guide and recorded using Zoom for Healthcare. Field notes from the interviews will be used to extract information that could inform the design and delivery of a pilot RCT.
Data Analysis Descriptive statistics will be used to characterize participants at initial assessment (e.g., sex, diagnosed or suspected autism) and to summarize recruitment, retention, and data completion. Means and standard deviations were used for continuous variables (e.g., CSBS-Total, raw scores, FOCUS-34), and frequencies and percentages were used for categorical and scale variables (e.g., satisfaction, measure completion, JERI coding, CFCS levels).
This study is not intended or powered for detecting statistical effects. To support assessment of the preliminary effectiveness and sensitivity to change of quantitative post-post outcome measures, the investigators will report effect sizes (using Cohen's d) and their confidence intervals. Descriptive and statistical analyses will be completed using JASP version 0.19.1.0.
Content analysis will be used for SLP interview data and open-ended parent survey responses that relate to the acceptability and feasibility of the study procedures and use of strategies. This method is well suited for use with small sample sizes as it can be used to quantify and describe participants' experiences (e.g., attitudinal and behavioral responses to questions).
Inclusion Criteria A. Children
B. Parents
C. Speech-language pathologists