Routine Outcome Monitoring and Feedback Informed Therapy in Italy: 1st Italian RCT Large Population Study (PRIN - Italian Minister of Universities and Research Project)
Routine Outcome Monitoring and Feedback Informed Therapy in Italy: 1st Italian RCT Large Population Study (PRIN - Italian Minister of Universities and Research Project)
The research project OutProFeed - Italy: Routine outcome monitoring and feedback informed therapy in Italy: 1st Italian RCT large population study, represents the first Italian randomised controlled trial (RCT) to evaluate the effectiveness of feedback-informed psychotherapy (FIT) to improve patients mental health outcomes and psychotherapy processes through the use of the digital platform Mindy.
The following project was awarded PRIN 2022 funding by the Ministry of University and Research (MUR). This project has the University of Bergamo as lead partner, with the auxiliary participation of the University of Palermo and the Polytechnic University of Milan.
Given the innovative-experimental nature of the following research project, a digital platform -Mindy- will be used, which allows the recruited professionals (once registered) an all-round management of all the professional activities inherent to the psychologist/psychotherapist profession (informed consent, online therapy, administration of psychometric instruments, etc.) and which also facilitates the administration of psychometric instruments. ) and which also facilitates data collection, subsequent analysis and Routine Outcome Monitoring (ROM) and Feedback Informed Therapy (FIT) procedures with ad hoc charts and digital tools, in accordance with the guidelines dictated by the GDPR regulations on health and clinical data. Each therapist involved in the project, after having received guidance on the procedures and after having been instructed to use the platform will involve 5 new patients in the following project. The only inclusion criteria for patients are the following:
I) Being of legal age II) Not having a diagnosis of psychosis and/or neurocognitive disorders III) Consent to participate in the following research project IV) Being a new patient (new or less than three sessions carried out) We will randomly assign the previously recruited psychotherapists and their patients to one of three conditions: (1) treatment as usual (TAU) in which therapists and patients proceed with psychotherapy as they normally would without receiving any initial training and using only Mindy as a simplified digital folder (2) process and outcome monitoring (OPM) in which patients will complete process and outcome measures related to each psychotherapy session therapists will receive basic feedback without Clinical Support Tools on these measures, they will use the Mindy platform as a medical record and will not receive specific training on FIT (3) process and outcome monitoring with feedback (OPM-F) where patients will complete process and outcome measures related to each session and therapists will receive feedback on the patient progress and experiences of the therapeutic alliance with specific training on feedback informed therapy (FIT) and will use the Mindy platform with all feedback and Clinical Support Tools information, They will also receive monthly supervision/coaching (for three months) to complete the FIT and ROM training on-going, with practical discussion of the most difficult clinical cases and possible solutions to be implemented in therapy.
Recruitment of therapists will take place with the strategic participation of a pool of MIUR-recognised Schools of Specialisation in Psychotherapy (https://www.miur.it/ElencoSSPWeb/).
Professionals qualified to practise as psychologists who are in their third year of the Schools of Psychotherapy will be selected, as well as therapists who already have a regular diploma of specialisation in psychotherapy recognised by the MIUR (https://www.miur.it/ElencoSSPWeb/), and who are in active practice. There will be no restrictions on the type of psychotherapy used by the professionals who will be recruited (the only criterion for inclusion is that it must be psychotherapy for a mental health issue). Therapists will not be aware of the randomisation procedure to reduce bias (Zelen design). Specifically, for the following research project we will use data from a maximum of 15 sessions (endpoints) carried out by all therapists of the 3 groups with the Clinical Support System only for the therapists of the group (OPM-F) who, as previously mentioned, will receive ad hoc training; furthermore, various instruments (described in the special section Instruments) will be compiled through the functional use of Mindy and in detail (cadence, purpose and research hypotheses) in the complete protocol in the next section. Finally, through the administration of special questionnaires, factors indicative of therapists aptitude and competences for the use of digital tools in professional practice will be collected in order to obtain statistical clusters that can be used as independent or control variables.
- Backgrounds:
Routine outcome monitoring (ROM), including feedback, has its roots in the practice-based evidence paradigm, particularly patient-centered research. The administration of outcome measures, either continuously or at regular intervals as a means of checking progress and informing treatment decision making, can produce an ameliorative effect in addition to the effects produced by standard psychological therapies. ROM has been referred to by various terms, but they all contain features that can be grouped into three sequential steps: (i) collecting patient data in a regular and structured manner; (ii) providing the data to the therapist and, on many occasions, to the patient as well; and (iii) when appropriate, adjusting the process or focus of therapy in light of the feedback that emerged. These three steps have been presented as a transtheoretical model of measurement-based care- Collect, Share, Act -while a definition of ROM that captures these three steps describes it as the implementation of standardized measures, usually session by session, to guide clinical decision making, monitor treatment progress, and indicate when functional modification of treatment is needed. ROM has been presented as a relatively simple evidence-based practice that the clinician can add to any type of psychotherapy without requiring changes in that psychotherapy. The use of ROM and feedback in routine care has long been recommended. A recent advisory committee advocated for professional practice guidelines that focus on measurement-based care and feedback. These guidelines include a statement that therapists should engage in regular evaluation of the treatment process and outcomes, incorporating this information into ongoing collaboration with patients. International scientific organizations are called to increasingly use Feedback Informed Therapy (FIT) and the use of ROM, to assess outcomes, inform goals, objectives,and to monitor individual progress and guide decisions about individual care, treatment or service plans. The use of ROM is also supported by the Roadmap for Mental Health Research in Europe. In addition, national policymakers and regulatory bodies in some countries have made measurement of treatment outcomes mandatory.
ROM has been widely adopted in various therapeutic modalities for adults, for example, in psychodynamic therapy, couples therapy, and group therapy, as well as for specific problems such as substance abuse and in youth settings. It is a pan-theoretical approach and, as a therapeutic method, combines elements of supervision, ongoing assessment, and overall quality assurance.
In addition to considerations of the effects and benefits of ROM, there is also a relevant component of substantial change in the practice of professional activity in the project due to the introduction of digital technology. While there is consensus in the scientific community that digital can be of benefit to professionals, similarly, evidence indicates that such benefits occur under various conditions, including the effective integration of these technologies into daily practice. In this scenario, the possession and development of appropriate skills in the use of digital tools assumes a central role.
Equally central to the implementation of digitally supported ROM is the usability of digital technology, which will be tested through special experience and usability questionnaires on both the patient and therapist sides. Usability will also be assessed through targeted interviews with a subsample of psychotherapists, following the track of validated technology usability questionnaires and analyzed through thematic analysis.
For the reasons expressed so far, the following research project "OutProFeed - Italy: Routine outcome monitoring and feedback informed therapy in Italy: 1st Italian RCT large population study" is of fundamental scientific and clinical interest. It not only represents the first Italian randomized controlled trial (RCT) to evaluate the effectiveness of feedback-informed psychotherapy (FIT) in improving patients' mental health outcomes and psychotherapy processes, but also obtained PRIN 2022 funding from the Ministry of University and Research.
-Aims and Objectives: The main objective of this study is to conduct the first Italian randomized controlled trial to evaluate the effectiveness of feedback-informed psychotherapy in improving patient mental health outcomes and psychotherapy processes. To achieve this goal, after administering some batteries of questionnaires at baseline (T0) to therapists and patients, previously recruited psychotherapists and their patients will be randomly assigned to one of three conditions: (i) treatment as usual (TAU) in which therapists and patients proceed with psychotherapy as they normally would, (ii) process and outcome monitoring (OPM) in which patients will complete process and outcome measures related to each psychotherapy session but therapists will receive "basic" feedback without Clinical Support Tools on these measures, they will use the Mindy platform as a medical record and will not receive specific training on FIT (iii) process and outcome monitoring with feedback (OPM-F) in which patients will complete process and outcome measures related to each session and therapists will receive feedback on the patient's progress and the patient's experiences of the therapeutic alliance. Therapists in condition (iii) will receive specific training on the procedures and techniques inherent in feedback informed therapy (FIT), during the intervention phase (duration 15 sessions) the Clinical Support System in FIT will guide them in the clinical application of the procedures. In addition, they will receive monthly supervision (for three months) apt to provide on-going clinical support on FIT and ROM, with practical discussion of the most difficult clinical cases and possible solutions to be implemented in therapy. Therapists will be blinded to the randomization procedure to reduce bias and prejudice. To assess outcomes, psychological distress will be measured before each session and symptom severity at the beginning, at the end of treatment (set at session number 15, up to 6 months) and three months later (follow-up). To evaluate the process in psychotherapy, the level of therapeutic alliance will be assessed after each session. To account for the impact of individual propensity and skills toward digital tools, a clustering of therapists performed on data collected at T0 aimed at creating 'distinct profiles' and non-overlapping respondents with respect to the measured variables will be considered. The therapists involved in the study (in all conditions) after agreeing to the conditions of the study, will be introduced to the use of the digital platform Mindy (they will receive a user manual) and its functionalities (a user manual with technical specifications will be sent to participants and ad hoc training will be organized on its use, declined according to the experimental conditions: TAU (use of the medical record in BASIC mode, without feedback and without routine outcome monitoring, thus only to facilitate standard administration of questionnaires through MINDY and encourage "simple" use of the medical record with the possibility of sitting also online); OPM (use of full MINDY but without the possibility of Clinical Support Tools on Feedback); OPM-F (full use of MINDY with attached Clinical Support Tools on feedback and supervision), as reflected in the research design.
In addition, therapists who are part of the OPM-F condition will receive, after the randomization phase, specific training on feedback informed therapy (FIT) and will use the platform with all the information pertaining to feedback, plus they will receive monthly supervision with the trainer specializing in FIT. The other two conditions will be placed on the "waiting list" and will receive training/supervision later after the experimental phase. The Clinical Support Tools will offer digital alerts that will indicate in the dashboard those conditions defined as "Not on track", i.e., patients who are not responding well to therapy or who are deteriorating in some aspect (outcomes or therapeutic alliance). Scores on the CORE-10 questionnaire that significantly worsen by at least 6 (Reliable Change Index) will be flagged, and a clinical support system will be provided to help the therapist manage the clinical situation with purpose-built charts and questions. Working Alliance Inventory-C scores will be flagged with an alert with the following rationale: (i) First five sessions, the method described will be used, in which the patient's averages (session-by-session) at WAI are used and the possibility of the risk of breakdown is declined with level indications; if the difference between the averages (session-by-session) is between (-0. 25 and +n) there will be a stable WAI, if the difference is between (-0.26 and -1) there will be a moderate rupture risk, while with a drop of one point or more between the two averages (for one or more consecutive sessions) there will be a severe rupture risk alert. (ii) Instead, in the following sessions (sixth through fifteenth), the method described will be used, which involves the use of an idiographic approach, based on the individual patient's "moving average," i.e., an average updates after each assessment made and which also takes into account the previous five assessments. In this case, the risk of breakdown is represented by changes in the total score at the individual patient's WAI with the rationale of using standard deviations from the mean, as a functional indicator: If the score is lower by at least 1.5 (d.s) there will be a moderate alert, 2 d.s will be severe instead. A return within 2-3 sessions to pre-risk levels of rupture (moderate or severe), i.e., increase in scores with raising the mean by at least +1.5/+2 d.s will indicate resolution of rupture.
Management and administration of the tools will be available in the platform in all three conditions, with the differences between the three conditions being detailed. Therapy sessions will be either virtual, on a virtual platform managed and integrated by the platform or live (as needed by the therapist). The decision between virtual vs. in-person session will be investigated through interviews with therapists in the follow-up phase in order to gather drivers leading to the choice. In both cases, the platform will be used for session and patient management and for administering the instruments in a computerized manner.
Primary Hypotheses.
Patients whose therapists were in the OPM-F condition will have better outcomes in terms of reduction in symptom severity from before to six months after treatment than patients of therapists in the OPM condition and the TAU condition.
Patients whose therapists were in the OPM-F condition will have better results in terms of decreasing discomfort from session to session than patients of therapists in the OPM condition and the TAU condition.
Patients of therapists in the OPM condition will have better results in terms of reduction of symptom severity from before to three months after treatment than patients and therapists in the TAU condition.
Patients whose therapists were in the OPM condition will have better results regarding a decrease in discomfort from session to session than therapists in the TAU condition.
Patients of therapists in the OPM-F condition will report greater increases in therapeutic alliance over the course of therapy sessions than patients in the OPM and TAU conditions.
Patients of therapists in the OPM condition will show a greater increase in therapeutic alliance during therapy sessions than patients in the TAU condition.
-Participants and instruments:
Psychotherapy residents and/or Psychotherapists:
180 to 350 will be recruited.
Criteria for inclusion of therapists:
Therapists with a regular MIUR-postgraduate degree in psychotherapy(https://www.miur.it/ElencoSSPWeb/), in active practice; and/or psychotherapists in at least their third year will be selected.
There will be no restrictions on the type of psychotherapy used by the professionals being recruited (the only criterion for inclusion is that it must be psychotherapy for a mental health issue).
Measures:
Demographic questionnaires constructed for therapists and patients (i.e. age, gender, marital status, educational qualification, contact details that are collected in MINDY platform after the digital registration procedure of both therapist and related patients.
These data are recorded and stored upon agreement between the professionals and patients with the MINDY platform and the University of Bergamo.
Questionnaire battery for Therapists*:
Attachment:
Italian version of the Experiences in Close Relationships Scale (ECR12). The Italian version of the ECR-12 is a self-report measure of attachment to partners. The ECR-12 measures two dimensions of attachment to partners, avoidant attachment (6 items) and anxious attachment (6 items). An example item is "I am afraid of being left" (1= Strongly Disagree; 7 = Strongly Agree).
Mentalization:
Italian version of The Reflective Functioning Questionnaire-8 (RFQ-8) is a brief self-report measure of reflective functioning (i.e., the ability to understand the mental states of self and others) that is supposed to capture individual differences in hypo- and hyper-mentalization. The RFQ-8 includes eight items that form the two subscales "certainty about mental states" (RFQ_C) and "uncertainty about mental states" (RFQ_U). Rated on a 7-point Likert scale ranging from 1 (totally disagree) to 1 (totally disagree).
The Italian version of the Multitheoretical List of Therapeutic Interventions-Therapists Version (MULTI30) is a brief, valid and reliable instrument that is used to assess patients' and therapists' perceptions of the use of interventions and techniques of major therapeutic approaches. Each item is rated on a 5-point Likert scale.
Epistemic Trust:
The Italian version of the Epistemic Trust, Mistrust, and Credulity Questionnaire (ETMCQ) is a self-report questionnaire consisting of 15 items that assesses epistemic trust, mistrust, and credulity toward communication or communicated knowledge ; Epistemic trust refers to an adaptive attitude in relatively benign social circumstances in which the individual is selectively and appropriately open to social learning opportunities in the context of relationships. Epistemic distrust reflects the tendency to treat any source of information as unreliable or malicious, seeking to avoid being influenced by the communication of others. Epistemic gullibility refers to a marked lack of vigilance and discrimination, signaling a general lack of clarity about one's position and resulting in vulnerability to misinformation and potential risk of exploitation. Higher scores indicate greater presence of the relative trait for each factor.
Therapeutic Alliance:
The Working Alliance Inventory-Short Revised- Therapist Version (WAI-SR-T) The Working Alliance Inventory-SR-T (WAI-SR-T) is the 12-item short version of the therapist modules of the WAI. It measures the therapist's emotional bonding and level of agreement with therapy tasks and goals. Each item is rated on a 5-point Likert scale, from 1 never to 5 always. The reliability and validity of the WAI-SR-T have been repeatedly supported in a wide range of studies. The Italian version of the WAI-SR-T was used in the present study.
Battery of questionnaires for Therapists related to the use of digital**:
(with distinction between questionnaires to be delivered at t0 (*), t1 (**), t2 (***))
Intensity of use of digital technologies in psychotherapeutic practice*
The "Intensity of Digital Technology Use in Psychotherapeutic Practice" questionnaire surveys how frequently psychotherapists use digital technologies in their practice. This survey includes two key metrics: first, the frequency of online counseling sessions, measured on a 5-point Likert scale ranging from "No sessions are conducted online" to "Almost all sessions were conducted online [rather than in-person]." Secondly, the intensity of use of key digital technologies in professional practice (Digital Psychotherapy Clinical Record, Platforms dedicated to psychotherapist-patient communication, Telemedicine/Telepsychotherapy Platforms, Diary/Appointment Management Platforms, E-mail for patient communication, WhatsApp or similar Apps for patient communication, Social media (e.g. dissemination of content related to his profession or promotion of his professional activity), Elearning platforms for continuing education/updating, Software for administrative/accounting management) using a separate 5-point Likert scale ranging from "Never" to "Daily."
Prescription or actual recommendation of mental health apps*. The questionnaire captures whether in the past year the therapist has recommended mental health apps of various types (Apps for tracking symptoms or psychological states (diary for mood, anxiety states, etc.), Apps for keeping track of a patient's diary, Apps for supporting therapeutic adherence, Apps for relaxation and mindfulness, Apps for supporting addiction management, Serious Games, Apps for collecting PREMs/PROMs, Digital Therapies (clinically validated digital treatments).
Digital-related technical competencies in professional practice*. The questionnaire on "Digital-related technical competencies in professional practice" is derived from the work of Jarva, specifically at "Factor 3: Information and communication technology (ICT) competence." It uses a 4-point Likert scale to measure respondents' level of agreement with various statements related to their competence in using digital tools and Information and Communication Technology (ICT) in their practice.
Ethical competencies related to digital in professional practice* The questionnaire on "Ethical competence related to digital in professional practice" is derived from Jarva, specifically from their "Factor 5: Ethical competence related to digital solutions" section. Like the previous questionnaire, a 4-point Likert scale is used to assess the level of agreement among respondents. The objective is to assess competence related to ethical issues arising from the use of digital technologies in psychotherapeutic practice.
Propensity to integrate digital technologies into one's clinical practice* The questionnaire on "Propensity to Integrate Digital Technologies into One's Clinical Practice" investigates how willing and inclined psychotherapists are to incorporate digital technologies into their practice. This is also adapted from Jarva, with reference to " Factor 2: Digital solutions as part of work." Again, a 4-point Likert scale is used.
Net Promoter Score (NPS)**, *** The Net Promoter Score, measures the likelihood that therapists would recommend a specific digital platform to their colleagues. It uses a scale from 1 (low probability of recommendation) to 10 (high probability), thus assessing users' satisfaction with and loyalty to the platform. It is a useful index for estimating the degree of "reception" and impact of such tools in the professional context.
Post-Study System Usability Questionnaire (PSSUQ)**, *** The Post-Study System Usability Questionnaire, developed by Lewis, provides detailed feedback on the user experience with MINDY after use. Using a Likert scale of 1 to 7, therapists rate statements on overall satisfaction, ease of use, efficiency, and clarity of information. The PSSUQ is useful in identifying areas for improvement, ensuring that the system is functional and user-friendly.
Battery of questionnaires for Patients**:
(Will be administered at T0,T2 and at follow up after 3 months). Notes: T0= Baseline, T1= After the tenth session, T2= End of treatment,Follow up: Six months after the end of treatment.
The Italian version of the PhillyACES-Philadelphia Adverse Childhood Events Scale; is a 10-item measure used to measure childhood trauma. The questionnaire assesses 10 types of childhood trauma measured in the ACE study. Five are personal: physical abuse, verbal abuse, sexual abuse, physical neglect and emotional neglect. Five are related to other family members: an alcoholic parent, a mother victim of domestic violence, a family member in prison, a family member diagnosed with mental illness, and the disappearance of a parent due to divorce, death or abandonment .
The Italian version of the Experiences in Close Relationships Scale (described above); The Italian version of the Generalized Anxiety Disorder scale (GAD-7) is a 7-item self-report screening instrument for the assessment of generalized anxiety disorder (GAD). Each item is rated on a 4-point Likert scale (0 = "Not at all" to 3 = "Almost every day"), with higher scores reflecting greater severity of GAD. Specifically, scores from 0 to 4 indicate minimal anxiety, 5 to 9 indicate mild anxiety, 10 to 14 indicate moderate anxiety, and 15 to 21 indicate severe anxiety
The Italian version of the Patient Health Questionnaire; (PHQ-9) is a 9-item self-report instrument that assesses symptoms of depression. Each item is rated on a 4-point Likert scale (from 0 = "Not at all" to 3 = "Almost every day"), with higher scores indicating greater severity of depression. Specifically, scores from 0 to 4 indicate no depressive symptoms, 5 to 9 mild depression, 10 to 14 moderate depression, 15 to 19 moderately severe depression, and 20 to 27 severe depression
The Italian version of the Clinical Outcomes in Routine Evaluation (CORE-10) is a short 10-item measure for exploring psychological distress developed for monitoring outcomes in clinical settings. The CORE-10 is a shortened version of the 34-item CORE-OM. It features three domains: i) problems: depression (2 items), anxiety (2 items), physical (1 item), and trauma (1 item); ii) functioning: general functioning (1 item), social functioning (1 item), and close relationships (1 item); and iii) risk: toward self (1 item). In addition, two items (i.e., item 2, "I felt that I had someone to turn to for support when I needed it" and item 3, "I felt able to adapt in case of difficulty") are positively worded and thus are reverse rated. The items are rated on a 5-point Likert-type scale (from 0 = not at all to 4 = most or all of the time), and higher total scores (i.e., the sum of all items, ranging from 0 to 40) indicate greater distress.
The Italian version of the Working Alliance Inventory-Short Revised- Client Version (WAI-SR-C). The Working Alliance Inventory-Short Revised (WAI-SR-C) is the 12-item short version of the WAI's patient modules. It measures patients' emotional bonding and their level of agreement with the tasks and goals of the therapy of therapy. Each item is rated on a 5-point Likert scale, from 1 never to 5 always. The reliability and validity of the WAI-SR-C have been repeatedly supported in a wide range of studies.
After the 10th session (T1;Therapist version and patient version). The Italian version of the Multitheoretical List of Therapeutic Interventions (MULTI30; Solomonov et al., 2019) is a brief, valid and reliable instrument that is used to assess patients' and therapists' perceptions of the use of interventions and techniques of major therapeutic approaches. Each item is rated on a 5-point Likert scale.
Additional instruments (session-by-session) for OPM and OPM-F conditions.
Pre-session (for patients):
Notes: Will be administered before EACH session only in the OPM and OPM-F conditions .
CORE-10: Clinical Outcomes in routine evaluation.
Post -session: (for both therapists and patients):
Notes: Will be administered after each session only in the OPM and OPM-F conditions (After the second session).
WAI-Session Alliance Inventory; (Version for both therapists and patients).
Procedure The study will be conducted subject to the approval of the Institutional Review Board of the Department of Humanities and Social Sciences, University of Bergamo.
Before proceeding with the completion of the battery of questionnaires, the participant will be asked to review the informed consent form. The disclosure will outline the following:
Statistical Analysis
Data Analysis Plan:
To test the primary hypotheses, 3-level hierarchical mixed-effects models will be used with repeated measures of psychological distress, symptom severity or therapeutic alliance values at level 1 of the model, nested in patients at level 2 and therapists at level 3. Study conditions (OPM-F, OPM, TAU) will be coded with "dummy" variables and modeled at level 3. Investigators will use full maximum likelihood estimation and allow all parameters to vary (they will be treated randomly). Data useful for clustering with respect to digital propensity and skills will be analyzed using a two-step cluster analysis (hierarchical clustering and K-means). Latent variables will be identified and reduced through a Principal Component Analysis.
Missing Data:
Investigators will use post-treatment and follow-up data from all patients discontinuing treatment when possible. Data analyses will be conducted with the intent to treat sample. (intent to treat, ITT) which appears to be the gold standard for randomized clinical trials. In ITT analysis, data from all subjects initially enrolled in a clinical trial are included in statistical analyses and analyzed according to the group to which they were originally assigned, regardless of the treatment (if any) they received. This method allows the researcher to draw accurate (unbiased) conclusions about the effectiveness of an intervention. This method preserves the advantages of randomization, which cannot be assumed when using other methods of analysis. Investigators will evaluate the impact of missing data with pattern mixing models (PMM). Mixed-effects modeling with maximum likelihood estimation will estimate reliable parameters with random missing data. If necessary, investigators will control for significant missing data patterns.
Inclusion Criteria:
There will be no restrictions on the type of psychotherapy used by the professionals who will be recruited (the only criterion for inclusion is that it must be psychotherapy for a mental health issue).
Exclusion Criteria:
angelo.compare@unibg.it+39 0352052989
antonino.latona@unibg.it3884445674