Impact Study of the MSProgress Quality Approach Support Tool for Healthcare Centers Multiprofessionals (MSP) of the AUvergne-Rhône-Alpes Region Using a Mixed Method Evaluating the Evolution of Indicators Chosen by MSPs and Their Use in Multi-professional Teams
Impact Study of the MSProgress Quality Approach Support Tool for Healthcare Centers Multiprofessionals (MSP) of the AUvergne-Rhône-Alpes Region Using a Mixed Method Evaluating the Evolution of Indicators Chosen by MSPs and Their Use in Multi-professional Teams
Since 2008, the Federation of Health Centers and Coordinated Practice in Auvergne Rhône-Alpes (FemasAURA&Co) has been supporting healthcare centers Multiprofessionals (MSPs) in their development in the Auvergne-Rhône-Alpes (AuRA) region. It aims to support and study the development of the quality approach in MSPs as part of the MSProgress project. The first phase of this project aimed to create a tool to support the quality approach in MSPs. In 2022, a qualitative study using semi-structured group interviews in 7 MSPs followed by a consensus method made it possible to develop a tool composed of quantitative indicators, identified by MSP professionals and queryable in the Health Insurance databases. In 2023, a pilot study was conducted to test the tool prototype with the seven MSPs that participated in its creation, as well as with five MSPs new to the project, in order to gather collective and individual opinions on the design, understanding, and use of this tool. This feedback enabled the finalization of the tool. The second phase of this project aims to evaluate the deployment of the MSProgress tool on a larger scale in MSPs in the AuRA region and to study the selection, monitoring, and analysis of three team indicators over time, with a view to improving their practices.
Our hypothesis is that the availability of this tool will encourage a process of improving practices and lead to improved results for the indicators chosen by the teams.
The main objective is to evaluate the impact of the MSProgress quality approach support tool on the evolution of the indicators chosen by the MSPs in comparison with standard care.
We will use a mixed quantitative and qualitative method.
To assess the impact of the provision of the MSProgress quality approach support tool on the evolution of the indicators selected by MSPs, we will conduct a stepped-wedge cluster randomized controlled trial in 60 MSPs in the AURA region, distributed as follows:
MSPs who already participated in the tool development phase and/or the MSProgress pilot study (n=12) will not be considered for randomization but will be followed in a parallel open-label arm.
The inclusion of MSPs in the study begins upon receipt of their MSProgress tool. One month later, they will be asked to choose three indicators from the list of 43 proposed by the MSProgress tool.
Statistical analyses will focus on the evolution of the three indicators selected by the MSPs between 2021 and 2027, with the years 2021 to 2023 allowing for monitoring the natural evolution of the results. These analyses will be performed at the patient level (resulting from an export of data queried from health insurance databases) (main analysis) and at the MSP level.
To assess the methods of choosing and using these indicators in teams and understand what the tool contributes to the quality approach in MSPs, we will conduct a qualitative study with the included MSPs selected by theoretical purposive sampling (these teams will be part of the MSProgress 2024 group of the randomized controlled trial, since the qualitative study will extend over the 3 years of the study). Data collection will be carried out during an annual semi-structured group interview for each MSP over the 3 years of the study by an observer/moderator pair from the research team following the established interview guides. Each interview will be transcribed and the data pseudonymized. The verbatim reports will be submitted for review to the team referent. The analysis will benefit from triangulation by comparing the results of two coders on the NVivo software, according to a grounded theory approach: open, axial and selective coding. Processing this data will enable us to conduct an integrative analysis and propose an explanatory model.
At the end of each interview, a questionnaire with an anonymized Likert scale will be distributed to each participant to gather their individual opinions on the format, understanding of the data and results, usage projections, and positioning on the quality approach (data triangulation).
• Number of subjects required To assess the effectiveness of the MSProgress quality approach support tool on the evolution of the indicators selected by the MSPs, the estimation of the number of subjects required (NSR) will be based on the comparison between groups, with and without the MSProgress quality approach support tool, of the percentage of indicators selected at inclusion, satisfied at 12 months. For a criterion met at 20% at baseline, an absolute difference of 10% (i.e., 20% vs. 30%) could be demonstrated with 1,850 patients per group (with and without the MSProgress quality approach support tool), according to the stepped-wedge cluster randomization design described above (with 3 randomization sequences and 3 study periods (years), for a two-sided Type I error risk of 5%, a power greater than 90%, an intraclass correlation coefficient of 0.025 (to account for intra- and inter-cluster variability), and a mean number of 100 patients affected by the indicator, per MSP (steppedwedge command, Stata software, version 15, StataCorp, College Station).
If we consider 4,500 patients per MSP based on the data provided by the French National Health Insurance, the inclusion of 20 MSPs per group (with and without the MSProgress quality approach support tool) could involve 90,000 patients. With 2,000 patients per group (with and without the MSProgress quality approach support tool) (i.e., 20 MSPs with an average number of 100 patients affected per indicator for a calculated minimum population of around 1,850), the selected indicator should cover 2 to 3% of an MSP's patient base, which seems entirely consistent and realistic in terms of feasibility.
For a criterion met at 30% at inclusion, 800 patients are required per group (with and without the MSProgress quality approach support tool) under the assumptions described above; For a criterion met at 15%, 2,000 patients are required per group (with and without the MSProgress quality approach support tool), which is compatible with the inclusion of 20 MSPs and an average of 100 patients affected by each indicator.
• Statistical Methods Statistical analysis will be performed using Stata software (version 15, StataCorp, College Station).
All statistical tests will be performed at a 5% risk of Type I error (α). As discussed by Feise in 2002, adjustment for Type I error will not be proposed systematically, but rather based on clinical and not solely statistical considerations. No correction will be proposed for secondary objectives except for comparisons between the three sequences of the stepped-wedge cluster randomization design.
The statistical individual will be the patient. The data used will be statistical and epidemiological data from the French National Health Insurance databases, which do not allow for patient identification (apart from sex and age), but simply for the identification of a number of patients affected by the indicator chosen by the MSPs.
A description of the MSPs will be provided according to the three sequences of the stepped-wedge cluster randomization design.
Continuous variables will be presented as mean and standard deviation, subject to normal distribution (Shapiro-Wilk test if necessary). In cases of non-normality, they will be presented as median, quartiles, and extreme values. Qualitative variables will be expressed as numbers and associated percentages.
Graphical representations will be included with these analyses whenever possible.
The initial comparability of the groups with and without the MSProgress quality approach support tool will be assessed based on the participants' main characteristics (age and gender).
A description of protocol deviations, patients distributed according to these deviations, and the reasons for discontinuation will also be provided. The number of patients included and the inclusion curve will be presented by groups with and without the MSProgress quality assurance tool.
The primary analysis will be conducted on an intention-to-treat basis as a first-line method.
In a second phase, MSPs who discontinue their participation during the study may be removed as part of a per-protocol analysis. As the data used will be statistical and epidemiological data from the French National Health Insurance databases, which do not allow patient identification, it will not be possible to consider any other per-protocol analysis.
To assess the impact of the MSProgress quality approach support tool on the change in the indicators chosen by MSPs compared to standard care (primary objective), the comparison between groups, with and without the MSProgress quality approach support tool, regarding the primary outcome measure will be carried out using a mixed model to account for intra-cluster variability as well as period and duration effects and their interactions.
Indeed, in addition to the precautions required for all types of cluster-randomized studies, particularly the consideration of intra-cluster correlation, a significant bias specific to the stepped-wedge design must be sought, studied, and taken into account. In this type of study, there are more clusters (MSPs) exposed at the end of the study than at the beginning; the effect of the intervention can therefore be confused with any other underlying time trend (such as an overall improvement in the quality of care or an improvement in the quality of life of the patients included in the study). This effect is likely to reduce the precision of the study. It must be discussed and the results statistically adjusted for this "time effect".
Given the statistical nature of the variable studied (percentage of indicators chosen at inclusion, satisfied at 12 months), the statistical model will be of the generalized linear type (logit function for binary dependent variable). These regression models consist of an extension of regression models applied to hierarchically structured data by the introduction of a so-called random effect (MSP cluster effect); in order to model the differences, not only between micro-units (patients), but also between macro-units (MSPs).
The results will be expressed in terms of odds ratio and 95% confidence interval. The intra-class correlation coefficients will be presented with a 95% confidence interval for the entire population and by group, with and without the MSProgress quality approach support tool.
The secondary analyses, which will aim to (1) compare the impact of the MSProgress quality support tool on the evolution of the indicators chosen by MSPs according to the number of years of use of the tool, (2) evaluate the impact of the MSProgress quality support tool on the evolution of all the indicators made available to MSPs, and compare the impact of the MSProgress quality support tool on the evolution of all the indicators between the MSPs that participated in the pilot study and the MSPs included in the randomized controlled trial, will be based on the models described for the main analysis.
For longitudinal analyses, the time effect (period) will be considered as a fixed effect; the group (with and without the MSProgress quality support tool) x time interaction will be studied. The statistical analyses will focus on the evolution of the three indicators selected by the MSPs between 2021 and 2027, with the years 2021 to 2023 allowing for the natural evolution of the results.
To study the "having participated in the pilot study yes/no," this variable will be considered as a covariate in the models described above.
The results will be presented as described above, in terms of odds ratios and 95% confidence intervals.
MPSs who have already participated in the tool development phase and/or the MSProgress pilot study will not be considered for randomization but will be followed in a parallel open-label arm. The comparative analyses described above for the MSPs included in the cluster-randomized study will then be replicated considering these MSPs who have already participated in the tool development phase and/or the MSProgress pilot study.
Any subsequent changes to the statistical analysis plan must be justified and will result in a new version of the document. These deviations from the analysis plan will be reported in the final study report. All documents will be kept in the study file.
Inclusion Criteria:
After randomization of the 60 MSPs for the cluster randomized controlled trial, among the first established group named "MSProgress 2024," volunteer MSPs will be able to participate in the qualitative study. To do so, professionals agree to participate in three semi-structured group interviews (one per year over the three years of the study), by signing an individual consent form. We will ensure that the MSPs included are representative of the diversity of MSP profiles:
Exclusion Criteria: