Effectivenness and Implementation of an Intervention to Improve Primary Care Patient Safety

Evaluation of SINERGIAPS, an Intervention to Improve Patient Safety in Primary Care Centers in Spain Based on Patients' Perceptions and Experiences. Hybrid Type I Clinical Trial

Aims: To evaluate the effectiveness of SinergiAPS (a patient-centered audit and feedback intervention) in reducing avoidable hospital admission, and; to explore the factors that may affect its implementation.

Design: 24-month, parallel, open-label, multicentre, pragmatic, hybrid type 1 randomized clinical trial.

Setting, sample, and randomization: 118 primary healthcare centers from multiple regions in Spain will be randomly assigned (ratio 1:1) to two groups (control and intervention). The intervention group will receive two audits (baseline and intermediate at 12 months). The audits will consist of the administration of the PREOS-PC questionnaire (a measure of patient-reported patient safety) to a sample of around 100 patients per center. The intervention group will receive reports on the results of both audits, along with resources aimed at facilitating the design and implementation of safety improvement plans. The intervention will be deployed through the SinergiAPS web tool, developed and validated in previous projects. The control group will have access to the intervention after the end of the clinical trial (waitlist).

Outcomes: Primary outcome: rate of avoidable hospitalizations (electronic health records). Secondary outcomes: patient-reported patient safety (PREOS-PC questionnaire); patient safety culture perceived by professionals (MOSPC questionnaire); adverse events experienced by healthcare professionals (ad hoc questionnaire); the number of safety improvement actions (ad hoc questionnaire). Outcome data will be collected at baseline and at 24 months follow-up.

Implementation evaluation: Drawing on the CFIR model, we will collect and analyze qualitative (30 individual interviews, implementation logbooks) and quantitative (questionnaires for professionals from intervention centers, level of use of the SinergiAPS web tool) data to examine the implementation of the intervention in the Spanish primary healthcare centers.

GENERAL OBJECTIVE To evaluate the effectiveness and implementation of SinergiAPS, an intervention based on patient perceptions and experiences, aimed at improving patient safety in primary healthcare centers.

SPECIFIC OBJECTIVES

• To evaluate, through a hybrid type I randomized clinical trial with a 24-month follow-up, the effectiveness of the SinergiAPS intervention in improving patient safety in primary healthcare centers by reducing avoidable hospitalizations (primary outcome variable), as well as increasing the safety culture, increasing the number of patient safety improvement actions in the centers, and increasing patient-perceived safety (secondary outcome variables).
• To study the degree of usage of the SinergiAPS intervention in the primary healthcare centers allocated to the intervention group; as well as the contextual factors associated with its successful implementation.

METHODS Study design: Hybrid type 1 clinical trial, pragmatic, multicenter, open-label, with a 24-month follow-up. The primary healthcare centers will be randomly assigned to the intervention group (which will receive the SinergiAPS intervention - described below) and the control group (waitlist design: usual clinical practice during the 24-month follow-up, after which they will have access to the SinergiAPS intervention).

Description of SinergiAPS

SinergiAPS is an assessment and monitoring tool designed to support primary healthcare centers to identify potential problems and areas for improvement related to patient safety based on information provided by their own patients. This intervention consists of three main elements:

1. Patient safety audit of healthcare centers: Patient safety is evaluated from the perspective of patients using the validated Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) questionnaire. A baseline audit will be conducted at the beginning of the intervention, followed by an intermediate audit at 12 months.
2. Feedback of results to healthcare centers: Once the questionnaires are completed, SinergiAPS automatically generates a report with the audit results. This report is specific to each healthcare center and includes a comparison with other participating centers to facilitate benchmarking. Healthcare centers are encouraged to form a working group, consisting of approximately 3-6 professionals with a designated leader. This team is granted access to the result reports and other functionalities of the SinergiAPS web tool.
3. Design of action plans: The working group of each center meets to design patient safety improvement plans based on the problems identified in their center's result report. For this purpose, the SinergiAPS web tool provides resources, training materials, and recommendations on how to improve patient safety in primary healthcare. It also offers a tracking template to collect and evaluate the measures proposed by the centers to address the identified safety issues.

Eligibility criteria for healthcare centers: We will include public primary healthcare centers from any autonomous community in Spain that willingly agree to participate through informed consent. Centers that exclusively provide specific services (such as pediatrics or women's health centers), and centers that have been established for less than 12 months at the time of recruitment will be excluded.

Recruiting healthcare centers: We will recruit primary healthcare centers from various regions in Spain. Each participating node will recruit centers from their own community and, if possible, from neighboring regions. Researchers from eight autonomous communities (Balearic Islands, Catalonia, Andalusia, Madrid, Castilla y León, Aragon, Galicia, and the Canary Islands) are involved in this multicenter study. In autonomous communities where we do not have a node or researcher, we will utilize our network of key informants, including researchers from the Patient Safety group of semFYC, the research boards of semFYC, and the national network of the European General Practice Research Network. This network will facilitate the recruitment of healthcare centers for this study. The study will be presented to the relevant territorial management authorities and healthcare centers. A healthcare center will be considered recruited once informed consent is obtained from the center coordinator.

Sampling of healthcare centers: For each region, an intentional sample of healthcare centers will be selected, aiming for heterogeneity in terms of size, rurality, and inclusion of both teaching and non-teaching centers.

Collection of healthcare center characteristics: We will collect the following information: the number of visits to the center in the last 12 months, the number of registered patients at the center, the number of patients per physician, the percentage of patients over 65 years old, the percentage of female patients, whether the center is a teaching center or not, the rurality index, and the adjusted morbidity group (GMAs - percentage of patients in each of the five complexity groups). This data will be obtained from territorial management authorities or our information systems platforms.

Randomization of healthcare centers: After recruiting the centers and collecting baseline data, the centres will be randomly assigned in a 1:1 ratio to either the control group or the intervention group. This randomization process will be conducted using specialized software to generate a list of random numbers. Stratification for randomization will be based on the autonomous community (CCAA) and whether the center is a teaching center or not.

Selection of primary healthcare professionals: All professionals, including healthcare and non-healthcare professionals, who have been working at the center for at least 3 months will be invited to participate in the study. Invitations and questionnaire links (detailed in the "outcome variables") will be sent to professionals via email. The questionnaires will be self-administered electronically.

Selection of patients: Using our information systems, we will randomly select adult patients (18 years or older) who are registered at each participating healthcare center and have had at least one recent consultation (either in-person or remote) for a health-related issue within the last three months. These patients will be invited to self-complete the validated PREOS-PC Compact questionnaire over the phone or via email. In addition, patients may also be invited to complete the questionnaire face to face in the centre waiting room - in which case they will be consecutively approached by a research assistant and invited to self-complete the questionnaire using tablet computers. To collect post-intervention data, we will follow the same procedure to obtain another separate sample of patients. The following patient characteristics will be also collected: age, gender, educational level, nationality, employment status, number of visits to the healthcare center in the last 12 months, duration of registration at the healthcare center, self-perceived health status, number of medications, and presence of chronic diseases.

Primary outcome variable: The rate of avoidable hospitalizations will be assessed based on the definition provided by the Agency for Health Research and Quality. Data for this assessment will be extracted from the Minimum Basic Data Set (CMBD) using predefined CIE-9 codes. The rate of avoidable hospitalizations will be calculated specifically for conditions such as asthma, chronic obstructive pulmonary disease, congestive heart failure, angina, and chronic kidney disease. For each participating healthcare center, we will obtain the total number of registered patients and the total number of avoidable hospitalizations recorded during the previous twelve months.

Secondary outcome measures:

1. Perceptions and experiences of patients (scores from the scales of the PREOS-PC Compact questionnaire): This questionnaire will be administered to a sample of around 100 patients per healthcare center. This sample size is estimated to achieve a reliability of at least 0.7 in the mean scores of the healthcare centers for the questionnaire's scales. The following scales will be utilized

   • "Activation of healthcare centers": 4 items (Cronbach's α = 0.70) that measure the proactivity of healthcare centers towards patient safety.
   • "Experiences of safety incidents (errors)": 12 items (Cronbach's α = 0.67) that capture whether patients have experienced specific safety problems such as diagnostic errors, medication issues, etc.
   • "Experience of harm (severity)": 3 items (Cronbach's α = 0.85) that assess the level of harm suffered by patients as a result of the healthcare they received.
   • "Burden of harm": 3 items (Cronbach's α = 0.73) that evaluate the impact and consequences of the harm experienced by patients.
   • "Overall assessment of the center's safety level" (1 item). Visual analog scale.

2. Patient safety culture of healthcare professionals: We will calculate the average center-level score of the Patient Safety Culture Composite Score (ranging from 1-5 points), based on the responses of professionals to the Spanish version of the MOSPC questionnaire.

3. Adverse events experienced by healthcare professionals in the last 12 months: An ad hoc questionnaire will be adapted from a previous study to assess the adverse events experienced by professionals.

4. Number of actions aimed at improving patient safety in the center in the last 12 months: An ad hoc questionnaire will be administered to the safety representative of each center, including both intervention and control centers, to determine the number of actions taken to enhance patient safety.

Sample size calculation: The sample size calculation was performed based on the primary outcome variable of this trial, the rate of avoidable hospitalizations. Assuming an alpha risk of 0.05 and a beta risk of 0.2 in a one-sided test, a total of 118 healthcare centers (59 in the intervention group and 59 in the control group) are required to detect a difference equal to or greater than 1.5 units in the rate of avoidable hospitalizations per 1000 patients. This corresponds to a 15% reduction in the rate of avoidable hospitalizations (Cohen's d = 0.49). Based on the results of our previous study, we assume a mean rate of 11.2 and a standard deviation of 3.3. A follow-up loss rate of 0.5% has been estimated.

Statistical analysis First, we will conduct descriptive statistical analysis to characterize the participating healthcare centers. For each of the two groups (control and intervention), we will calculate frequencies (percentages) for binary or categorical variables, and medians (along with interquartile range) or mean (standard deviation) for continuous variables. The effectiveness of the intervention will be analyzed by comparing the mean score of the primary outcome variable (rate of avoidable hospitalizations) between the control and intervention groups after 24 months of follow-up. A linear regression model will be used, including the baseline rate as an adjusting variable. Stratified effectiveness analyses will be performed based on baseline audit scores (which will allow us to measure the independent effect of the intervention in centers with different levels of potential improvement according to the audit results). All analyses will be conducted on an intention-to-treat basis. An equivalent methodology will be applied for the secondary outcome variables. All outcome variables (including scores from patient and healthcare professional questionnaires) will be analyzed at the level of healthcare centers (mean score per center).

Study of the implementation of SinergiAPS The second objective of this project is to assess the degree of success in the implementation of the SinergiAPS intervention and identify contextual factors associated with greater implementation success. This will provide the necessary evidence to design a future large-scale implementation strategy for this intervention. To achieve this, we will evaluate each of the five constructs of the Consolidated Framework For Implementation Research (CFIR) model: the intervention itself, the inner setting, the outer setting, the individuals involved, and the processes by which the intervention implementation is achieved.

The data collection will be carried out through:

• In-depth qualitative interviews with professionals from the intervention group (approximately 30 qualitative interviews, until reaching saturation of discourse). These interviews aim to: examine the overall assessment of the SinergiAPS intervention, identify aspects for improvement, and gather suggestions to facilitate the large-scale implementation of the intervention. Sampling will be intentional, including informants of both genders, different age groups, professional categories, and territorial areas.
• Quantitative interviews: Through an online questionnaire administered quarterly to the project referent in each center, we will measure the usability of the intervention, as well as the degree of use, acceptability, and perceived usefulness.
• Analysis of the degree of usage of the web tool: We will examine the degree of usage of the tool by monitoring the number of accesses to it by healthcare centers. We will determine whether the healthcare centers have accessed the feedback report or not. We will also determine the number of accesses and usage time of the module of the tool that contains the training and support materials for the design and implementation of action plans.
• Each of the three main modules of the web tool (results report, training materials, and action plan record) will include a space (free-text box) for healthcare centers to provide their comments and suggestions for improvement.
• Analysis of the action plans proposed by the healthcare centers: The web tool includes a module aimed at systematically recording the action plans that healthcare centers decide to carry out. This information remains registered in the tool and provides valuable insights for examining the implementation of the intervention.
• Implementation diary: In each node, the researchers will maintain an active record of the barriers encountered during the implementation process of the intervention in the healthcare centers.

In this study, we will apply specific criteria for 1) primary health care centers (unit of randomization and analysis), and 2) primary care professionals, as detailed below:

Inclusion Criteria for primary care CENTRES: We will include public primary healthcare centers from any autonomous community in Spain that willingly agree to participate through informed consent signed by the manager of the center.

Exclusion Criteria for primary care CENTRES:

• Centers that exclusively provide specific services (such as pediatrics or women's health centers)
• Centers that have been established for less than 12 months at the time of recruitment

Inclusion Criteria for primary care PROFESSIONALS: All professionals in primary care health centers meeting the criteria above described, including healthcare and non-healthcare professionals, who have been working at the center for at least 3 months will be invited to participate in the study.

Exclusion Criteria for primary care PROFESSIONALS:

• professionals declining to complete the baseline questionnaires.
• professionals having been worked in the center for less than three months at the moment of recruitment