Evaluation of Maternal & Newborn Health Simulation Lab Centers of Excellence in Nepal

Evaluation of Maternal & Newborn Health Simulation Lab Centers of Excellence in Seven Referral Hospitals of Nepal

Postpartum hemorrhage and birth asphyxia are the leading causes of maternal and neonatal mortalities worldwide. Prevention and adequate treatment are, therefore crucial. While Nepal has made significant improvements in maternal and neonatal health, these efforts are insufficient to achieve the Sustainable Development Goal 3. Several studies have revealed critical gaps in the knowledge and clinical skills of maternal and neonatal health (MNH) providers in Nepal, likely due to limited clinical experience and practical exposure. Therefore, One Heart Worldwide is implementing the 'Maternal and Newborn Health Simulation Lab Centers of Excellence' project in 7 referral hospitals of Nepal wherein in simulation labs will be established and simulation-based training will be provided to hospital staff with a focus on essential care of labor and birth, bleeding after birth, and helping babies breathe modules. This implementation study will evaluate the project implementation and effectiveness using a REAIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. A mixed-methods approach will be used to evaluate each component of RE-AIM using a quasi-experimental pre-test and post-test design. The pre-test data collection will be of 6 months' duration. During this phase, the investigators will collect daily data on maternal and neonatal health outcomes (Post-partum hemorrhage (PPH) incidence, proportion of newborns with APGAR score<7 at 5 minutes' assessment, and rate of maternal blood transfusion after PPH), and also assess the skills assessment scores of the MNH service providers. After the completion of pre-test data collection, training will be provided to MNH service providers of the hospital in the established simulation labs. The daily MNH routine data collection will continue during project implementation phase, and till six months after the training MNH service providers of the hospitals. In addition to the assessments done in the pre-test, the investigators will also assess the reach, implementation status, challenges, utilization, and maintenance of established simulation labs. The post-test assessment of skills of MNH service providers will be conducted six months after the completion of the in-hospital trainings. Written informed consent will be obtained from the study participants. For the quantitative data, descriptive and inferential statistical methods will be used for data analysis. Qualitative data will be analyzed using thematic analysis supported by NVIVO 12 software.

Post-partum hemorrhage (PPH) and birth asphyxia are significant contributor to maternal and neonatal mortalities globally. Postpartum hemorrhage leads to around 70,000 deaths, mostly in low and middle-income countries. Despite evidence indicating that active management of the third stage of labor (AMTSL) can mitigate PPH occurrence by 40-70%, adherence to this practice remains limited according to guidelines. Similarly, birth asphyxia accounts for an estimated 900,000 deaths each year and is one of the primary causes of early neonatal mortality. Addressing these critical issue requires multiple approach including the competency of health care providers. In order to optimally utilize the knowledge and skills learnt by the students and health care service providers, it is important to create opportunities for reorientation not only with regard to knowledge but also to reinforce skills. Creating simulation based skills lab for pre-service and in-service with linkages for post training mentorship is a step in this direction. Comprehensive simulation lab with skills stations are designed with the aim of acquisition and upgradation of skills of healthcare providers to enhance their capacity in providing quality maternal and neonatal health care. A simulation skills laboratory has the advantage that allows learners opportunities for repetition and feedback and permits individualized learning. Globally, simulation based intervention studies report evidence of impact of patient level outcomes including, reduction is maternal complication such as PPH and retained placenta; as well as neonatal outcomes such as birth asphyxia, stillbirth, and newborn deaths. Simulation lab provides a safe and controlled environment for healthcare providers to practice and refine their skills particularly in managing obstetric emergencies such as post-partum hemorrhage and birth asphyxia. Therefore, this study will help to bridge the gap in healthcare provider's skills and improve patient safety in maternal and neonatal healthcare by establishing high fidelity simulation labs in seven strategically located referral hospitals. The hospital staff will also benefit from training available within the simulation labs. The current study will assess the effect of establishing high-fidelity simulation labs on both implementation and patient outcomes in the hospitals. In this study the patient-level assessments include assessment of the proportion of PPH, rate of blood transfusions after PPH, and proportion of newborns with APGAR score less than 7 at 5 minutes' assessment. Our study will use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework for the evaluation as it focuses on the issues, dimensions, and steps in program design, dissemination, and implementation process, which has been translated and used in different contexts and settings.

Research Method:

Concurrent nested mixed- method study to evaluate each component of RE-AIM using a quasi-experimental pre and post-test design.

Study sites:

The study sites will be the seven referral hospitals from five different Provinces of Nepal- Paropakar Maternity and Women's Hospital (Bagmati Province), Provincial Hospital Janakpur (Madhesh Province), Koshi Hospital (Koshi Province), Narayani Hospital (Madhesh Province), Province Hospital Surkhet (Karnali Province), BPKIHS (Koshi Province), and Pokhara Academy of Health Sciences (Gandaki Province) with annual deliveries of 24344; 8058; 8065; 7118; 5239; 3620; and 8915 respectively. Overall, these sites have a total of 65,359 deliveries in a year (Source HMIS data (2078/79))

Research design:

Quasi-Experimental pre-test post-test study design

Study Population:

Study population will consist of MNH providers for skill assessment, women who have recently delivered in study hospitals via normal vaginal delivery for assessment of maternal outcomes, and newborns delivered in the hospital for assessment of neonatal outcomes.

Number of participants and justification:

1. Skills assessment: 132 participants will be involved for skill assessment (66 before the intervention and 66 after the intervention).

2. Team performance assessment: 14 teams (5 participants in each team) = 70 participants

3. Maternal and neonatal health outcomes assessment: The annual number of deliveries in the study hospitals in 65,359 (source- HMIS data 2078/79). Since the data collection time period is 6 months for each phase (pre-test and post-test), the investigators expect to assess the maternal and neonatal health outcomes in about 32,500 recent deliveries in each phase. This sample size is sufficient to detect statistically significant differences (refer to sample size section). Assuming a 2.9% incidence of PPH cases, the investigators expect to collect data from about 940 PPH cases25. Similarly, assuming a 0.6% incidence of birth asphyxia among live births26, the investigators expect to collect data of about 200 neonates.

4. Key Informant Interviews: 28 participants

   1. Hospital Director/ medical superintendent- 7
   2. Maternal and Neonatal Health Focal person of the hospital from medical department (OBGYN/ Pediatrics)- 7
   3. Maternal and Neonatal Health Focal person of the hospital from nursing department- 7
   4. Simulation lab operator- 7

5. Focused group discussion- 14 (with 6 to 8 participants in each group)

DATA COLLECTION TECHNIQUE AND TOOLS A. REACH

Under reach domain, the investigators will gather following data:

i. Number and characteristics of MNH service providers trained as mentors/ trainers ii. Number and characteristics of MNH service providers trained by the mentors/ trainers in established simulation labs.

iii. Perceived challenges in reaching the target population (all MNH service providers of the study hospitals).

Tools: Simulation lab log book, Hospital information recording tool, training enrollment records, In-depth interview guideline, FGD guideline

Technique: Record review, In-depth interviews with key informants (hospital focal person, focal person from medical department, focal person from nursing department, simulation lab operator); Focused Group Discussion (FGD) with doctors and nurses. In each hospital, the investigators will conduct 4 in-depth interviews and 2 FGDs (with 6 to 8 participants), for a total of 28 IDIs, and 14 FGDs from 7 hospitals.

B. EFFECTIVENESS

To measure the effectiveness, following data will be collected pre and post intervention:

1. Objective structured Clinical Examination (OCSE):

   Tools: OSCE tools to measure the change in competence scores among MNH providers before the completion of intervention, and 6 months after the completion of intervention.

   Data collection for OSCE will be done by trained simulation experts who are involved in providing simulation based training in the hospital in three skills- Bleeding after Birth (BAB), Essential Care for Labor and Birth (ECLB), and Helping Babies Breathe (HBB).

2. Direct observation of team performance of MNH providers in a simulated setting:

   Tool: Team performance assessment tool to assess team performance and communication before the completion of intervention, and 6 months after the completion of intervention.

   Team performance assessment tool will be developed for conduction of normal delivery (essential newborn care), management of PPH (shock?) and neonatal resuscitation skills. The investigators will conduct a total of 14 team evaluations (2 at each hospital) for each phase of the study. Each team will comprise of 3 nurses, 1 doctor and 1 helper. Data collection for team performance assessment will be done by same trained simulation experts who involved in assessing the OSCEs.

3. Review patient charts and ward records, and record information related to health outcomes:

Tools-

1. Daily maternal and neonatal information recording form
2. Form for recording details of PPH cases occurring in the hospital during data collection time period (6 months before and 6 months after the intervention)
3. Form for recording details of newborns with APGAR <7 at 5 minutes' assessment occurring in the hospital during data collection time period (6 months before and 6 months after the intervention)
4. Socio-demographic information form (for PPH and APGAR<7 cases)

One ward focal person (labor and delivery nurse) will be responsible for collecting data during and immediately after delivery. One research assistant will be posted in each study hospital, and they will collect daily patient level data from the ward focal person. They will review charts of postpartum mothers and newborns in labor room, ICU, and Postnatal ward every day. They will collect the data on following patient level outcomes:

1. Quantitative blood loss more than 500 ml: Number of women having from more than 500ml blood loss during or within 24 hours after normal vaginal delivery will be recorded.
2. Rate of blood transfusion after birth: Women delivering in the same hospital via normal delivery with quantitative blood loss more than 500 ml within 24 hours after birth during the data collection time period will be included.
3. APGAR score <7 at 5 minutes' assessment: Charts of babies delivered via normal vaginal delivery at selected hospital, with birth weight more than or equal to 2500gms will be reviewed. The number of babies with APGAR score <7 at 5 minutes' assessment recorded form the partograph daily.

C. ADOPTION Tools: Practice logs, simulation lab register, semi-structured questionnaire

Techniques:

Review of practice logs and simulation lab register to track the frequency and types of training conducted and skills practiced in the simulation labs. The mean number of practice sessions per MNH provider will be calculated from MNH providers' recording of their practice sessions on log books.

Semi-structured questionnaire will be self-administered to MNH providers to gather data on the utilization of simulation labs. The questionnaire will be administered to all the MNH staff working in the hospital for more than 3 months.

IMPLEMENTATION Tool: lab establishment plan, lab and training records and reports, In-depth interview guidelines, FGD guideline

Techniques:

Review project/ skills lab renovation activities of each hospital to track the delivery of program activities as planned,

Conduct in-depth interviews with construction field engineer, Lab establishment focal person from OHW, and lab establishment focal person from each hospital

FGD with MNH service providers to identify perceived challenges and enablers during program implementation, and recommendations.

MAINTENANCE Semi-structured interviews of key informant to explore the measures used by the hospital for institutionalization of simulation-based methodology or initiatives from hospital management for regular use of simulation lab. Available initiatives/ measures will be directly observed/ reviewed.

Plan for data management and analysis The quantitative data will be collected using KOBO tool installed in mobiles/ tablets. Then, the data will be exported and cleaned in Excel sheets, and analyzed in STATA version 18. Descriptive and inferential statistical methods will be used for analysis of data. For the descriptive statistics data will be summarized in frequency, percentage, mean and standard deviation. Data will be presented in the form of frequency table, bar diagram and pie-chart.

For all OSCEs, individual provider will be the unit of analysis. Mean scores obtained by the participants will be computed in both marks and percentage. Then, independent t-test will be used to compare the difference in mean scores before and after the intervention.

For comparing the incidence of PPH, normal deliveries will be the unit of analysis and for comparing the incidence of APGAR score<7 at 5 minutes' assessment, live birth will be the unit of analysis. The data on these health outcomes will be presented as proportions, with 95% CI. The count data will be aggregated by month and health facility category. The observed crude incidence will be calculated on the basis of poisson distribution. Negative binominal regression analysis will be done to compare the outcomes before and after the intervention.

For comparing the proportion of blood transfusion among PPH cases, relative risk will be calculated to compare the outcome rates in two groups (before intervention and after intervention) as follows:

Relative risk = (proportion of outcome in after intervention)/ (proportion of outcome before intervention) A relative risk of 1 will indicate identical proportion in two groups, a risk of greater than 1 will indicate an increased proportion in post-intervention group, and a risk ratio of less than 1 will indicate a decreased proportion for post-intervention group. A p-value of<0.05 will be considered significant.

All the qualitative data (from IDIs and FGDs) will be audio-recorded. The data will be transcribed and translated into Nepali. Thematic analysis method will be used for analyzing qualitative data using NVIVO 12 software. The investigators will follow the steps of familiarization, identification of codes and themes, and interpretation of identified themes.

Inclusion Criteria:

• All women who have recently delivered in the study hospital via vaginal delivery
• Newborn delivered at the same hospital with gestational age ≥ 37 weeks, birth weight ≥2500 gm, and a fetal heart sound present at the time of admission
• Maternal and Newborn Health staff working in selected hospitals for at least 3 months

Exclusion Criteria:

• Women with Postpartum Hemorrhage (PPH) referred from other hospital, babies born outside of study hospital, and newborns with congenital abnormalities will be excluded.