Evaluation of an mHealth Intervention Using an Automated Text Message (SMS) Invitation System to Increase Participation in Mammography Screening Among Women Aged 50-69 Years in the Province of Santa Fe, Argentina
Evaluation of an mHealth Intervention Using an Automated Text Message (SMS) Invitation System to Increase Participation in Mammography Screening Among Women Aged 50-69 Years in the Province of Santa Fe, Argentina
We carried out with a mixed-method study including a pragmatic randomized controlled trial to evaluate effectiveness of the intervention and quantitative and qualitative evaluation of the implementation from women and stakeholders' perspective. In this report, we present results of the pragmatic randomized control trial and implementation evaluation from women´s perspective guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, specifically developed to assess interventions beyond efficacy across multiple public health criteria. RE-AIM framework was integrated in all stages of the research process, including conceptualization (e.g., selecting implementation processes that would be evaluated), data collection and analysis. Following Proctor´s Taxonomy of Implementation Outcomes [34], we also measured acceptability and appropriateness, which are essential to understand the success (or failure) of the implementation of an intervention from users' perspective. This approach is the same as we used to evaluate implementation in our previous study (The ATICA study).
Breast cancer is the most common cause of death from cancer in women worldwide. In the Latin America and Caribbean region, each year 210,000 new cases are diagnosed, and 58,000 women die due this cancer (1). Access to early detection and appropriate treatment is needed for reducing the burden of this disease. According to the World Health Organization guidelines (2), in countries with well-established health systems, organized screening program with one mammography every two years has been proven effective to reduce mortality in women aged 50-74. Argentina is one of the Latin American countries with highest incidence and mortality rates in the region with 22,000 new cases and 6,800 deaths annually.(1) Breast cancer screening is opportunistic and despite several efforts to promote mammography, screening coverage remains low, around 40% in women with public health insurance (3). Therefore, the development of innovative and low-cost strategies for increasing access to breast cancer screening are needed.
The aim of the study was to evaluate effectiveness and implementation of an mHealth intervention based on SMS messages (the ATICA strategy) to promote participation in breast cancer screening among women aged 50 to 69 in Santa Fe, Argentina.
We carried out with a mixed-method study including a pragmatic randomized controlled trial to evaluate effectiveness of an mHealth intervention and quantitative and qualitative evaluation of the implementation guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework (4), specifically developed to assess interventions beyond efficacy across multiple public health criteria. RE-AIM framework was integrated in all stages of the research process, including conceptualization (e.g., selecting implementation processes that would be evaluated), data collection and analysis. Following Proctor´s Taxonomy of Implementation Outcomes (5), we also measured acceptability and appropriateness, which are essential to understand the success (or failure) of the implementation of an intervention from users' perspective. This approach is the same as we used to evaluate implementation in our previous study (The ATICA study (6-8)
Setting The study was conducted in the city of Santa Fe, the capital city of the province of Santa Fe, Argentina. The province of Santa Fe is located in the central region of Argentina. It is the third most populated province in the country, with 3,556,000 inhabitants of which 30% are women aged 50 and over and among them around 20% had public health insurance (9). Around 90% of the households had a cell phone (10).
Sample size The recruitment was conducted across ten selected healthcare centers (five in the north and 5 in the south of Santa Fe region) over a span of six months. Through this strategy, a total of 248 eligible women were successfully recruited between December 4, 2021, and July 7, 2022.
Randomization and masking
Women were randomly allocated either to the intervention or to the control group (1:1 ratio) using a computer-generated random number list generated by the study statistician. Blinding allocation was guaranteed, because neither the recruiters nor the field coordination knew the group to which women would be assigned. The local field coordinator sent the list of recruited women to the study coordinator on a weekly basis.
Intervention development
The intervention was designed with participation of women and Santa Fe health authorities in consultation with staff from the Agency for Cancer Control of Santa Fe.
We carried out formative research to design the SMS content. We also adapted the automated messaging system (MATYS, for its initials in Spanish) which had been designed for the first ATICA study. MATYS was designed to register data on delivery and reception of SMS messages; a phone number was considered valid if MATYS did not kick back an error notification signalling the number was non-existent.
Women who were assigned to the intervention group (IG) received up to four SMS messages. The intervention included two series of SMS messages. First, women of IG received one weekly SMS messages for 2 weeks. Those women that did not register a mammography in SITAM 45 days after the 2nd message, received the second series, consisting of one weekly SMS message over two weeks. The intervention assumed an average of 30 days to obtain appointments and perform the mammography, plus 15 days until the report was uploaded in SITAM and available for delivery. Messages stopped if a mammography was registered in SITAM.
The women in the control group received standard care, which in this context is opportunistic screening. Women in the control group had the possibility of asking for an appointment for mammography, if they wanted to do so. To ask for the appointment, women must complete different steps: 1) women have to ask for an appointment with a health professional in a primary health center; 2) a health professional has to prescribe a mammography, 3) then women ask for the appointment at the administrative office in the health center.
Procedures Administrative staff of ten health care centers identified eligible women who attended the health center for any reason and invited them to participate. They checked the eligibility criteria and invited them to participate in the study. Once women consented, the recruiter described the objectives and procedures of the study. They informed woman that she will be randomly assigned to intervention group (she will receive up to four SMS messages inviting her to ask for an appointment via WhatsApp) or control group (usual care, described above).
Training
The health system staff that participated in the study (recruiters) received one day training session carried out in November 2021. This training included presentations about study design, methods, and ethical considerations. In addition, they were trained about how to: recruit women, obtain informed consent, and register data in the provincial health information system. These sessions were delivered by ATICA researchers and staff of Agencia del Control del Cancer de Santa Fe.
Data collection
Recruitment took place between December 2021 and July 7, 2022. Recruiters registered in the provincial information system (SICAP, for its initials in Spanish) the following information: age, telephone number, health insurance, name and region of health center, date of recruitment. Information about previous breast cancer screening and number of previous contacts with the health system were extracted from the provincial information system. Data on breast cancer screening were extracted from SITAM.
Implementation evaluation
Data collection
After the implementation phase we carried out a quantitative evaluation of the implementation. We evaluated women´s perspectives about implementation through a telephone survey among women who had participated in the IG. The list of women of the IG and their contact details were extracted from the RCT database.24 All women from the intervention group were contacted by trained interviewers for a phone interview.24 Interviews took place between September and November 2022. We interviewed 69 (56%) women. The questionnaire included open-ended questions with dimensions related to women's perceptions about acceptability and appropriateness of SMS message content.
Implementation outcomes
We evaluated the implementation of the intervention using selected dimensions of the RE-AIM framework.
Reach is defined as the proportion of individuals who receive or are affected by a policy or programme. In our study, reach was measured as the proportion of total women who registered a mammography during the follow up period (July 2023, 12 months after the last recruited women).
Effectiveness of the intervention was evaluated in the RCT (above described). The effectiveness of the ATICA strategy was defined as the percentage of women with mammography 45/105 days after their inclusion in the study (date of recruitment) in the intervention group vs. control group.
Implementation is defined as to the extent to which an intervention is delivered as intended. In our study, we evaluated implementation of intervention activities through the following indicators: 1) percentage of health care centers that enrolled at least one eligible woman in the study; 2) percentage of SMS messages that reached women´s valid phone number; 3) percentage of women in the intervention group who asked for an appointment and 4) percentage of surveyed women who mentioned that they did not receive or did not remember receiving the SMS message.
In addition, following Proctor´s Taxonomy of Implementation Outcomes, we measured acceptability and appropriateness.
Data analysis We conducted a descriptive analysis using frequencies and percentages for each variable included in the RE-AIM dimensions. Differences in percentage of participant women with mammography at 45/105 days between intervention and control group were examined using Chi-squared tests. Significance was assumed at a two-sided value of p < 0.05. Odds ratios and 95% confidence intervals (CIs) were also calculated, with mammography registration in SITAM as the outcome variable and assigned group as the exposure.
Effect modification by baseline factors (age, health insurance, health center region previous breast cancer screening and annual average health care visits) for each effectiveness outcomes (women with a mammography recorded in SITAM within 105 and 145 days) was explored using a logistic model including group (intervention/control), the potential effect modifier and the interaction modifier by group. We report the estimated effect of the intervention within levels of the modifier and the difference of effect between levels of the modifier alongside 95% confidence intervals.
We also calculated the percentage of agreement (strongly agree/agree responses) with acceptability and appropriateness statements: number of answers strongly agree/agree out of the total of women who responded. All percentages of agreement with acceptability and appropriateness statements were calculated among women who mentioned that had received SMS messages.
We used R statistical software (version 3.5.0) for the analysis.
Inclusion Criteria:
Exclusion Criteria: