Device incorrect colour-coded classification of fast breathing for RR

Pneumonia is the leading infectious cause of death among children under 5 years of age globally. Many pneumonia deaths result from late care seeking and inappropriate treatment due to misdiagnosis of symptoms. The United Nations Children's Fund's (UNICEF's) Acute Respiratory Infection Diagnostic Aids (ARIDA) project aims to introduce automated respiratory rate (RR) counting aids for use by frontline health workers in resource limited community settings and health facilities. These RR counting aids aim to offer improved accuracy, effectiveness and acceptability compared to current practices for counting and classifying RR to detect fast breathing pneumonia.

The general aim of the controlled accuracy study is to understand whether the ARIDA test device accurately measures RR in children under 5 years of age with cough and/or difficult breathing. It is a cross-sectional, prospective study in a controlled setting comprising three types of device evaluations:

1. The accuracy of the ARIDA test device in measuring RR in young infants 0 to <2months, children 2 to <12 months and 12 to 59 months when compared to a video panel reference standard will be established through the first evaluation.
2. The consistency of the ARIDA test device will be established by determining the level of agreement between the measures of RR for two ARIDA test devices when used on the same child at the same time in those aged 2 to <12 months and 12 to 59 months through the second evaluation.
3. A third evaluation on a different group of normal-breathing children aged 2 to 59 months will assess RR fluctuation over time due to ARIDA test device attachment.

Evaluations 1 and 2 for accuracy and consistency will also be undertaken with expert clinicians (EC) conducting a manual RR count to further the evidence base around the performance of current standard practice in a controlled setting.

The study is a cross-sectional, prospective study and will be conducted in paediatric in and outpatient departments Saint Paul's Hospital and Millennium Medical Collage in Addis Ababa, Ethiopia.

General aim:

To understand whether the ARIDA test device accurately measures respiratory rate (RR) in children under 5 years of age with cough and/or difficult breathing.

Primary objective:

To determine the performance of the ARIDA test device, as defined by accuracy and consistency in children under 5 years with cough or difficulty breathing in a controlled setting.

Secondary objectives:

To determine the performance of expert clinicians (ECs) counting RR, as defined by accuracy and consistency, in children under 5 years with cough or difficulty breathing in a controlled setting.

To measure RR fluctuation over time after ARIDA test device attachment, in normal breathing children aged 2 to 59 months, in a controlled setting.

Sample size: A total 290 respiratory rate assessments will be done. The accuracy evaluation will aim to enrol 156 children (52 per age group: 0 to <2 months, 2 to <12 months and 12 to 59 months). The consistency evaluation will aim to enrol 104 children (52 per age group: 2 to <12 months and 12 to 59 months). The accuracy and consistency evaluations will aim to enrol children in a ratio of 3:1 fast:normal breathers in each age group. The RR fluctuation evaluation will aim to enrol 30 normal breathing children age 2 to 59 months.

Data collection:

Overview of evaluation stages:

1. Training of the research team: 3 days
2. Pre-testing of procedural activities and debrief: 5 days
3. Screening procedure will take place to ascertain study eligibility. Written parent or guardian informed consent will be obtained.
4. Children will be enrolled prospectively based on eligibility determined by the screening procedure.
5. Each child will participate in at least one of the evaluations: accuracy, consistency, RR fluctuation

Steps for device evaluation:

All ECs conducting manual RR counts are blinded to the RR result from the pre-screen RR evaluation and the ARIDA test device readings during the evaluation. The RR reading during the pre-screening is recorded by the RA on a data collection form that is not seen by the EC manual counter(s). The ARIDA test device screen is covered during the assessment and the RR readings were recorded by the RA and entered on to password protected tablets. VEPs are blinded to all EC, ARIDA and pre-screen RR readings and the other VEP's RR readings.

Accuracy evaluation:

Up to three attempts are made to obtain an ARIDA test device and EC manual RR reading with the MK2 ARI timer. The RA attaches the ARIDA test device to the child and ensures the child is positioned correctly according to device instructions. The child is calmed by the parent or guardian or by the EC who distracts the child through clicking their fingers or make hushing sounds. When the child is calm, the videographer starts video recording and the ARIDA test device and manual EC RR count start simultaneously. The time taken to get an ARIDA test device reading is also obtained using a stopwatch. The EC RR count lasts for 60 seconds. After 60 seconds, if the EC has not obtained a RR reading, the EC attempt is recorded as unsuccessful and repeated for both the EC and ARIDA test device. The ARIDA test device count lasts up to five minutes. After five minutes, or if the ARIDA test displays an error message, the ARIDA test attempt is recorded as unsuccessful and repeated for both EC and ARIDA test device. If the third attempt is still unsuccessful for either device, the evaluation is recorded as a failure.

Consistency evaluation:

The consistency evaluation follows the same procedure as the accuracy evaluation. To obtain a measure of consistency between two ARIDA test devices, two test devices are attached to a single belt, in line with each nipple and measured RR in the same evaluation period. To obtain a measure of ECs consistency, two ECs conduct a manual RR count with MK2 ARI timers in the same evaluation period. For an EC or ARIDA test attempt to be successful, both ECs or ARIDA test devices must have obtained RR readings. The videographer record the entire evaluation for quality assurance purposes.

Respiratory rate fluctuation evaluation:

A baseline EC manual RR count for 60 seconds is obtained on a calm child using the MK2 ARI timer. The research team are allowed multiple attempts to get a baseline reading. Following this, the ARIDA test device is attached to the child and the EC does a further three manual RR counts for 60 seconds in the following time periods: 0-1 minute, 2-3 minutes, 4-5 minutes.

The reference standard for the accuracy and fluctuation evaluations is a video review by two to three independent video expert panel (VEP) members whose counting is standardised weekly to agreement (≤±2 breaths per minute (bpm)). Two VEPs independently review recorded videos of the child's chest movements and count the number of breaths observed in a full minute, as per the WHO guidelines for RR counting. Each video is edited with an audible beep and 60-second timer to ensure the VEP counts the same 60 seconds as the EC. If the panel agreed (≤±2bpm), a median VEP reading is used. If the panel disagreed (>±2bpm), a third VEP reviews the video and if two out of these three readings agree, the median RR is used. If all three panel members disagree the result is sent to a fourth VEP. If two out of four VEP readings agree, a median VEP reading is used. If the panel disagree, the data from this evaluation is excluded from the analysis.

There is no video reference standard for the consistency evaluation.

Data storage:

All data will be collected using electronic forms uploaded on tablets and backed up to a protected cloud account.

Patient identifiers will not be utilised on data forms; a UIC and associated barcode will be used to anonymise patient data and track patient data. The UIC to be used for the study will have three numerical characters.

ECs and RAs will each have a numerical unique identifier, assigned at the beginning of the study.

Data quality assurance:

As part of the quality assurance procedures, standard clinical protocols, standard operating procedures (SOPs) and job aids will be prepared. The tablet based data collection forms will also have pre-programmed checks and restrictions to ensure the quality of the data inputted, and the data will be reviewed centrally on a regular basis.

Regular site visits will be conducted by Malaria Consortium study investigators to provide oversight and quality assure that the protocols, SOPs and job aids are adhered to by the research team.

Research reports will be submitted to relevant Ministry of Health (MOH) departments and Institutional Review Board (IRB).

Subject safety procedures:

The ARIDA test device has CE medical device approval. RAs and ECs will be trained on how to use the device safely, according to instructions provided by the manufacturers.

This study will not evaluate device efficacy or safety as a study endpoint and treatment will not be based on the outcomes of the study. It is important to note that the device, as part of the dossier to the stringent regulatory bodies has conducted their own efficacy studies and internal approvals.

The child/young infant will be evaluated in a health facility; a research nurse or clinical officer, trained in management of children and in Integrated Management of Newborn Childhood Illnesses (IMNCI), will perform a medical history and physical examination prior to evaluation to ensure no children with danger signs or 'pink' IMNCI referral signs are included in the study. Children who are deemed very ill will be excluded. ECs and RAs performing the evaluations will also be trained in the management of children and in IMNCI. In case a child has a medical complication before, during or after the evaluation, the research nurse or clinical officer or other medical professionals will manage this immediately in the health facility.

Subject safety, anticipated risks:

The ARIDA test device to be used on the children will not be invasive. The ARIDA test device will not elicit pain or other noxious stimuli when near or in contact with the child/young infant's body. It is therefore anticipated it will not cause any bodily harm or injury to the child and no adverse events (AEs) associated with the device are anticipated.

In the event of an AE following an evaluation with the ARIDA test device, the research nurse/clinical officer will determine whether the AE is attributable to the device. The project manager and or clinical director will report the AE to the in-country research ethics committee and the study principal investigator as per applicable guidelines.

Subject safety, adverse event recording and reporting:

The parent or guardians of the children will be requested to report any suspected AE occurring during and/or immediately after the evaluation using the devices.

The ECs/RAs while evaluating the child/young infant will be requested to alert the research team and clinical director and/or another medical professional of any untoward occurrence.

Any observed and/or reported AE (serious or non-serious), or suspected causal relationship to the investigational device, will be recorded on a severe AE form For all AEs, sufficient information will be pursued and/or obtained so as to permit: 1) an adequate determination of the outcome of the effect (i.e., whether the effect should be classified as a serious AE); and 2) an assessment of the casual relationship between the AE and the investigational device.

AEs felt to be associated with the investigational device will be followed until the effect (or its sequelae) resolves or stabilises at a level acceptable to the local project manager in discussion with the clinical director.

The research team will promptly review documented AEs. If the research team's final determination of causality is "unknown and of questionable relationship to the investigational device(s)", the AE will be classified as "associated with the use of the investigational device" for reporting purposes.

If the research team's final determination of causality is "unknown but not related to the investigational device(s)", this determination and the rationale for the determination will be documented in the respective subject's case history.

A copy of the completed form will be provided to the research team in country and to the study principal investigator. The completed AE form will be submitted to the regulatory authorities as per the country-specific reporting requirements.

Subsequent to the initial submission of a completed form, the research team will submit additional information concerning the reported AE as requested by the regulatory authorities.

If an investigational device elicits pain or other noxious stimuli when near or in contact with the child/young infant's body, or leads to any other untoward medical occurrence, the use of the device on the child/young infant will be stopped immediately, the child/young infant will be withdrawn from the study and steps will be taken to assess and record the AE.

The child/young infant will be managed by the EC or other medical professionals, and if the AE(s) is felt to be associated with the investigational device(s), the child/young infant will be followed until the event (or its sequelae) resolves or stabilises at a level acceptable to the research team.

A child/young infant will also be withdrawn from the study if the parent or guardian withdraws consent at any point in the study or if the study is discontinued for any reason.

Ethical approval: This clinical protocol as part of the ARIDA project has been approved by the Ethiopian IRB at national level. The ARIDA test device will be registered with the Food, Medicine and Health Care Administration and Control Authority of Ethiopia.

Inclusion criteria for child in accuracy and consistency evaluations for ARIDA test device and expert clinician:

A child (2 to 59 months) who fulfils ALL of the following eligibility criteria will be included in the accuracy and consistency evaluations:

1. Age 2 to 59 months
2. Cough or difficulty breathing
3. Parent or Guardian consent

A young infant (0 to <2 months) who fulfils ALL of the following eligibility criteria will be included in the accuracy evaluation:

1. Age 0 to <2 months
2. Parent or Guardian consent

Inclusion criteria for child in RR fluctuation evaluations:

A child who fulfils ALL of the following eligibility criteria will be included in the respiratory rate fluctuation evaluation:

1. Age 2 to 59 months
2. Parent or Guardian consent
3. Normal breathing

Exclusion criteria for child/young infant in all elements of the study:

A child/young infant with the following criteria will be excluded from all elements of this study:

1. General danger signs:

   1. Newborns (<28 days): not feeding well, active convulsions/fits, chest indrawing, fever (37.5 degrees or more), low body temperature (less than 35.5 degrees) or movement only when stimulated/no movement at all.
   2. All other age groups: active convulsions/fits, unconscious/lethargic, not breastfeeding/not drinking or vomiting everything.

2. Signs of severe pneumonia:

   1. Chest indrawing or
   2. Stridor in a calm child

3. IMNCI pink referral signs for severe disease including stridor, severe dehydration, severe persistent diarrhoea, very severe febrile disease, severe complicated measles, mastoiditis, complicated severe malnutrition, and severe anaemia.

4. Parent or Guardian's age less than 16 years.

5. No Parent or Guardian consent.

7. Device Manufacturer Safety Exclusion Criteria:

If newborn or young infant (<2months):

1. Born before 37 weeks of gestation (pre-term)

   All age groups:

2. Wearing supportive device at area of chest/belly

3. Skin not intact in chest/belly

In addition, a child with the following criteria will be excluded from the respiratory rate fluctuation evaluation:

1. Cough or difficult breathing
2. Fast breathing (see table 2)

Additional exclusion criteria for child/young infant in an in-patient facility:

1. Child/young infant being managed by barrier nursing (such as severe burns, child with neutropenia, severe infectious diseases).
2. Child/young infant not eligible for research procedures as advised by the supervising clinician.

The study subjects will be:

1. Children under 5 years of age attending outpatient or in-patient departments at the selected hospital (St Paul's Hospital, Addis Ababa).
2. Two ECs with extensive experience in assessing and treating children with fast breathing pneumonia using IMNCI guidelines, selected based on education, training and years of experience.