Validation of Smart Mask V1 System: A Clinical Study Comparing a System for Detecting Sleep Stages and Arousals to Manual Polysomnography Scoring
Validation of Smart Mask V1 System: A Clinical Study Comparing a System for Detecting Sleep Stages and Arousals to Manual Polysomnography Scoring
The objective of this clinical study is to evaluate the accuracy of the Smart Mask V1 System (herein 'Smart Mask') in measuring sleep stages-Stage N1/N2, Stage N3, Rapid Eye Movement (herein 'REM'), and WAKE-arousals, and the Arousal Index in adults diagnosed with sleep-disordered breathing, such as obstructive sleep apnea (herein 'OSA'). The Smart Mask operates in concert with a Wireless Access Module (herein 'WAM'), which is connected to a standard positive air pressure (herein 'PAP') device used in the treatment of OSA. Collectively the Smart Mask and WAM operate neural network classifier algorithms to determine sleep stages, arousals, and Arousal Index. These algorithms are coded into an embedded software system called the Sleep Staging and Arousal Module (herein 'SSAM') that operates directly on the WAM. The SSAM processes the following parameters, collected while the participant is asleep: 1) instantaneous values of pulse rate, determined from embedded optical sensors within the Smart Mask that measure photoplethysmogram waveforms (herein 'PPG'); and 2) full-resolution flow waveforms measured by sensors within the PAP device and retrieved by the WAM.
During the study, volunteer participants (preferably those with OSA) will undergo an overnight sleep study in sleep testing facility located at three separate clinical sites. The test device (comprising the SSAM operating on the WAM) will retrospectively determine sleep stages and arousals, after the participant's sleep session has concluded. To evaluate the accuracy of the test device, its values of sleep stages, arousals, and Arousal Index will be compared to those parameters determined by polysomnography (herein 'PSG', a recognized gold-standard reference) and EnsoSleep (a FDA-cleared predicate device, and specifically a software package that uses artificial intelligence (AI) to determine sleep stages and arousals). Each volunteer participant will wear an FDA-cleared wrist-worn pulse oximeter called the 'Checkme O2' which generates data (specifically PPG waveforms and values of SpO2 and pulse rate) for the EnsoSleep cloud-based software platform.
The main questions this study aims to answer are:
Answers to these questions will be derived through comparative statistical analysis involving the test device, the gold-standard PSG reference, and the FDA-cleared predicate device, employing methodologies similar to those used in the validation of the EnsoSleep. The study will include two cohorts. The first cohort will include approximately 75 participants from a single clinical site and will be used for device training purposes. The second cohort will consist of approximately 72 different participants, and will be used to validate the test device. Participants in the second cohort will be distributed roughly evenly across two separate clinical sites.
CLINICAL APPROACH: To validate the SSAM's performance in determining sleep stages (N1/N2, N3, REM, Wake), arousals, and Arousal Index, two separate clinical studies are proposed. Both studies will use the same success criteria: to demonstrate that SSAM's performance is not statistically different from, or is statistically better, compared to the performance of its predicate device. More specifically, the studies, which will take place simultaneously, will compare the SSAM-derived measurements to those obtained from both a gold-standard (manual PSG scoring) and its predicate device (EnsoSleep, which uses AI-powered automated scoring). The proposed clinical approach is essentially identical to that used for Smart Mask's predicate device, EnsoSleep, as described in its 510(k) Summary. It will use the same number of participants, statistical analyses, and success criteria, as summarized below.
The SSAM detects autonomic arousals by processing cardio-respiratory signals, whereas PSG allows the scoring of cortical arousals through the measurement of brain electrical activity and other physiological signals. Despite these differences, PSG is proposed as the gold-standard reference based on the following considerations:
Based on these considerations, the proposed validation study will be conducted using two clinical cohorts tested at three sleep testing facilities. The first cohort will be used for training the neural-network classifiers used by SSAM (for both sleep stages and arousals), and will feature about 75 participants tested at a first clinical site. The second cohort will be used to validation the neural-network classifiers used by SSAM, and will feature at least 72 participants tested at a second and third clinical site, both independent of the first clinical site. The sample size for the second, validation cohort has been powered using a statistical model, and is consistent with the sample size used in the EnsoSleep validation study. All participants used in both the training and validation cohorts will be representative of the SSAM's intended-use population (i.e. patients suffering sleep-disordered breathing, e.g. from central or obstructive sleep apnea).
The study will adhere to guidelines published by the American Academy of Sleep Medicine ('AASM'). All relevant documentations will be submitted for review and approval by a third-party Institutional Review Board ('IRB') and an IRB associated directly with one of the study sites. Study activities will not begin until the IRBs approve the clinical protocol and Informed Consent Form associated with the study.
The proposed clinical study will be performed at sleep testing facilities associated with three separate clinical sites: 1) University of California, San Diego ('UCSD'); 2) the Peninsula Sleep Center (https://www.peninsulasleep.com/) located in Burlingame, CA ('Peninsula'); and Amnova Research (https://amnovaresearch.com) located in Irvine, CA ('Amnova). UCSD has a dedicated IRB associated with the university, whereas Amnova and Peninsula will rely on the third-party IRB. Participants will primarily be recruited from the patient populations associated with these facilities.
VALIDATION STUDY PROTOCOL: The proposed clinical studies will follow a two-phase protocol. Phase I of the study will use study participants from the first cohort. This Phase will be used to collect data from the test, reference, and predicate devices; these data will be used to further train the neural-network classifiers used by SSAM. Participants enrolled in this Phase will not be included in the second (validation) cohort and will not participate in Phase II. Phase II will use study participants from the second cohort, and will serve as the formal validation study and will support the premarket notification (510(k)) submission to the U.S. FDA. Participants enrolled will be individuals diagnosed with sleep-disordered breathing and prescribed PAP therapy.
Each participant in either phase will undergo an overnight PSG study at one of the designated clinical sleep testing sites. During the study, participants will wear the Smart Mask, which interfaces with a PAP device configured to match each participant's prescribed therapy settings. Participants will be selected to ensure representation across various PAP modalities, including continuous positive airway pressure (CPAP), bilevel positive airway pressure (BiPAP), auto-titrating PAP (Auto-CPAP) and auto-titrating BiPAP (Auto-BiPAP) .
Benchtop studies using the Smart Mask and PAP devices with a simulated breathing system, incorporating replay libraries for normal breathing, snoring, and OSA, have shown that variations in therapy modality and pressure exert minimal impact on the flow waveforms analyzed by SSAM. Therefore, stratified enrollment across therapy types, with an aggregate sample size of approximately 72 participants, is deemed sufficient for clinical validation.
PSG will be conducted using equipment and data collection parameters outlined in the 'AASM Manual for the Scoring of Sleep and Associated Events'. Throughout the night, several physiological parameters will be recorded using the following equipment (sensors and monitors) attached to the participant. These may include: 1) EEG to monitor brain activity; 2) EOG to track eye movements; 3) EMG to assess muscle tone; 4) ECG to measure heart rate and rhythm; 5) a chest belt to monitor respiratory effort; 6) a respiratory sensor to evaluate airflow and oxygen levels; 7) pulse oximeter to measure PR and SpO2; 8) sound probes to capture snoring and other noises; and 9) a synchronized camera to record video of leg and other movements. In addition to these PSG sensors, each participant will also wear the Checkme O2 device, worn on the wrist and fingers, to capture additional signals for the EnsoSleep cloud-based software platform (e.g., PPG waveforms and values of SpO2 and pulse rate).
Upon completion of the study sessions, clinicians in the sleep testing facility will remove the Smart Mask from the participant and connect it to the WAM, which will then calculate sleep stages, arousals, and Arousal Index as described above. These data will be downloaded onto a computer (used for testing purposes only and not included in the SSAM's submission for premarket approval) and included in their respective analysis, as described below.
Once PSG data is collected and stored, three independent registered PSG technologists ('RPSGTs') will manually score the data to estimate sleep stages and arousals by following guidelines in the 'The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications', which is the definitive reference for the scoring of PSG and HSATs. The RPSGTs will be blinded, and, when analyzing the PSG data, a 2/3 majority score will be required to indicate each sleep stage and arousal event.
PPG waveforms and values of SpO2 and pulse rate measured by the Checkme O2 device will be uploaded to EnsoSleep's cloud-based software platform for analysis. Once this is complete, data from this software system will be downloaded, and included in its respective analysis, as described below.
DATA ANALYSIS AND SUCCESS CRITERIA: For each participants in the trial, sleep stages, arousals, and arousal indices will be determined by the test, gold-standard, and predicate devices described herein. These data will be collected and then synchronized after the measurement is complete. For each parameter, an external statistician, provided by the Sponsor, will group data into 30-second epochs, aggregate all epochs from all participants, and then use bootstrap estimates on this data set to determine the percent positive agreement ('PA'), percent negative agreement ('NA'), and the overall agreement ('OA'), along with the 95% confidence intervals, for:
Data collected during the study will be displayed and analyzed in a manner similar to that described in the 510(k) Summary for EnsoSleep. Success criteria for sleep stages, arousals, and Arousal Index-as also described in EnsoSleep's 510(k) Summary-is that for Evaluation 1 the PA, NA, and OA performance that is observed is not statistically significantly different or is statistically significantly greater in all comparisons relative to Evaluation 2. Stated another way, the overall success criteria, similar to that used by EnsoSleep, is to definitively prove using the intended-use population that SSAM achieves clinical performance (PA, NA, OA with PSG) for sleep staging, arousals, and arousal index that is substantially equivalent to its predicate device's clinical performance (again, PA, NA, OA with PSG) for the same parameters.
Inclusion Criteria:
Exclusion Criteria:
Burlingame, California 94010, United States
mehran@peninsulasleep.com650-779-4055
traci@peninsulasleep.com
maggie@amnovaresearch.com(949)777-5818
rowens@health.ucsd.edu858-534-2230