Evaluating the Validity of an Eye Gaze Paradigm in Predicting Autism Spectrum Disorder

Evaluating the Validity of an Eye Gaze Tracking Assessment Tool in Identifying Autism Spectrum Disorder

The primary purpose of the present study is to evaluate the diagnostic validity of eye tracking measurements acquired during viewing of socially-relevant stimuli in predicting ASD diagnosis. The secondary purpose was to explore the potential prognostic value of eye tracking measures through cross-sectional associations with non-verbal cognitive ability.

Deficits in eye gaze are a hallmark sign of autism. A large and growing body of research supports the ability of eye-tracking based measurements to sensitively discriminate individuals with ASD and healthy participants. These investigations have identified that the core deficit in autism as disruption of social attention, reflecting an inability to appropriately engage and track socially- and emotionally-relevant aspects of the visual world. Thus, eye gaze tracking, acquired during viewing of socially-relevant stimuli, may be a useful approach to identifying objective markers of ASD. Eye tracking also carries the advantages of being less intrusive and expensive than MRI and genetic testing and specifically focuses on the core neurobehavioral characteristics of ASD - abnormalities in social attention.

After diagnosis of ASD, key clinical tasks in young children involve determining an accurate prognosis and tracking the progress of early interventions. Currently, the only prognostic indicators are clinical observations (subjective and expensive) and non-verbal cognitive ability testing (difficult to acquire, time-consuming, unavailable in many settings). Recently, eye gaze tracking was found to predict functional outcomes. Thus, in addition to being an objective marker for ASD, eye tracking measurements have potential to be useful for predicting cognitive and functional outcomes. Similarly, the only available methods for tracking treatment progress are parental reports (highly subjective), clinical observations (subjective and expensive), and cognitive measurements (expensive and unavailable in many settings. This study will evaluate, using cross-section data, the potential for eye tracking data to serve as a proxy for non-verbal cognitive ability scores in determining prognosis for ASD-affected children. Additionally, this study will evaluate the test re-test reliability of eye tracking parameters that can potentially be used to track treatment progress.

The current study will occur in four phases: pilot testing, development, validation, and re-test. In each phase, participants will view a visual attention stimulus with social elements (social attention paradigm) while eye tracking measurements are remotely acquired. The visual attention paradigm will be refined in the pilot testing but will remain the same for development, validation, and re-test phases. The entire process, including calibration and viewing of the visual paradigm, will take about 15 minutes. The text below describes each phase in detail and the reviews specific methodology for the social attention paradigm and eye tracking procedures.

1. Pilot Phase. Pilot testing of the social attention paradigm used to elicit eye gaze measurements to ensure they are sufficiently engaging across the ages of children participating. The visual attention paradigm during this phase is expected to be 10-12 minutes. This phase will also be used to determine optimal strategies for maintaining attention throughout the testing period. Initial data may also provide some insights into the most discriminating stimulus elements to include in future phases. Investigators anticipate recruiting approximately 10 ASD and 10 non-ASD participants in this phase. The pilot phase is expected to require 2.5 months of recruitment and 0.5 months of data reduction and analysis.
2. Development Phase. Once pilot testing is completed, 30 ASD-affected and 30 non-ASD children ages 1.5 to 18 will be recruited. Each child will complete the social attention paradigm while their eye gaze is remotely tracked. Eye gaze data is then scored for more than 100 parameters and these scores are entered into a database that includes diagnostic and cognitive information about each participant. The eye tracking data is then analyzed to create the diagnostic and prognostic algorithms that will refine the system's diagnostic sensitivity and specificity in the ASD population. The development period will require 6 months of data collection, followed by 3 months of scoring and algorithm computation.
3. Validation Phase. Collection of the validation sample will include 60 ASD-affected and 60 non-ASD affected children ages 1.5-18. The validation phase will use the same visual social attention paradigm and methods used in the development sample. The validation period will overlap algorithm development and require 8 months of data collection and 1 month of analysis.
4. Re-Test Phase. Participants in the re-test phase will include 30 ASD and 30 non-ASD children recruited in the first half of the validation phase. These participants will be explained in the validation phase that investigators will be re-recruiting them for the re-test phase approximately 3-6 months following their initial testing during the validation phase.

The target sample size varies depending on the phase of the study. Below are the target sample sizes for each phase:

Pilot Phase = 10 ASD and 10 Non-ASD Development Phase = 30 ASD and 30 Non-ASD Validation Phase = 60 ASD and 60 Non-ASD Re-Test Phase = 30 ASD and 30 Non-ASD (from the validation phase) The study population is individuals with autism spectrum disorder, or a clinical diagnosis of another developmental or psychiatric disorder (developmental/psychiatric controls), or have no specific developmental or psychiatric diagnosis (healthy controls), ages 1.5 (18 months) to 18 (120 months) at time of consent.

Eye gaze data will be collected using a remote eye tracker from Sensori-motoric instruments (SMI). Remote eye tracking offers minimal invasiveness to the viewer's field of view and collects time-stamped, 3D eye position, and binocular gaze and pupil data at a sampling rate of 120 Hz. Eye gaze capture is automatically calibrated to 2/5/9 points and provides position accuracy to 0.5° at a 60cm viewing distance. Gaze tracking data, screen recordings, user events, and gaze position will be recorded simultaneously. Data will be analyzed with emphasis on areas-of- interest and dwell time on specific targets on a second-by-second basis. Examples of the types of measures captured include: dwell time to any area of the stimulus, dwell time to the face and non-face regions of a human form on the video, fixation shifts between stimuli.

Additional tests and demographic data will be collected from standard of care autism diagnostic and behavioral health diagnostic appointments that can be found in the medical record.

Pilot Analyses. In the pilot phase, investigators will compute the effect size (Cohen's d) between ASD and non-ASD participants for each of the eye gaze parameters acquired for each of the individual stimuli in the social attention paradigm. Stimulus elements eliciting the largest discrimination between ASD and non-ASD patients will be retained in the development phase.

ASD Diagnostic Algorithm Analyses. In the development phase, all of the eye gaze measurements acquired from the social attention paradigm will be included as predictor variables in a random forest analysis. This analysis permits evaluation of the discriminative ability of a large number of variables in data sets with a modest number of cases. The variables with highest importance scores, indicating good diagnostic discrimination, will be entered into a logistic regression analysis with ASD diagnostic status (ASD vs. non-ASD) as the dichotomous dependent variable. Significant predictors will be retained and coefficients from the retained predictors will serve as the diagnostic algorithm.

The diagnostic algorithm scores will then be submitted to Receiver Operating Characteristic (ROC) curve analyses to provide detailed evaluation of sensitivity and specificity of the algorithm in the detection of ASD. Areas under the curve of >.90 are expected, indicating strong diagnostic validity.

Prognostic Algorithm Analyses. To identify a prognostic algorithm, a similar process will be conducted with all the available eye tracking measurements as predictors and non-verbal cognitive ability scores (dichotomized at <70 and 70 and above) as the dependent variable in random forest analyses. Non-verbal cognitive ability scores will be dichotomized based on previous data suggesting that individuals with ASD and intellectual disability show worse outcome. The predictor variables with highest importance, indicating good diagnostic discrimination, will then be entered into a logistic regression analysis with non-verbal cognitive ability (<70, >=70) as the dichotomous dependent variable. Significant predictors will be retained and coefficients from the retained predictors will serve as the prognostic algorithm. The prognostic algorithm scores will then be submitted to Receiver Operating Characteristic (ROC) curve analyses to provide detailed evaluation of sensitivity and specificity of the algorithm in the detection of non-verbal cognitive disability. Areas under the curve of >.80 are expected, indicating good validity in predicting cognitive disability. Non-verbal cognitive ability will be the primary focus of these analyses because of its documented relationship with outcome in individuals with autism. However, similar analyses will be computed for verbal cognitive ability and adaptive function scores. Investigators will also conduct the above analyses using continuous measurements as dichotomization may unnecessarily deflate validity.

Treatment Tracking Algorithm Analyses. To identify a treatment tracking algorithm, investigators will simply combine the eye tracking measurements identified in the diagnostic and prognostic algorithm into a single treatment tracking algorithm. To evaluate test re-test reliability of these measurements investigators will use intra-class correlation coefficients (model 2 - absolute agreement).

Inclusion Criteria:

• Clinical diagnosis of Autism Spectrum Disorder (299.0) following evaluation in Cleveland Clinic Center for Autism Diagnostic Clinic, or a clinical diagnosis of another developmental or psychiatric disorder, or have no specific developmental or psychiatric diagnosis.
• Age 1.5 to 18 years at time of consent.

Exclusion Criteria:

• Individuals whom, with corrective lenses are still legally blind.
• Individuals whom, it is determined at the discretion of hte Primary Investigator, after consultation with the evaluating psychologist in the Center for Autism Diagnostic Clinic, are not able to sufficiently attend to the stimulus presentation or have substantial challenging behaviors that would prohibit participation.