Eye-trackers are now widely available and are marketed as user-friendly, affordable, and easy to use in all populations. ²⁷ There are several factors that need to be considered when purchasing an eye-tracker for use in a preverbal cohort: available budget; necessity of a chin-rest, suitability for use in an infant cohort; if software and data processing is included; compatibility with study objectives; accuracy, precision, and sample rate of the eye-tracker. ^{19 , 28} Another factor that needs careful consideration is the type of eye-tracker (screen-based or head-mounted). Screen-based eye-trackers, often called remote eye-trackers, are mounted below a screen, allowing the participant to sit, free of attachments, in front of the screen displaying the visual stimuli. Head-mounted eye-trackers are usually mounted onto glasses and allow the participant to walk around freely and to participate in physical tasks. ^{19 , 28}

We chose the Tobii© (Tobii, Sweden) X2-60 eye-tracker provided by iMotions ( iMotions, Denmark). The Tobii X2-60 system tracks both eyes of the subject with a sample rate of 60 Hz and a rated accuracy of 0.3°. ²⁹ The eye-tracker can be used in various experimental designs, and the track box allows for freedom of head movement during the recordings within a 50 × 36 cm (Width × Height) area, which makes it more suitable for use in infants.

ET measures should be chosen and defined during the study design phase. Selecting the appropriate ET measures affects the type of questions that can be asked. Different ET measures are required for specific tasks and these tasks assess different aspects of cognitive function. The most common ET measures used in ET tasks are based on fixations or saccades, with examples of these listed below ^{30 – 32} : •

Fixations and gaze points

Each measure provides different information, and hence it is essential to have research questions planned prior to the study so that the correct measures can be selected. Six tasks were used in our study assessing visual attention, working memory, and social cognition. The measures collected during these tasks were: time to first fixation, mean duration of fixations, the percentage number of fixations, and the percentage duration of fixations within a selected area of interest.

Commercially available software may not be suitable for acquisition or analysis in the study of young children, as age-appropriate methodologies are required. ^{9 , 18} While the use of ET has increased, the software is generally designed for use in marketing or research with adolescents or adults and may not be suitable for use in preverbal children with variable receptive language. Many commercially available software packages do not allow for child-controlled tasks (i.e., tasks that are dependent on child behaviour) ¹⁸ nor allow for tasks to be paused, which is essential when the child is unsettled or needs a break during the assessment, to help prevent data loss. Calibration routines are also usually not child friendly; they are long, have too many calibration points, and often require the assessor to try and direct the infant to each of the points. The results of the calibration are often not provided. Commercial manufacturers do not include “attention-grabbers”. These are usually colourful images that are played on the screen accompanied by a sound such as a horn and are helpful in redirecting children back to the tasks on the screen.

We identified the ET tasks package from the British Autism Study of Infant Siblings (BASIS) in Birkbeck College at University of London as suitable for our purposes. ^{33 , 34} The tasks included in this package have been used in studies with both typically and atypically developing infants ^{24 – 26 , 33} and assess visual attention, working memory, and social cognition. This task package is designed to run within the Matlab (MathWorks, USA) programming environment. This allows for the programming of attention grabbers that can be played throughout the tasks to direct the child’s attention back to the monitor if they become distracted. The calibration provided is infant friendly, the tasks package can be paused when necessary, and some of the tasks are controlled by the infant (e.g., the working memory task). The Birkbeck tasks package was installed on an Apple Mac desktop PC using the macOS Sierra 10.12.6 operating system. Psychtoolbox (V3.0.14, a Matlab toolbox) with GStreamer (v1.0 runtime ³⁵ ) was also installed on the desktop. This program presents the tasks on a secondary monitor and runs the experiment by communicating with the eye-tracker, sequencing the tasks, and recording the ET data. The Birkbeck tasks package is not open-source code but can be reprogrammed using the freely available and open-source psychtoolbox and therefore the structure and arrangement of the tasks detailed in this report are easily reproducible.

Data quality is paramount and optimising this from the outset is crucial. There are many factors that can affect data quality, including the position of the eye-tracker, the monitor, how the infant is positioned, the accuracy and precision of the eye-tracker, the experience of the assessor conducting the experiment, the calibration, the behaviour of the infant, ^{11 , 19} and the physiology of the infant's eye (e.g. infants with blue eyes have more data loss than infants with brown eyes). ^{36 , 37} The accuracy and precision are dependent on the ET device and should be provided by the manufacturers. However, it is important to keep in mind that these figures are often acquired under optimal experimental conditions, usually with adult participants and with the use of chin rests to limit movement. ¹⁹ Chin rests will not be acceptable to a young preverbal child.

Data loss can affect the quality of data and can occur for many reasons, including blinking, turning the head away from the eye-tracker, or technical issues with the eye-tracker. This is especially problematic in a cohort of preverbal children who may find it hard to stay still for the duration of an ET assessment. The correct positioning of the participant with respect to the eye-tracker is imperative to reduce data loss and, in turn, maintain the quality of the data. Manufacturers often provide instructions regarding the optimal position and orientation of the participant. However, positioning an infant can be difficult, as they will move freely, resulting in data loss. ¹⁸

Data quality for the Birkbeck task package was validated in an international assessment. ³⁸ The task package is used by the Eurosibs consortium and is a nine-site European study of neurocognitive development in infants with an older sibling with ASD. As part of the Eurosibs consortium, Jones et al. reported the data quality from the Birkbeck task package across the nine sites involved in the consortium. The data used was from a subset of infants tested at 5, 10 and 14 months of age. The infants were seated in their caregiver’s lap throughout the assessment. Data quality was assessed using one-way analysis of variance with the site as the independent factor. Overall, the quality of the ET data was consistently high across the sites. ³⁸

The layout of the ET lab and the set-up of equipment is critical. Noise, lighting, the position of the eye-tracker, and the position of the infant can all influence the quality of data collected. ²¹ External distractions such as noise can impact data acquisition, incorrect lighting can make it difficult for the eye-tracker to capture good quality data, and incorrect positioning of the eye-tracker with respect to the child can also lead to the collection of poor quality data and data loss.

We positioned our eye-tracker according to the instructions from the manufacturer, i.e., at the bottom of the monitor presenting the stimuli. The ET lab should be set up in a soundproof room, or in a quiet area, with no natural light. In our lab, the room is soundproofed and had one window with a blackout blind. The room also has a dimmer light to control level of lighting and ensures consistency in lighting across the assessments. We set up our lab to study children in the 1-2 years range, who are very mobile. Caregiver support is therefore crucial in order to keep children focused during testing. In our lab, we allowed children to sit on their caregiver’s lap, 60 cm from the monitor. A comfortable armchair was provided for the caregiver. A computer monitor with the eye-tracker attached was placed on a table that could be raised or lowered to a height appropriate for the child (see Figure 1). Both the table and chair had wheels and brakes; wheels were necessary in order to get the eye-tracker lined up to the child’s position, and the brakes were used to hold the position of the table and chair once established. Caregivers were asked to close their eyes or to look away from the monitor to prevent the eye-tracker measuring their data (see Figure 2).

Implementation of a standard operating procedure (SOP) for testing ensures a consistent approach for all study participants. The SOP can be found in the Extended data.

To ensure the SOP and the set-up was appropriate, and to become familiar with the procedure and tasks, we first completed a pilot study. The pilot study (n=5) allowed for the identification of any issues which might affect data acquisition or quality. As a result of this, we identified the need to develop an observation sheet to capture rich qualitative information pertinent to the study objectives. ⁴⁴ This observation sheet allowed us to track when obvious data loss occurred (head turned away, crying, needing a break, eye-tracker capturing caregivers’ eyes etc.) and to track important information about social cognition. We observed, for example, that data loss could occur not only when the child was distracted or had lost interest but also when they were interacting with their caregiver.

A visual acuity test should be completed before the ET assessment to assess visual acuity. This ensures that participants can see the stimuli presented that will be presented during testing. Visual acuity tests in children are quick and easy to perform. In our study, we used the Keeler Visual Acuity Test. Examples of the cards used can be seen in Figure 3. The cards are presented to the child. The side where the child looks first is recorded. This is not a clinical assessment of visual acuity and is only used to ensure sufficient acuity for the ET assessment (acuity card number 5 needed to be completed). If children are unable to complete the acuity test, the ET assessment began, and the acuity test was presented to the child again. If the child was still unable to complete the test, the data collected from this child was excluded from the analysis.

Calibration of the eye-tracker is necessary before data collection for each assessment. Calibration before the start of the assessment allows the eye-tracker to measure the characteristics of the eye and ensures accurate data collection. ^{21 , 39} There are no standards for calibration in a preverbal cohort which makes it difficult to put protocols in place for evaluating the calibration output and deciding when recalibration is necessary. ¹⁸ A successful calibration is important as it is a necessary assumption for all ET measures. Good calibration results in good quality data; poor calibration can result in poor quality or unusable data. ¹⁹ Although standards are lacking, a five-point calibration is most commonly used. ⁹ However, variations in the type of visual stimulus to display and the sequence in which the visual stimuli is displayed can differ between studies. Some ET manufacturers provide a calibration process that is suitable for use in young children, and which provides a pictorial representation of the calibration results for visual inspection. Some manufacturers simply provide a yes/no answer to alert the user as to whether the calibration was successful. If calibration results are not available in pictorial form, then it is difficult to adjust the infant for recalibration, and so these products are not optimal for use with infants and children. The calibration from the Birkbeck ET package was used in our study.

The Birkbeck ET package uses the Tobii Pro SDK to link to the Tobii ET. This is a free software development kit for research applications. Before calibration, the Birkbeck ET program displays a cartoon that allows the assessor to locate the children’s eyes. At the bottom of the screen, a red bar appears with ‘Too far’ and ‘Too close’ written at either end. The infant and the table are adjusted until the bar is half-full and the eyes are in the centre of the screen.

Once satisfied with the position of the infant, the 5-point calibration can begin. Children are shown a moving stimulus in the top-right hand corner, the top-left hand corner, the centre, the bottom-right hand corner, and the bottom-left hand corner of the screen. The stimuli are presented in random order for each calibration. Once the calibration is complete, the programme displays the results. A calibration of 4/5 is deemed acceptable if a 5/5 calibration was not possible to achieve after four attempts. ^{40 , 41} Figures 4- 6 show calibration examples. The calibration process will repeat until accurate ET data can be captured. ³⁸ If possible, we recommend running a recalibration throughout the tasks to avoid drift due to movement of child and to ensure high data quality. ¹⁹

Data collection can be monitored during the assessment by watching a preview window that appears on the top-left hand corner of the operator’s monitor. If the eye-tracker loses detection of the eyes, the operator pauses tasks and adjusts the position of the child. See Figure 7 for an example of this preview window.

This preview window displays the task that is currently being shown, as well as the location of the child’s eyes in relation to the task (see the green, red, and blue dots in Figure 7). When the eye-tracker is unable to pick up the location of the eyes, these dots are no longer visible on the preview window, and the background changes from black to red. In the top-left hand corner of the preview window itself, there is a red rectangle that indicated where the infants head is in relation to the eye-trackers track box. The information gathered from the preview window allows for adjustment of the infant or eye-tracker in real-time to maintain the quality of the ET data.

If a break in the recording is necessary, the tasks can be paused. If the child is distracted or losing attention, an attention-grabbing sound can be played. If the child is unable to complete the ET assessment, the program can be stopped without losing data already collected. Once the assessment is finished, the data will be saved and stored securely. The duration for completion of all tasks should be noted.

To analyse ET data, some data processing of the raw gaze data is necessary. ^{18 – 20} Multiple steps may be necessary when processing the data to identify fixations, saccades, blinks, and lost data. ^{19 , 20} Caution is required when processing data, as each step may require decisions that could adjust the definition of the measures and therefore introduce bias into the results. For example, different studies use different fixation duration thresholds ranging from 100 ms–300 ms. ²⁰ Hence, it is important that each step is considered carefully and the rationale behind the decision is clear, transparent, and consistent for all participants to allow for reproducibility. The software which will complete the data processing automatically may be provided by the manufacturers. However, it is often not clear how the data is being processed, and decisions such as fixation duration threshold cannot be altered. ²⁰ Often, the ET measures generated by the manufacturer’s software are limited and may not be suitable for research. Data can be exported and processed independently, and while it may be more time consuming, it allows for complete control over how the raw gaze data is treated, including the fixation duration threshold and the ET measures generated. Access to expertise in programming is beneficial for data analysis.

We manually remove data of insufficient quality and time points where the eyes were not detected by the eye-tracker. The ET uses validity codes (0-4) to define how certain it is that it has found an eye. The lower the value, the more certain it is. A validity code ≤2 was discarded as per the Tobii Pro knowledge article. ⁴² Gaze data was then split according to the ET tasks used. For a subset of tasks, fixations were calculated using a dispersion-threshold identification algorithm previously used by Krassanakis, Filippakopoula and Nakos ⁴³ which is available on the repository GitHub. This algorithm works on the basis that fixation points tend to cluster together due to their low velocity. For our pilot study, the duration threshold was set at 150ms, and a normalised distance measure (t1) of 0.1 was used as the maximum separation distance between points to define a fixation. A t2 (tolerance) ²³ was estimated from the standard deviation of the cluster of points, which removed outliers from a fixation.

UCC endeavours to adhere to the FAIR Data principles and is very keen and willing to make this data open and accessible. To do this, ethical approval will be required in order to make the raw gaze data collected for the cohort available in an open repository. The ethics application process necessitates completing a Data Protection Impact Statement which requires review by the data controller’s data protection officer which may take time. The process of acquiring this approval to allow other researchers access to the data has commenced but in the meantime, we are happy for anyone interested in the raw gaze data to contact the author sonia.lenehan@ucc.ie and we will work with them to fulfil their request. We hope that this process will not take longer than 12 months.

Zenodo: Supplementary materials for: Computerised Eye-tracking as a Tool for Early Neurodevelopmental Assessment in the Pre-Verbal Child, https://doi.org/10.5281/zenodo.5896774. ⁴⁵

This project contains the following extended data: -

Supplementary Materials and Appendix.docx (A Word document containing: The full standard operation procedure and the observation sheet)