Alexithymia explains atypical spatiotemporal dynamics of eye gaze in autism

TL;DR: In this article, a multilevel polynomial modeling strategy was used to describe the spatiotemporal dynamics of eye gaze to emotional facial expressions, and it was shown that atypical gaze to the eyes is best predicted by alexithymia in both autistic and non-autistic individuals.

About: This article is published in Cognition.The article was published on 2021-03-22 and is currently open access. It has received 11 citations till now. The article focuses on the topics: Alexithymia & Facial expression.

Summary

1. Introduction 1

• Autism Spectrum Disorder (henceforth ‘autism’) is a condition defined by atypical social 2 interaction and communication, and restricted patterns of thought and behaviour (APA, 3 2013).
• While the exact mechanisms linking these symptoms have not been determined, two 6 hypotheses specify a role for arousal.
• Such redundancies may explain 6 why allocation of gaze to facial features is not always predictive of emotion recognition, 7 particularly with dynamic stimuli.
• Under the alexithymia hypothesis, 15 inconsistent results when testing various socioemotional abilities in the autism literature are 16 due to sampling variance with respect to alexithymia.

1.2.1. Spatiotemporal dynamics 4

• In addition to a failure to consider co-occurring alexithymia, there are at least three 5 potentially significant methodological limitations of previous studies.
• This practice does not adequately describe gaze 8 behaviour, which is a dynamic process serving to optimise eye movements for sequential 9 selection of relevant visual information (Land, 1999).
• 12 While it is possible that differences between groups in the timecourse of gaze allocation to 13 facial expressions are captured by aggregate metrics, some divergence between aggregate and 14 timecourse metrics is to be expected because dynamic face stimuli convey information which 15 varies in its spatial location over time.
• Non-linearities in the distribution of gaze allocation 16 are likely in response to more naturalistic and moving stimuli (Tatler et al., 2011).
• Importantly, these data (e.g., gaze data in visual world studies) have 7 highlighted that the underlying distribution of gaze is not always well captured in the 8 aggregate (“average”) pattern of gaze data collapsed across time (Barr, 2008; Mirman et al., 9 2008, Seedorff, 2018).

1.2.2. Eye-movement differences and data quality 11

• Eye movements are typically parsed into fixations (to gaze targets) and saccades.
• To do 23 so, areas of the visual scene are determined to contain ‘Areas of Interest’ (AOIs; e.g., the eye-24 region of a face).
• It is important to situate any theory of gaze allocation to emotional facial expressions within 11 wider theories of the function of gaze and visual perception.
• Given the demonstrated dependence of gaze patterns on the task to be 13 performed, thought to be mediated by task-dependent perceptual predictions, it is perhaps 14 unsurprising that different patterns of results have been found across these studies.
• Under this hypothesis, autistic participants may 18 be expected not to modulate their gaze behaviour based on task requirements or priors about 19 the stimuli, or not to do so to the same degree that neurotypicals do (Król & Król, 2019).

1.4. The present study and experimental hypotheses 6

• The overarching goal of this study was to examine whether atypical eye gaze to emotional 7 facial expressions is a product of alexithymia or autism.
• 11 We also aimed to address previous methodological issues in several ways.the authors.
• To investigate the role of task- and emotion-related priors on gaze behaviour, each 16 participant's gaze was recorded when they were, and were not, cued to the upcoming 17 emotion, and when they were asked to complete a range of tasks (free-gaze, emotion 18 recognition, and intensity judgements).
• It can therefore be determined whether any autism- or 19 alexithymia-related atypicality is dependent upon task parameters, or instead a general 20 feature of gaze behaviour.

2. Methods 8

• 1.Participants 9 75 participants were recruited from a database of research volunteers and compensated with 10 either an honorarium or course credit.
• Data from 45 neurotypical individuals 13 and 25 individuals diagnosed with autism were therefore included in the final sample (for 14 details see Table 1 and Table S.2. in Supplementary materials).
• Participants were diagnosed 15 by specialist clinicians all of whom were independent from the research team.
• As part of their 18 inclusion in the participant database used for recruitment in this study, participants were 19 assessed using the original ADOS algorithm administered by a certified professional.
• On the 4 day of the study, participants were tested individually in a soundproofed, dimly-lit room, and 5 took short breaks between tasks.

2.2.1. Dynamic Emotional Face Paradigm 10

• The dynamic emotional face paradigm consisted of four conditions in which participants 11 were shown videos of naturalistic dynamic emotional facial expressions from a validated 12 dataset (Yitzhak et al., 2017).
• 192 trials were 16 presented, 48 in each condition (free-gaze, emotion recognition, cued, and intensity 17 judgement), eight of each emotion (neutral, happy, sad, angry, fear, disgust) displayed by four 18 male and four female actors.
• Note that these analyses do not control for alexithymia.
• This was supplemented by a short system delay of random length at the end of each 4 trial sequence to allow for time-critical logging and trial preparation.
• The experiment was 9 programmed and presented in PyGaze (Dalmaijer, Mathôt, & Van der Stigchel, 2014).

2.2.2. Eye-tracking 11

• A Tobii TX300 (Tobii, Sweden) screen-based remote eye-tracker was used with the original 12 screen, sampling at 300Hz and tracking an area of 1920 x 1080 pixels, managed by an 13 experienced operator.
• A 5-point calibration was administered at the start of each block and repeated until all 17 points were successfully calibrated according to the eye tracker’s default criteria.
• 23 Recalibration after every 20 trials was based on a pilot experiment showing that participants 24 could complete approximately 20 trials at a time while maintaining an optimal position, with 25 minimal movement, and without triggering drift correction.
• The first 150ms of 7 each trial were excluded from analysis to account for gaze reorientation during screen 8 transitions and fixations initiated before stimulus onset, and the viewing window was locked 9 to 7800ms of full data after re-zeroing.
• 15 16 Trials and participants with more than 35% of data loss were not analysed.

2.2.4. Derivation of AOIs 1

• Objective AOIs were derived using an adapted implementation of a recently proposed 2 Limited-Radius Voronoi-tessellation method (LRVT) that builds on open-source facial 3 recognition and landmarking software (Baltrušaitis & Robinson, 2016; Hessels et al., 2018).
• The method allows identification of the location of facial features in a video over time.
• AOIs 5 are then created by segmenting the faces using the distances to predefined points indicating 6 distinct features (eyes, mouth, nose).
• This 9 method is noise-robust for sparse stimuli like faces and recommended for group comparisons 10 of gaze metrics (Hessels et al., 2018).
• The TAS-20 was used to measure alexithymia and was scored according to the original 16 norms, with each item rated on a scale from 1 (completely disagree) to 5 (completely agree) 17 which were then summed for a total score (Parker, Taylor, & Bagby, 2003).

2.3.2. Autism Spectrum Quotient – AQ28 and AQ50 19

• To quantify autistic traits, participants completed the AQ28 (Hoekstra et al., 2011).
• One total 20 score was computed by scoring the 4-point Likert-scale according to the original AQ scoring 21 criteria.
• The AQ28 was used to quantify autistic traits as recent work suggests it is more 22 consistent with clinical judgment than the AQ50 (Ashwood et al., 2016), and that the AQ50 23 in its entirely likely contains redundancies that do not necessarily provide improved precision 24 of measurement (Cuve et al., 2021, Lundqvist & Lindner, 2017).
• AQ50 scores were also 1 available for the autistic participants, and these were used to establish whether autistic 2 participants' scores were above established clinical thresholds.

2.3.3. Depression, Anxiety and Stress Scale – DASS21 4

• To control for any effect of depression and anxiety, these traits were assessed using the 5 DASS21 (Lovibond & Lovibond, 1995).
• 15 Unlike traditional analyses which aggregate gaze behaviour across time, modelling gaze 16 behaviour in this way allows four metrics to be derived.
• In order to identify when significant effects were observed, a non-parametric 15 cluster-based bootstrapping analysis was performed (Maris & Oostenveld, 2007).
• These 18 models could then be directly contrasted to test whether adding alexithymia or autism 19 improved model fit over simpler models.
• 22 For all models, quasi-maximal random structures were initially fitted.

2.4.1. Significance testing and model comparison 11

• An additional validation 16 used the step function from the lmerTest package to fit a backwards elimination regression 17 and identify eliminated terms.
• As part of the model 19 assessment, information criteria (AIC and BIC) were also used to contrast different models 20 (e.g., alexithymia vs. autism models).
• The full set of comparisons are listed in Table S.3 in 21 Supplementary materials.
• Control analyses 1 Control analyses ensured that data quality or effects of other covariates (e.g., autistic traits, 2 age, IQ, anxiety, ADOS and DASS scores) did not change the results of the main analyses 3 (due to space limitations these are presented in the Supplementary materials - Control and 4 additional analyses).

3. Results 6

• To enable comparison with previous literature, the authors present first the standard fixation measures 9 that aggregate over the stimulus viewing window.
• Two metrics are reported, the number of 10 fixations, and the total duration of fixations.
• 11 Gaze distribution to face regions followed expected patterns (see Figure 2 A & B).
• These results are line with other research on face scanning and 16 supported the focus on eye-gaze in subsequent analyses.
• There was also a slight preference 17 for the nose over the mouth, and interactions with condition; however, to simplify 18 interpretation of results in relation to the research hypotheses, the authors fit separate models for each 19 region of interest (see below).

3.1.1. Fixation count to the eyes 6

• 2 3.1.1.2.Condition effects relative to the free-gaze baseline: effects of alexithymia and 3 autism 4.
• Thus, data from aggregate measures suggest that individuals with alexithymia adapt 1 their gaze behaviour according to the task to be performed.
• 2 Overall, traditional analyses of fixation counts support H1; that alexithymia is a better 3 predictor of reduced eye gaze than autism.

3.2. Timecourse models for eye gaze 10

• The timecourse models of eye gaze data use polynomial regressors locked to the stimulus 11 viewing window to describe the changes in eye gaze proportion across time and how these 12 patterns are impacted by alexithymia and autism (whether measured as the presence or 13 absence of an autism diagnosis or the degree of autistic traits) and conditions.
• Initially, an 14 analysis of the free-gaze condition alone is presented, which is followed by an analysis of 15 how the pattern of eye gaze changes when comparing each condition to the free-gaze 16 condition.
• This latter analysis also describes how alexithymia and autism impact the changes 17 in eye gaze patterns as a function of condition.
• Only critical terms are described in the text 18 below.

3.2.1. Free-gaze condition: Effects of alexithymia and autism 20

• Similarly, 3 the backwards stepwise validation analysis based on AIC eliminated both autism diagnosis 4 and autistic traits and their interactions with polynomial terms, but not alexithymia and 5 interactions with higher order polynomials.
• 6 Estimates of the alexithymia model suggested a significant intercept (Estimate = 2.72, SE = 7 .15, p < .001) indicating that overall, participants were more likely to look at the eyes than 8 not.
• There was a significant main effect of alexithymia (Estimate = -. 50, SE = .14, p < .001), 9 indicating that alexithymia is associated with decreased gaze allocation to the eye region.
• 10 Alexithymia interacted with higher order polynomials suggesting that temporal evolution of 11 gaze behaviour differs as a function of alexithymia, with highly alexithymic individuals 12 reaching near asymptote after reduced attention to the eyes early in the trial (see Figure 3).
• 13 Full details and mouth gaze models are presented in Table S.4 in Supplementary materials.

3.2.2. Condition effects relative to the free-gaze baseline: Effects of alexithymia and 15

• Autism 16 Timecourse data and model predictions are illustrated in Figure 3. 24 for visualisation purposes only, as alexithymia was treated as a continuous predictor.
• 10 11 Similarly, the backwards stepwise selection validation analysis removed the majority of the 12 autism terms and none of the alexithymia terms.
• A potential concern that can be visualised in Figure 3 relates to the rapid increase in gaze to 4 the eyes seen at the start of the viewing window.
• Excluding the first 1000ms resulted in similar 20 results, with the only notable difference being that the best fitting model was achieved with 21 up to the cubic polynomial, without the quartic term being required.
• Thus, with respect to the study hypotheses, the timecourse analysis supported H1 in showing 10 that alexithymia explained atypical eye gaze to emotional facial expressions better than 11 autism (either the presence of a diagnosis or autistic traits).

3.2.3. Non-parametric cluster-based analysis 17

• Whereas the previous analyses allow the overall trajectory of gaze to the eyes to be described 18 across time, they do not allow identification of the particular time period(s) over the course of 19 a trial when effects of alexithymia or autism can be seen.
• There were no 15 significant clusters for the autistic traits model.
• 19 Overall, these analyses highlight that alexithymia is related to consistently reduced gaze to 20 the eyes across conditions and over time (see Figure 4), and therefore support H1; that 21 alexithymia explains reduced 22 eye gaze better than autism.
• Simpler 2 models including only the effects of condition and the polynomials were favoured in model 3 selection for the mouth (see ‘Timecourse for the Mouth AOI’ in Supplementary materials for 4 full details).
• 5 6 3.3.Stationary (SGE) and transition (GTE) gaze entropy 7 Hypothesis 4 stated that gaze will be more predictable (as demonstrated by reduced entropy) 8 in the emotion recognition and intensity judgement tasks, and when participants are cued to 9 the upcoming emotion, compared to the free-gaze condition.

3.4. Emotion recognition and gaze 1

• Both autism diagnosis and alexithymia predicted emotion recognition accuracy.
• There was however no effect of looking time to eyes on emotion recognition accuracy (Z = 8 .80, p = .4) nor interactions between looking time and clinical predictors on this measure (all 9 p > .05), which was consistent when replacing eyes with nose and mouth looking measures.
• 10 Similarly, correlations between gaze measures, accuracy and intensity ratings were all below 11 .1 and non-significant (see Table S2 in Supplementary materials).

4. Discussion 16

• This study investigated whether autism or alexithymia was the better predictor of eye gaze 17 while individuals with and without autism and alexithymia completed four tasks with 18 dynamic emotional facial expressions.
• Previous studies addressing eye gaze in autism have 19 aggregated gaze behaviour across time, analysing the number of fixations on, and the total 20 duration of attention to, the eyes (Grynszpan & Nadel, 2014; Hadjikhani, Åsberg Johnels, et 21 al., 2017; Kliemann et al., 2012).
• Analysis and revising the manuscript, also known as 10 BS.
• Eye 34 movements during emotion recognition in faces.

Figures

Figure 2. Aggregated fixation count (A) and duration (B) for eyes, mouth and nose. 2 Participants showed preferential looking to the eyes. B. Fixation count (C) and duration (D) 3 as a function of alexithymia (condition indicated by line colour). Alexithymia predicted both 4 reduced fixation count and reduced duration of eye gaze. 5

Figure 3. Timecourse analysis. A – B. Raster plots showing reduced gaze to the eye with 2 increasing alexithymia (A) and separated into groups with high and low alexithymic traits 3 (for visualisation only; B); C. Timecourse model for gaze to the eye region. Shaded ribbons 4 represent 95% Confidence Intervals. Solid lines represent predictions obtained from the 5 multilevel polynomial models. The alexithymia model outperforms both the autism diagnosis 6 and autistic traits models in predicting non-linear eye gaze behaviour. The analyses above 7 excluded the first 150 ms of gaze data; the results remained consistent when excluding the 8 first 500 ms or 1000ms. Note that the dichotomisation between high and low alexithymia is 9

Figure 5. Entropy effects. A. Transition matrices showing the difference in conditional 2 transition probabilities between AOIs (eyes, mouth, nose) as a function of alexithymia 3 (contrasted as high vs. low; median split for visualisation). Positive values indicate increased 4 transition probability for highly alexithymic individuals whereas negative values represent 5 decreased probability of transitions for highly alexithymic individuals. B-C. Scatter plots for 6 linear mixed models showing that increasing alexithymia is associated with increased entropy 7 in both autistic and non-autistic individuals. Dashed line represents global trend line and 8 faded lines illustrate random variability across trials. 9

Figure 4. Cluster analysis of timecourse effects. A-D. Shaded areas indicate clusters. If 2 significant, clusters are marked: *** p < .001, ** p < .01, * p < .05. Y axis represent the size 3 of the effect of the standardized predictors on eye gaze. Dashed y intercepts represent lower 4 critical significance limits for estimates from Linear Mixed Models. Alexithymia 5 consistently predicted significant reduced eye-looking behaviour across time and conditions 6 (red shades); E-H. Density plots from the nonparametric cluster bootstrapping. X axis 7 indicate summed estimates for all the time bins in clusters and x intercepts indicate individual 8 clusters coloured by alexithymia (red) and autism diagnosis (green), autistic traits (blue). 9

Figure 1. Study design and analysis workflow. A. An example trial from the free-gaze 5 condition. B. Sample stimuli with facial landmarks from OpenFace. C. Sample time series of 6 raw eye movements. D. Sample stimuli with superimposed AOI segmentation. E. Sample 7 gaze data for a single trial for an example participant, showing the proportion (elog) of gaze 8 points in the eye AOI across time (top). Gaze points per AOI were then modelled using the 9

Frequently asked questions

Q1. What are the contributions in this paper?

The authors propose that, where observed, atypical visual exploration of emotional facial expressions 5 is due to alexithymia, a distinct but frequently co-occurring condition. In this eye-tracking 6 study the authors tested the alexithymia hypothesis using a number of recent methodological 7 advances to study eye gaze during several emotion processing tasks ( emotion recognition, 8 intensity judgements, free gaze ), in 25 adults with, and 45 without, autism. 

Q2. What are the future works in this paper?

The current results can inform future research in four ways. Second, models of social attention in autism need to incorporate 9 potential causal influences of alexithymia on socioemotional behaviour. The findings highlight potential underlying 4 visual processing mechanisms of atypical emotion processing in alexithymia and autism. 

Q3. What is the likely explanation for the reduced allocation of gaze to eyes?

Non-linearities in the distribution of gaze allocation 16 are likely in response to more naturalistic and moving stimuli (Tatler et al., 2011). 

Q4. What is the AQ28 used to measure autistic traits?

The AQ28 was used to quantify autistic traits as recent work suggests it is more 22 consistent with clinical judgment than the AQ50 (Ashwood et al., 2016), and that the AQ50 23 in its entirely likely contains redundancies that do not necessarily provide improved precision 24of measurement (Cuve et al., 2021, Lundqvist & Lindner, 2017). 

Q5. How did the authors model eye gaze behaviour over time?

14Moving beyond traditional gaze metrics, separate analyses incorporated the temporal 15 domain by modelling eye gaze behaviour over time. 

Q6. What is the effect of reduced entropy on gaze?

Hypothesis 4 is that gaze will be more predictable (as 14 demonstrated by reduced entropy) in the emotion recognition and intensity judgement tasks, 15 and when participants are cued to the upcoming emotion, compared to the free-gaze 16 condition. 

Q7. What is the effect of free gaze on autism?

Free-20 gaze conditions may therefore be more likely to reveal reduced eye attention in the autistic 21 population, as a result of co-occurring alexithymia, than active tasks such as emotion 22 recognition. 

Q8. What is the effect of condition on the predictability of gaze transitions?

consistent with the entropy analysis of the dispersion of gaze fixations (SGE), the 11 predictability of gaze transitions as a consequence of condition reveals less of an influence of 12 top-down priors relating to task, or knowledge of the upcoming emotional face stimulus, in 13 alexithymic individuals. 

Q9. What is the effect of task on the predictability of gaze behaviour?

if autism (or alexithymia) is characterised by a reduced reliance on priors, 4 these effects of task on the predictability of gaze behaviour should be reduced. 

Q10. What is the motivation for the manipulations?

Such manipulations were motivated by previous findings of task-dependant gaze 11 behaviour (Del Bianco et al., 2018; Hessels et al., 2019; Ricciardelli, Carcagno, Vallar, & 12 Bricolo, 2013) and theories suggesting that autistic perception is less affected by priors than 13 neurotypical perception (Pellicano & Burr, 2012). 

Q11. What were the three groups of statistical models tested for the gaze timecourse and entropy?

20To summarise, the authors tested three groups of statistical models for both the gaze timecourse and 21 entropy: 22Group A. Separate models for Autism and Alexithymia: 23A.1. Autism Diagnosis Models: Simple models (orthogonal polynomials: linear, quadratic, 1 cubic and quartic for timecourse analyses; condition: free gaze, emotion recognition, intensity 2 judgement, cued) compared to more complex models including autism diagnosis, condition, 3 polynomials (for timecourse analyses) and their interactions.