Individuals with autism are known to face challenges with emotion regulation, and express their affective states in a variety of ways. With this in mind, an increasing amount of research on automatic affect recognition from speech and other modalities has recently been presented to assist and provide support, as well as to improve understanding of autistic individuals' behaviours. As well as the emotion expressed from the voice, for autistic children the dynamics of verbal speech can be inconsistent and vary greatly amongst individuals. The current contribution outlines a voice activity detection (VAD) system specifically adapted to autistic children's vocalisations. The presented VAD system is a recurrent neural network (RNN) with long short-term memory (LSTM) cells. It is trained on 130 acoustic Low-Level Descriptors (LLDs) extracted from more than 17 h of audio recordings, which were richly annotated by experts in terms of perceived emotion as well as occurrence and type of vocalisations. The data consist of 25 English-speaking autistic children undertaking a structured, partly robot-assisted emotion-training activity and was collected as part of the DE-ENIGMA project. The VAD system is further utilised as a preprocessing step for a continuous speech emotion recognition (SER) task aiming to minimise the effects of potential confounding information, such as noise, silence, or non-child vocalisation. Its impact on the SER performance is compared to the impact of other VAD systems, including a general VAD system trained from the same data set, an out-of-the-box Web Real-Time Communication (WebRTC) VAD system, as well as the expert annotations. Our experiments show that the child VAD system achieves a lower performance than our general VAD system, trained under identical conditions, as we obtain receiver operating characteristic area under the curve (ROC-AUC) metrics of 0.662 and 0.850, respectively. The SER results show varying performances across valence and arousal depending on the utilised VAD system with a maximum concordance correlation coefficient (CCC) of 0.263 and a minimum root mean square error (RMSE) of 0.107. Although the performance of the SER models is generally low, the child VAD system can lead to slightly improved results compared to other VAD systems and in particular the VAD-less baseline, supporting the hypothesised importance of child VAD systems in the discussed context.

/pdf/evaluating-the-impact-of-voice-activity-detection-on-speech-1gxiphj8.pdf

Evaluating the Impact of Voice Activity Detection on Speech Emotion Recognition for Autistic Children

Research has recognized the importance of individ-ual differences of children with Autism Spectrum Disorder (ASD) that require interventions to meet their heterogeneous needs. This relatively large-scale study investigates a robot-assisted autism therapy (RAAT) with 34 children with diverse forms of ASD and Attention Deficit Hyperactivity Disorder (ADHD). We conducted a multi-session study with multi-purposeful activities targeting the socio-emotional abilities of children in a rehabilitation setting. We found a number of quantitative results suggesting various autism-related and demographic differences such as diverse forms of ASD, co-occurrence of ADHD, verbal skills, and age groups. The main findings are: 1) severity of ASD forms may not predict intervention outcomes but instead the co-occurrence of ADHD with LFA diagnosis may negatively impact social smiling; 2) verbal children were more generally engaged and less aggressive with the robot than non-verbal children whose curiosity rose over sessions; and 3) younger children (3.4 y.o.) showed more affection, while older children (7–12 y.o.) were better engaged through speaking more words and having longer engagement and eye contact with the robot.

Individual Differences of Children with Autism in Robot-assisted Autism Therapy

Social robots have great potential for the therapy of children with autism spectrum disorder, but the practical use of them is challenging. In this article, we presented a social robot YANG, which can interact with children with autism in their daily training. We propose a therapist‐robot interactive (TRI) model, which integrates with the practice of discrete trial training (DTT), a basic method utilized in autism training. To evaluate the TRI model, we implemented the model in YANG and conducted a single‐subject experiment in a rehabilitation training center. Data were collected on three children (ages 3–4) and their three therapists as they interacted with YANG in training sessions. Results showed that the children's learning ability significantly improved. YANG formed natural and friendly relationships with the children and delivered substantial support to therapists. Our research brings insight into using social robots for children with autism.

Using a social robot for children with autism: A therapist‐robot interactive model

Various benefits are being envisioned for enhancing autism interventions with a robot. But what features should such interventions have if they are to be successful? While there are quite a few papers that describe specific user requirements or needs, a more comprehensive account thereof should help to inform the development of such interventions. We therefore present a literature review on the user requirements for robot-assisted interventions. We report on various themes that emerged from our analysis and discuss how enhancing an intervention with a robot might fulfil those requirements.

User Requirements for Developing Robot-Assisted Interventions for Autistic Children

Research has recognized the importance of individ-ual differences of children with Autism Spectrum Disorder (ASD) that require interventions to meet their heterogeneous needs. This relatively large-scale study investigates a robot-assisted autism therapy (RAAT) with 34 children with diverse forms of ASD and Attention Deficit Hyperactivity Disorder (ADHD). We conducted a multi-session study with multi-purposeful activities targeting the socio-emotional abilities of children in a rehabilitation setting. We found a number of quantitative results suggesting various autism-related and demographic differences such as diverse forms of ASD, co-occurrence of ADHD, verbal skills, and age groups. The main findings are: 1) severity of ASD forms may not predict intervention outcomes but instead the co-occurrence of ADHD with LFA diagnosis may negatively impact social smiling; 2) verbal children were more generally engaged and less aggressive with the robot than non-verbal children whose curiosity rose over sessions; and 3) younger children (3.4 y.o.) showed more affection, while older children (7–12 y.o.) were better engaged through speaking more words and having longer engagement and eye contact with the robot. 

Predictability is important to autistic individuals, and robots have been suggested to meet this need as they can be programmed to be predictable, as well as elicit social interaction. The effectiveness of robot-assisted interventions designed for social skill learning presumably depends on the interplay between robot predictability, engagement in learning, and the individual differences between different autistic children. To better understand this interplay, we report on a study where 24 autistic children participated in a robot-assisted intervention. We manipulated the variance in the robot’s behaviour as a way to vary predictability, and measured the children’s behavioural engagement, visual attention, as well as their individual factors. We found that the children will continue engaging in the activity behaviourally, but may start to pay less visual attention over time to activity-relevant locations when the robot is less predictable. Instead, they increasingly start to look away from the activity. Ultimately, this could negatively influence learning, in particular for tasks with a visual component. Furthermore, severity of autistic features and expressive language ability had a significant impact on behavioural engagement. We consider our results as preliminary evidence that robot predictability is an important factor for keeping children in a state where learning can occur.

https://dl.acm.org/doi/pdf/10.1145/3468849

Predictable Robots for Autistic Children—Variance in Robot Behaviour, Idiosyncrasies in Autistic Children’s Characteristics, and Child–Robot Engagement

Carlos Neves

Papers

Predictable Robots for Autistic Children—Variance in Robot Behaviour, Idiosyncrasies in Autistic Children’s Characteristics, and Child–Robot Engagement