Showing papers on "Surprise published in 2022"

Shared computational principles for language processing in humans and deep language models

Abstract: Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

Predictability and Surprise in Large Generative Models

Abstract: Large-scale pre-training has recently emerged as a technique for creating capable, general-purpose, generative models such as GPT-3, Megatron-Turing NLG, Gopher, and many others. In this paper, we highlight a counterintuitive property of such models and discuss the policy implications of this property. Namely, these generative models have a paradoxical combination of predictable loss on a broad training distribution (as embodied in their ”scaling laws”), and unpredictable specific capabilities, inputs, and outputs. We believe that the high-level predictability and appearance of useful capabilities drives rapid development of such models, while the unpredictable qualities make it difficult to anticipate the consequences of model deployment. We go through examples of how this combination can lead to socially harmful behavior with examples from the literature and real world observations, and we also perform two novel experiments to illustrate our point about harms from unpredictability. Furthermore, we analyze how these conflicting properties combine to give model developers various motivations for deploying these models, and challenges that can hinder deployment. We conclude with a list of possible interventions the AI community may take to increase the chance of these models having a beneficial impact. We intend for this paper to be useful to policymakers who want to understand and regulate AI systems, technologists who care about the potential policy impact of their work, funders who want to support work addressing these challenges, and academics who want to analyze, critique, and potentially develop large generative models.

Deep Learning Based Vulnerability Detection: Are We There Yet?

Abstract: Automated detection of software vulnerabilities is a fundamental problem in software security. Existing program analysis techniques either suffer from high false positives or false negatives. Recent progress in Deep Learning (DL) has resulted in a surge of interest in applying DL for automated vulnerability detection. Several recent studies have demonstrated promising results achieving an accuracy of up to 95 percent at detecting vulnerabilities. In this paper, we ask, “how well do the state-of-the-art DL-based techniques perform in a real-world vulnerability prediction scenario?” To our surprise, we find that their performance drops by more than 50 percent. A systematic investigation of what causes such precipitous performance drop reveals that existing DL-based vulnerability prediction approaches suffer from challenges with the training data (e.g., data duplication, unrealistic distribution of vulnerable classes, etc.) and with the model choices (e.g., simple token-based models). As a result, these approaches often do not learn features related to the actual cause of the vulnerabilities. Instead, they learn unrelated artifacts from the dataset (e.g., specific variable/function names, etc.). Leveraging these empirical findings, we demonstrate how a more principled approach to data collection and model design, based on realistic settings of vulnerability prediction, can lead to better solutions. The resulting tools perform significantly better than the studied baseline—up to 33.57 percent boost in precision and 128.38 percent boost in recall compared to the best performing model in the literature. Overall, this paper elucidates existing DL-based vulnerability prediction systems’ potential issues and draws a roadmap for future DL-based vulnerability prediction research.

Cyber Security in the Age of Covid-19: A Timeline and Analysis of Cyber-Crime and Cyber-Attacks during the Pandemic

Abstract: The entire world has been impacted by Covid-19, which has forced the nations to undergo lockdowns, which have opened the new horizons of virtual learning and work from home culture. Though virtual learning and online meetings were available prior to the lockdowns, use of these platforms have intensified. There is a lack of awareness among many in the public concerning the perils of using these platforms which makes them susceptible to cyber-crime attacks phishing, sexual or verbal abuse, eve teasing etc. Lockdowns have provided cyber criminals with new criminal opportunities and evidence shows that there has been a rampant increase in the number of cyber-crimes during this period. Public lack of awareness has led to innocent people falling prey in the hands of attackers. Since the pandemic and ensuing lockdowns came as a surprise, public and private authorities lacked the opportunity to make appropriate arrangements for training people and making these platforms secure. There is no question that if these platforms are made secure, then they will prove to be as an asset to the society but much work must be done to achieve the goal of complete cyber security.

How does the presence of a surgical face mask impair the perceived intensity of facial emotions?

Abstract: The use of surgical-type face masks has become increasingly common during the COVID-19 pandemic. Recent findings suggest that it is harder to categorise the facial expressions of masked faces, than of unmasked faces. To date, studies of the effects of mask-wearing on emotion recognition have used categorisation paradigms: authors have presented facial expression stimuli and examined participants’ ability to attach the correct label (e.g., happiness, disgust). While the ability to categorise particular expressions is important, this approach overlooks the fact that expression intensity is also informative during social interaction. For example, when predicting an interactant’s future behaviour, it is useful to know whether they are slightly fearful or terrified, contented or very happy, slightly annoyed or angry. Moreover, because categorisation paradigms force observers to pick a single label to describe their percept, any additional dimensionality within observers’ interpretation is lost. In the present study, we adopted a complementary emotion-intensity rating paradigm to study the effects of mask-wearing on expression interpretation. In an online experiment with 120 participants (82 female), we investigated how the presence of face masks affects the perceived emotional profile of prototypical expressions of happiness, sadness, anger, fear, disgust, and surprise. For each of these facial expressions, we measured the perceived intensity of all six emotions. We found that the perceived intensity of intended emotions (i.e., the emotion that the actor intended to convey) was reduced by the presence of a mask for all expressions except for anger. Additionally, when viewing all expressions except surprise, masks increased the perceived intensity of non-intended emotions (i.e., emotions that the actor did not intend to convey). Intensity ratings were unaffected by presentation duration (500ms vs 3000ms), or attitudes towards mask wearing. These findings shed light on the ambiguity that arises when interpreting the facial expressions of masked faces.

Neurons learn by predicting future activity

Abstract: Understanding how the brain learns may lead to machines with human-like intellectual capacities. It was previously proposed that the brain may operate on the principle of predictive coding. However, it is still not well understood how a predictive system could be implemented in the brain. Here we demonstrate that the ability of a single neuron to predict its future activity may provide an effective learning mechanism. Interestingly, this predictive learning rule can be derived from a metabolic principle, where neurons need to minimize their own synaptic activity (cost), while maximizing their impact on local blood supply by recruiting other neurons. We show how this mathematically derived learning rule can provide a theoretical connection between diverse types of brain-inspired algorithms, thus, offering a step toward development of a general theory of neuronal learning. We tested this predictive learning rule in neural network simulations and in data recorded from awake animals. Our results also suggest that spontaneous brain activity provides "training data" for neurons to learn to predict cortical dynamics. Thus, the ability of a single neuron to minimize surprise: i.e. the difference between actual and expected activity, could be an important missing element to understand computation in the brain.

Impact of face masks and sunglasses on emotion recognition in South Koreans

Abstract: Due to the prolonged COVID-19 pandemic, wearing masks has become essential for social interaction, disturbing emotion recognition in daily life. In the present study, a total of 39 Korean participants (female = 20, mean age = 24.2 years) inferred seven emotions (happiness, surprise, fear, sadness, disgust, anger, surprise, and neutral) from uncovered, mask-covered, sunglasses-covered faces. The recognition rates were the lowest under mask conditions, followed by the sunglasses and uncovered conditions. In identifying emotions, different emotion types were associated with different areas of the face. Specifically, the mouth was the most critical area for happiness, surprise, sadness, disgust, and anger recognition, but fear was most recognized from the eyes. By simultaneously comparing faces with different parts covered, we were able to more accurately examine the impact of different facial areas on emotion recognition. We discuss the potential cultural differences and the ways in which individuals can cope with communication in which facial expressions are paramount.

Ensemble-based face expression recognition approach for image sentiment analysis

Abstract: Sentiment analysis based on images is an evolving area of study. Developing a reliable facial expression recognition (FER) device remains a difficult challenge as recognizing emotional feelings reflected in an image is dependent on a diverse set of factors. This paper presented an ensemble-based model for FER that incorporates multiple classification models: i) customized convolutional neural network (CNN), ii) ResNet50, and iii) InceptionV3. The model averaging ensemble classifier method is used to ensemble the predictions from the three models. Subsequently, the proposed FER model is trained and tested on a dataset with an uncontrolled environment (FER-2013 dataset). The experiment demonstrated that ensembling multiple classifiers outperformed all single classifiers in classifying positive and neutral expressions (91.7%, 81.7% and 76.5% accuracy rate for happy, surprise, and neutral, respectively). However, when classifying disgust, anger, and sadness, the ResNet50 model alone is the better choice. Although the Custom CNN performs the best in classifying fear expression (55.7% accuracy), the proposed FER model can still classify fear expression with comparable performance (52.8% accuracy). This paper demonstrated the potential of using the ensemble-based method to enhance the performance of FER. As a result, the proposed FER model has shown a 72.3% accuracy rate.

Detection of Emotion of Speech for RAVDESS Audio Using Hybrid Convolution Neural Network

Abstract: Every human being has emotion for every item related to them. For every customer, their emotion can help the customer representative to understand their requirement. So, speech emotion recognition plays an important role in the interaction between humans. Now, the intelligent system can help to improve the performance for which we design the convolution neural network (CNN) based network that can classify emotions in different categories like positive, negative, or more specific. In this paper, we use the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) audio records. The Log Mel Spectrogram and Mel-Frequency Cepstral Coefficients (MFCCs) were used to feature the raw audio file. These properties were used in the classification of emotions using techniques, such as Long Short-Term Memory (LSTM), CNNs, Hidden Markov models (HMMs), and Deep Neural Networks (DNNs). For this paper, we have divided the emotions into three sections for males and females. In the first section, we divide the emotion into two classes as positive. In the second section, we divide the emotion into three classes such as positive, negative, and neutral. In the third section, we divide the emotions into 8 different classes such as happy, sad, angry, fearful, surprise, disgust expressions, calm, and fearful emotions. For these three sections, we proposed the model which contains the eight consecutive layers of the 2D convolution neural method. The purposed model gives the better-performed categories to other previously given models. Now, we can identify the emotion of the consumer in better ways.

Facial Expression Recognition Using a Temporal Ensemble of Multi-Level Convolutional Neural Networks

Abstract: Emotion recognition is indispensable in human-machine interaction systems. It comprises locating facial regions of interest in images and classifying them into one of seven classes: angry, disgust, fear, happy, neutral, sad, and surprise. Despite several breakthroughs in image classification, particularly in facial expression recognition, this research area is still challenging, as sampling in the wild is a demanding task. In this study, a two-stage method is proposed for recognizing facial expressions given a sequence of images. At the first stage, all face regions are extracted in each frame, and essential information that would be helpful and related to human emotion is obtained. Then, the extracted features from the previous step are considered temporal data and are assigned to one of the seven basic emotions. In addition, a study of multi-level features is conducted in a convolutional neural network for facial expression recognition. Moreover, various network connections are introduced to improve the classification task. By combining the proposed network connections, superior results are obtained compared to state-of-the-art methods on the FER2013 dataset. Furthermore, the performance of our temporal model is better than that of the single architecture of the 2017 EmotiW challenge winner on the AFEW 7.0 dataset.

Customer Emotions in Service Robot Encounters: A Hybrid Machine-Human Intelligence Approach

Abstract: Understanding consumer emotions arising from robot-customers encounters and shared through online reviews is critical for forecasting consumers’ intention to adopt service robots. Qualitative analysis has the advantage of generating rich insights from data, but it requires intensive manual work. Scholars have emphasized the benefits of using algorithms for recognizing and differentiating among emotions. This study critically addresses the advantages and disadvantages of qualitative analysis and machine learning methods by adopting a hybrid machine-human intelligence approach. We extracted a sample of 9707 customers reviews from two major social media platforms (Ctrip and TripAdvisor), encompassing 412 hotels in 8 countries. The results show that the customer experience with service robots is overwhelmingly positive, revealing that interacting with robots triggers emotions of joy, love, surprise, interest, and excitement. Discontent is mainly expressed when customers cannot use service robots due to malfunctioning. Service robots trigger more emotions when they move. The findings further reveal the potential moderation effect of culture on customer emotional reactions to service robots. The study highlights that the hybrid approach can take advantage of the scalability and efficiency of machine learning algorithms while overcoming its shortcomings, such as poor interpretative capacity and limited emotion categories.

The NAFLD‐MAFLD debate: Is there a Consensus‐on‐Consensus methodology?

Abstract: Polarizing opinions have recently arisen in hepatology on the name and redefinition of fatty liver disease associated with metabolic dysfunction. In spite of growing and robust evidence of the superior utility of the term metabolic (dysfunction) associated fatty liver disease (MAFLD) definition for clinical and academic practice, controversy abounds. It should therefore come, as no surprise that the most common arguments used in contrarian op‐eds is that there are no consensus on any name change. In this context, we suggest that discourse on an accurate understanding of what scientific consensus means, the various methods of achieving consensus, as well as other alternative models for reaching agreement is pivotal for the field. In this opinion piece, we provide an overview of these aspects as it applies to the case of fatty liver disease. We provide evidence that consensus on a change from non‐alcoholic fatty liver disease (NAFLD) to MAFLD has already been achieved. We believe that the time has come for redirecting stakeholder focus and energy on capitalizing on the momentum generated by the debate to improve the lives of people at its centre, our patients.

Emognition dataset: emotion recognition with self-reports, facial expressions, and physiology using wearables

Abstract: The Emognition dataset is dedicated to testing methods for emotion recognition (ER) from physiological responses and facial expressions. We collected data from 43 participants who watched short film clips eliciting nine discrete emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness. Three wearables were used to record physiological data: EEG, BVP (2x), HR, EDA, SKT, ACC (3x), and GYRO (2x); in parallel with the upper-body videos. After each film clip, participants completed two types of self-reports: (1) related to nine discrete emotions and (2) three affective dimensions: valence, arousal, and motivation. The obtained data facilitates various ER approaches, e.g., multimodal ER, EEG- vs. cardiovascular-based ER, discrete to dimensional representation transitions. The technical validation indicated that watching film clips elicited the targeted emotions. It also supported signals' high quality.

Photobiocatalysis in Continuous Flow

Abstract: In the last years, there were two fields that experienced an astonishing growth within the biocatalysis community: photobiocatalysis and applications of flow technology to catalytic processes. Therefore, it is not a surprise that the combination of these two research areas also gave place to several recent interesting articles. However, to the best of our knowledge, no review article covering these advances was published so far. Within this review, we present recent and very recent developments in the field of photobiocatalysis in continuous flow, we discuss several different practical applications and features of state-of-the art photobioreactors and lastly, we present some future perspectives in the field.

Wearing N95, Surgical, and Cloth Face Masks Compromises the Perception of Emotion

Abstract: According to the familiar axiom, the eyes are the window to the soul. However, wearing masks to prevent the spread of viruses such as COVID-19 involves obscuring a large portion of the face. Do the eyes carry sufficient information to allow for the accurate perception of emotions in dynamic expressions obscured by masks? What about the perception of the meanings of specific smiles? We addressed these questions in two studies. In the first, participants saw dynamic expressions of happiness, disgust, anger, and surprise that were covered by N95, surgical, or cloth masks or were uncovered and rated the extent to which the expressions conveyed each of the same four emotions. Across conditions, participants perceived significantly lower levels of the expressed (target) emotion in masked faces, and this was particularly true for expressions composed of more facial action in the lower part of the face. Higher levels of other (non-target) emotions were also perceived in masked expressions. In the second study, participants rated the extent to which three categories of smiles (reward, affiliation, and dominance) conveyed positive feelings, reassurance, and superiority, respectively. Masked smiles communicated less of the target signal than unmasked smiles, but not more of other possible signals. The present work extends recent studies of the effects of masked faces on the perception of emotion in its novel use of dynamic facial expressions (as opposed to still images) and the investigation of different types of smiles.The online version contains supplementary material available at 10.1007/s42761-021-00097-z.

LSD and creativity: Increased novelty and symbolic thinking, decreased utility and convergent thinking

Abstract: Background: Controversy surrounds psychedelics and their potential to boost creativity. To date, psychedelic studies lack a uniform conceptualization of creativity and methodologically rigorous designs. Aims: This study aimed at addressing previous issues by examining the effects of lysergic acid diethylamide (LSD) on creativity using multimodal tasks and multidimensional approaches. Methods: In a randomized, double-blind, placebo-controlled, crossover study, 24 healthy volunteers received 50 μg of LSD or inactive placebo. Near drug peak, a creativity task battery was applied, including pattern meaning task (PMT), alternate uses task (AUT), picture concept task (PCT), creative metaphors task (MET) and figural creativity task (FIG). Creativity was assessed by scoring creativity criteria (novelty, utility, surprise), calculating divergent thinking (fluency, originality, flexibility, elaboration) and convergent thinking, computing semantic distances (semantic spread, semantic steps) and searching for data-driven special features. Results: LSD, compared to placebo, changed several creativity measurements pointing to three overall LSD-induced phenomena: (1) ‘pattern break’, reflected by increased novelty, surprise, originality and semantic distances; (2) decreased ‘organization’, reflected by decreased utility, convergent thinking and, marginally, elaboration; and (3) ‘meaning’, reflected by increased symbolic thinking and ambiguity in the data-driven results. Conclusion: LSD changed creativity across modalities and measurement approaches. Three phenomena of pattern break, disorganization and meaning seemed to fundamentally influence creative cognition and behaviour pointing to a shift of cognitive resources ‘away from normal’ and ‘towards the new’. LSD-induced symbolic thinking might provide a tool to support treatment efficiency in psychedelic-assisted therapy.

What the fake? Probing misinformation detection standing on the shoulder of novelty and emotion

Abstract: One of the most time-critical challenges for the Natural Language Processing (NLP) community is to combat the spread of fake news and misinformation. Existing approaches for misinformation detection use neural network models, statistical methods, linguistic traits, fact-checking strategies, etc. However, the menace of fake news seems to grow more vigorous with the advent of humongous and unusually creative language models. Relevant literature reveals that one major characteristic of the virality of fake news is the presence of an element of surprise in the story, which attracts immediate attention and invokes strong emotional stimulus in the reader. In this work, we leverage this idea and propose textual novelty detection and emotion prediction as the two tasks relating to automatic misinformation detection. We re-purpose textual entailment for novelty detection and use the models trained on large-scale datasets of entailment and emotion to classify fake information. Our results correlate with the idea as we achieve state-of-the-art (SOTA) performance (7.92%, 1.54%, 17.31% and 8.13% improvement in terms of accuracy) on four large-scale misinformation datasets. We hope that our current probe will motivate the community to explore further research on misinformation detection along this line. The source code is available at the GitHub. 2

CMCL 2022 Shared Task on Multilingual and Crosslingual Prediction of Human Reading Behavior

Abstract: We present the second shared task on eye-tracking data prediction of the Cognitive Modeling and Computational Linguistics Workshop (CMCL). Differently from the previous edition, participating teams are asked to predict eye-tracking features from multiple languages, including a surprise language for which there were no available training data. Moreover, the task also included the prediction of standard deviations of feature values in order to account for individual differences between readers.A total of six teams registered to the task. For the first subtask on multilingual prediction, the winning team proposed a regression model based on lexical features, while for the second subtask on cross-lingual prediction, the winning team used a hybrid model based on a multilingual transformer embeddings as well as statistical features.

OUP accepted manuscript

Abstract: It would not surprise me if to a relative outsider, most musical-technical writings on jazz would seem rather puzzling, possibly even bewildering, and also—or perhaps especially—to the musically literate. Those who want to inform themselves about jazz would, I imagine, quickly come across books on jazz theory and harmony, only to be confronted with a mix of mysterious symbols and an abundance of chord formations within a web of modes and scales that range from the familiar, through the exotic, to the highly artificial. These rather one-sidedly pitch-oriented matters have a central role in jazz’s performance practice, but while for the uninitiated this theoretical vocabulary can make for a rocky start, for scholars by no means does it provide a sufficient analytical framework. However, this vocabulary is so central to the music that anyone dealing with jazz will have to engage with it one way or another, and will have to find a balance between a critical stance and adopting some of the elements.

Emotion Detection with Facial Feature Recognition Using CNN & OpenCV

Abstract: Emotion Detection through Facial feature recognition is an active domain of research in the field of human-computer interaction (HCI). Humans are able to share multiple emotions and feelings through their facial gestures and body language. In this project, in order to detect the live emotions from the human facial gesture, we will be using an algorithm that allows the computer to automatically detect the facial recognition of human emotions with the help of Convolution Neural Network (CNN) and OpenCV. Ultimately, Emotion Detection is an integration of obtained information from multiple patterns. If computers will be able to understand more of human emotions, then it will mutually reduce the gap between humans and computers. In this research paper, we will demonstrate an effective way to detect emotions like neutral, happy, sad, surprise, angry, fear, and disgust from the frontal facial expression of the human in front of the live webcam.

One bad apple can spoil your IPv6 privacy

Abstract: IPv6 is being more and more adopted, in part to facilitate the millions of smart devices that have already been installed at home. Unfortunately, we find that the privacy of a substantial fraction of end-users is still at risk, despite the efforts by ISPs and electronic vendors to improve end-user security, e.g., by adopting prefix rotation and IPv6 privacy extensions. By analyzing passive data from a large ISP, we find that around 19% of end-users' privacy can be at risk. When we investigate the root causes, we notice that a single device at home that encodes its MAC address into the IPv6 address can be utilized as a tracking identifier for the entire end-user prefix---even if other devices use IPv6 privacy extensions. Our results show that IoT devices contribute the most to this privacy leakage and, to a lesser extent, personal computers and mobile devices. To our surprise, some of the most popular IoT manufacturers have not yet adopted privacy extensions that could otherwise mitigate this privacy risk. Finally, we show that third-party providers, e.g., hypergiants, can track up to 17% of subscriber lines in our study.

Recognition of Emotions for People with Autism: An Approach to Improve Skills

Abstract: Autism spectrum disorder refers to a neurodevelopmental disorders characterized by repetitive behavior patterns, impaired social interaction, and impaired verbal and nonverbal communication. The ability to recognize mental states from facial expressions plays an important role in both social interaction and interpersonal communication. Thus, in recent years, several proposals have been presented, aiming to contribute to the improvement of emotional skills in order to improve social interaction. In this paper, a game is presented to support the development of emotional skills in people with autism spectrum disorder. The software used helps to develop the ability to recognize and express six basic emotions: joy, sadness, anger, disgust, surprise, and fear. Based on the theory of facial action coding systems and digital image processing techniques, it is possible to detect facial expressions and classify them into one of the six basic emotions. Experiments were performed using four public domain image databases (CK+, FER2013, RAF-DB, and MMI) and a group of children with autism spectrum disorder for evaluating the existing emotional skills. The results showed that the proposed software contributed to improvement of the skills of detection and recognition of the basic emotions in individuals with autism spectrum disorder.