List of Figures List of Tables Foreword Introduction Robin P. Fawcett, M.A.K. Halliday, Sydney M. Lamb and Adam Makkai 1 Language as Code and Language as Behaviour: A Systemic-Functional Interpretation of the Nature and Ontogenesis of Dialogue M.A.K. Halliday 2 Metaphors of Information John Regan 3 How Universal is a Localist Hypothesis? A Linguistic Contribution to the Study of 'Semantic Styles' of Language Yoshihiko Ikegami 4 Some Speculations on Language Contact in a Wider Setting Jeffrey Ellis 5 Ways of Saying: Ways of Meaning Ruqaiya Hasan Index

Language as social semiotic

The complete title of this book runs ‘Analyzing Linguistic Data: A Practical Introduction to Statistics using R’ and as such it very well reflects the purpose and spirit of the book. The author guides the reader in about 350 pages from descriptive and basic statistical methods over classification and clustering to (generalised) linear and mixed models. Each of the methods is introduced in the context of concrete linguistic problems and demonstrated on exciting datasets from current research in the language sciences. In line with its practical orientation, the book focuses primarily on using the methods and interpreting the results. This implies that the mathematical treatment of the techniques is held at a minimum if not absent from the book. In return, the reader is provided with very detailed explanations on how to conduct the analyses using R [1]. The first chapter sets the tone being a 20-page introduction to R. For this and all subsequent chapters, the R code is intertwined with the chapter text and the datasets and functions used are conveniently packaged in the languageR package that is available on the Comprehensive R Archive Network (CRAN). With this approach, the author has done an excellent job in enabling researchers and students alike to reproduce the analyses and learn by doing. Another quality as a textbook is the fact that every chapter ends with Workbook sections where the user is invited to exercise his or her analysis skills on supplemental datasets. Full solutions including code, results and comments are given in Appendix A (30 pages). Instructors are therefore very well served by this text, although they might want to balance the book with some more mathematical treatment depending on the target audience. After the introductory chapter on R, the book opens on graphical data exploration. Chapter 3 treats probability distributions and common sampling distributions. Under basic statistical methods (Chapter 4), distribution tests and tests on means and variances are covered. Chapter 5 deals with clustering and classification. Strangely enough, the clustering section has material on PCA, factor analysis, correspondence analysis and includes only one subsection on clustering, devoted notably to hierarchical partitioning methods. The classification part deals with decision trees, discriminant analysis and support vector machines. The regression chapter (Chapter 6) treats linear models, generalised linear models, piecewise linear models and a substantial section on models for lexical richness. The final chapter on mixed models is particularly interesting as it is one of the few text book accounts that introduce the reader to using the (innovative) lme4 package of Douglas Bates which implements linear mixed-effects models. Moreover, the case studies included in this

Analyzing linguistic data: a practical introduction to statistics using R

Emotion recognition in human-computer interaction

We present the MSP-IMPROV corpus, a multimodal emotional database, where the goal is to have control over lexical content and emotion while also promoting naturalness in the recordings. Studies on emotion perception often require stimuli with fixed lexical content, but that convey different emotions. These stimuli can also serve as an instrument to understand how emotion modulates speech at the phoneme level, in a manner that controls for coarticulation. Such audiovisual data are not easily available from natural recordings. A common solution is to record actors reading sentences that portray different emotions, which may not produce natural behaviors. We propose an alternative approach in which we define hypothetical scenarios for each sentence that are carefully designed to elicit a particular emotion. Two actors improvise these emotion-specific situations, leading them to utter contextualized, non-read renditions of sentences that have fixed lexical content and convey different emotions. We describe the context in which this corpus was recorded, the key features of the corpus, the areas in which this corpus can be useful, and the emotional content of the recordings. The paper also provides the performance for speech and facial emotion classifiers. The analysis brings novel classification evaluations where we study the performance in terms of inter-evaluator agreement and naturalness perception, leveraging the large size of the audiovisual database.

MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception

The Transregional Collaborative Research Centre SFB/TRR 62 "A Companion Technology for Cognitive Technical Systems", funded by the German Research Foundation (DFG) at Ulm and Magdeburg sites, deals with the systematic and interdisciplinary study of cognitive abilities and their implementation in technical systems. The properties of multimodality, individuality, adaptability, availability, cooperativeness and trustworthiness are at the focus of the investigation. These characteristics show a new type of interactive device which is not only practical and efficient to operate, but as well agreeable, hence the term "companion". The realisation of such a technology is supported by technical advancement as well as by neurobiological findings. Companion technology has to consider the entire situation of the user, machine, environment and (if applicable) other people or third interacting parties, in current and historical states. This will reflect the mental state of the user, his embeddedness in the task, and how he is situated in the current process.

A companion technology for cognitive technical systems

We report about design and characteristics of the LAST MINUTE corpus. The recordings in this data collection are taken from a WOZ experiment that allows to investigate how users interact with a companion system in a mundane situation with the need for planning, re-planning and strategy change. The resulting corpus is distinguished with respect to aspects of size (e.g. number of subjects, length of sessions, number of channels, total length of records) as well as quality (e.g. balancedness of cohort, well designed scenario, standard based transcripts, psychological questionnaires, accompanying in-depth interviews).

/pdf/last-minute-a-multimodal-corpus-of-speech-based-user-2kqgg2o4fz.pdf

LAST MINUTE: a Multimodal Corpus of Speech-based User-Companion Interactions

The LAST MINUTE corpus comprises multimodal recordings (e.g. video, audio, transcripts) from WOZ interactions in a mundane planning task (Rosner et al., 2011). It is one of the largest corpora with naturalistic data currently available. In this paper we report about first results from attempts to automatically and manually analyze the different modes with respect to emotions and affects exhibited by the subjects. We describe and discuss difficulties encountered due to the strong contrast between the naturalistic recordings and traditional databases with acted emotions.

/pdf/towards-emotion-and-affect-detection-in-the-multimodal-last-2b0xif79ps.pdf

Towards Emotion and Affect Detection in the Multimodal LAST MINUTE Corpus

This paper addresses issues of automatically detecting significant dialog events (SDEs) in naturalistic HCI, and of deducing trait-specific conclusions relevant for the design of spoken dialog systems. We perform our investigations on the multimodal LAST MINUTE corpus with records from naturalistic interactions. First, we used textual transcripts to analyse interaction styles and discourse structures. We found indications that younger subjects prefer a more technical style in communication with dialog systems. Next, we model the subject’s internal success state with a hidden Markov model trained using the observed sequences of system feedback. This reveals that younger subjects interact significantly more successful with technical systems. Aiming on automatic detection of specific subjects’s reactions, we then semi-automatically annotate SDEs—phrases indicating an irregular, i.e. not-task-oriented subject behavior. We use both acoustic and linguistic features to build several trait-specific classifiers for dialog phases, which showed pronouncedly different accuracies for diverse age and gender groups. The presented investigations coherently support age-dependence of both expressiveness and problem-solving ability. This in turn induces design rules for future automatic designated “companion” systems.

Analysis of significant dialog events in realistic human–computer interaction

We report about a WOZ experiment with a carefully designed scenario that allows to investigate how users interact with a companion system in a mundane situation with the need for planning, re-planning and strategy change. The data collection from the experiments comprises multimodal records (audio, video, biopsychological parameters) and transcripts of the verbal interaction, and all subjects fill out a battery of well established psychometric questionnaires about various aspects especially of their personality. This will allow to correlate observed behaviour and detected affects and emotions with measured aspects of the personality of subjects and is expected to serve as a basis for defining a typology of users. In addition, a subgroup of the subjects takes part in semiformal in-depth interviews that focus on retrospective reflexion of the users' subjective experience during the experiments and especially on the intentionality that users ascribed to the system during the course of interaction.

/pdf/intentionality-in-interacting-with-companion-systems-an-3ii3m61urt.pdf

Intentionality in interacting with companion systems: an empirical approach

Wizard of Oz (WOZ) systems and WOZ experiments are an important tool for basic and applied research in HCI. We report about using SEMAINE as a flexible component based middleware with a loose coupling of components as software infrastructure for WOZ experiments in human companion interaction. We focus on our experimental WOZ designs, their realisation within the SEMAINE framework and lessons learned from deploying the implemented solutions as the basis for ongoing controlled experiments with 120 subjects.

Rafael Friesen

Papers

LAST MINUTE: a Multimodal Corpus of Speech-based User-Companion Interactions

Towards Emotion and Affect Detection in the Multimodal LAST MINUTE Corpus

Analysis of significant dialog events in realistic human–computer interaction

Intentionality in interacting with companion systems: an empirical approach

Message oriented middleware for flexible wizard of Oz experiments in HCI