This book is an edited collection of chapters on artificial companions (ACs) resulting from a workshop. It purports to discuss the philosophical and ethical issues associated with ACs, what ACs should be like and how to construct them, and to provide examples of special-purpose ACs. Table 1 shows the chapters of the book and their respective authors. When I bought a vacuum-cleaning robot called Roomba, it never occurred to me that this was actually a companion, as claimed by Peltu and Wilks in this book’s afterword. Yes, it independently helps me, autonomously recharges its battery, and occasionally talks: “Clean brushes.” I had higher expectations of a companion, however. Perhaps it would feel more like a companion if it expressed disgust with my mess (cf. Bee et al.), inquired knowledgeably about my holidays and family (cf. Wilks), empathized with my feelings (cf. Bevacqua et al.), and advised scraping off spilled porridge with a spoon (cf. Sloman). There is no consensus in this book on what an AC should be like; each chapter tends to express an alternative (often quite radically different) opinion. In the foreword, Wilks defines ACs as “conversationalists or confidants” that get to know their owners, assist with Internet interactions, provide company and companionship, and build their owner’s biography. However, when analyzing the necessary conditions for being an AC, Pulman concludes that conversation is not needed, and Boden strongly objects to ACs as confidants because of privacy concerns. Rather than providing company, according to O’Hara, ACs will represent (technophobe) owners in complex dealings with technology. Lowe takes this one step further and questions whether ACs are really others or whether they are cooler-headed versions of ourselves. Whereas Romano defines an AC as a good friend who, among other things, makes you laugh, shares your emotions, and listens, Sloman focuses on the (in his opinion) more difficult problem of creating ACs that provide help and assistance. Turkle sees ACs as offering care as well as being good company, teachers, and lovers (!). Taylor et al. deviate from the anthropomorphic view of ACs, and discuss a robot that has no human-like properties other than that it is fed on organic material.

Book review: close engagements with artificial companions: Key social, psychological, ethical, and design issues yorick wilks (editor) (university of oxford) amsterdam: john benjamins publishing company (natural language processing series, edited by ruslan mitkov, volume 8), 2010, xxii+315 pp; hardbound, isbn 978-90-272-4994-4, $149.00, €99.00; e-book, isbn 978-90-272-8840-0, $149.00, €99.00

Natural language processing of conversational speech requires the availability of high-quality transcripts. In this paper, we express our skepticism towards the recent reports of very low Word Error Rates (WERs) achieved by modern Automatic Speech Recognition (ASR) systems on benchmark datasets. We outline several problems with popular benchmarks and compare three state-of-the-art commercial ASR systems on an internal dataset of real-life spontaneous human conversations and HUB’05 public benchmark. We show that WERs are significantly higher than the best reported results. We formulate a set of guidelines which may aid in the creation of real-life, multi-domain datasets with high quality annotations for training and testing of robust ASR systems.

/pdf/wer-we-are-and-wer-we-think-we-are-4z5rt7kt1n.pdf

WER we are and WER we think we are

We report about evaluations of the LAST MINUTE corpus which comprises multimodal recordings (audio, video, biopsychological data, verbatim transcripts, psychological questionnaires, in-depth user interviews) of Wizard of Oz simulated naturalistic human companion interactions in German. Based on the results of these analyses we discuss consequences for the design of future companion systems.

Desiderata for the Design of Companion Systems

Acoustic-phonetic properties of Siri- and human-directed speech

When our task is to detect social signals such as laughter and filler events in an audio recording, the most straightforward way is to apply a Hidden Markov Model–or a Hidden Markov Model/Deep Neural Network (HMM/DNN) hybrid, which is considered state-of-the-art nowadays In this hybrid model, the DNN component is trained on frame-level samples of the classes we are looking for In such event detection tasks, however, the training labels are seriously imbalanced, as typically only a small fraction of the training data corresponds to these social signals, while the bulk of the utterances consists of speech segments or silence A strong imbalance of the training classes is known to cause difficulties during DNN training To alleviate these problems, here we apply the technique called probabilistic sampling, which seeks to balance the class distribution Probabilistic sampling is a mathematically well-founded combination of upsampling and downsampling, which was found to outperform both of these simple resampling approaches With this strategy, we managed to achieve a 7–8 percent relative error reduction both at the segment level and frame level, and we efficiently reduced the DNN training times as well

/pdf/social-signal-detection-by-probabilistic-sampling-dnn-4bfh0ri16q.pdf

Social Signal Detection by Probabilistic Sampling DNN Training

Nowadays a diverse set of technical solutions is implemented to detect if a system should react to an uttered speech command. Unfortunately, the preferred methods of wake words can result in confusions e.g. when the word has been said but no interaction with the system was intended by the user. Therefore, technical systems should be able to detect their addressing by itself. In order to achieve this goal research concentrates on analyzing speech. Analysing the speaker’s self-assessment of his speech characteristics while addressing a system can provide further information, which up to now wasn’t considered in the field. Utilizing a new generated voice assistant conversation corpus, this paper presents insights of the participant’s addressee behavior and correlates objective and subjective changes in speaking style characteristics between human-human and human-computer conversations. It could be shown that users could recognize changes in some of their speech characteristics. Furthermore, the objective identiﬁable changes are heavily dependent on the type of interaction. Mostly affected are intonation and stress patterns as well as melody and rhythm patterns. The presence of a confederate speaker does not reveal differences on the addressing behavior.

How do we speak with ALEXA: Subjective and objective assessments of changes in speaking style between HC and HH conversations

Whether or not a Companion-system is experienced as a confidant and empathic assistant d epends on users’ individual ascriptions to it. An ascription-based understanding of users’ experiences in user-compani on interaction is proposed, which focuses primarily on ascriptions of human-like characteristics, intentions, motivati ons or emotions to the system. It is examined with regard to the individualization-focused interactions pivotally required for the adaptation of a Companion-system to the user. T he study is based on a Wizard of Oz experiment and subsequently conducted in-depth user interviews. By applying qua litative content analysis, four categories describing dimens ions of users’ ascriptions to the system were worked out (n ature, capabilities, requirements, and relational offer) as well as two categories describing users’ reactions based on the se (adaptation work and self-disclosing behavior). The findings are discussed with regard to theories of psychology , philosophy and human-robot interaction. Moreover, two needs inherent in humans, which seem to be relevant for users’ ascrip tions in user-companion interaction, are referred to: the ned for safety and the need to belong. Keywords-users’ ascriptions; user experience; Compa nionsystems; individualization; human-computer interact ion.

/pdf/making-the-system-a-relational-partner-users-ascriptions-in-6hrx34d5rr.pdf

Making the System a Relational Partner: Users’ Ascriptions in Individualization-focused Interactions with Companion-systems

Burnette et al. reported a study that they sought to undertake to validate common eating disorder questionnaires in sexual and gender minorities. The researchers took advantage of the online recruitment platform Amazon Mechanical Turk (MTurk). Contrary to their expectations, the study proved not feasible due to invalid answering. Thus, Burnette et al. raise concerns against the trustworthiness of crowd-sourced data that may be undermined by financial interests and other kinds of motivations. Our commentary highlights the potential of the COVID-19 pandemic to inflate especially those intentions, which are monetary. Against the background of the COVID-19 pandemic, a further problem seems to be that the anonymity of online crowd sourcing platforms might tempt participants to provide inconsistent answers, possibly reflecting tendencies of reactance. The reported pattern of paradoxical responses in Burnette et al.'s work does not reflect malingering; rather we believe that the study might have served some participants as an outlet for negative emotions. We discuss mechanisms of quality control and highlight the lack of interpersonal interaction associated with online data collections.

Eating disorder related research using Amazon Mechanical Turk (MTurk): Friend or foe?: Commentary on Burnette et al. (2021).

Nowadays, a diverse set of addressee detection methods is discussed. Typically, wake words are used. But these force an unnatural interaction and are error-prone, especially in case of false positive classification (user says the wake up word without intending to interact with the device). Therefore, technical systems should be enabled to perform a detection of device directed speech. In order to enrich research in the field of speech analysis in HCI we conducted studies with a commercial voice assistant, Amazon’s ALEXA (Voice Assistant Conversation Corpus, VACC), and complemented objective speech analysis with subjective self and external reports on possible differences in speaking with the voice assistant compared to speaking with another person. The analysis revealed a set of specific features for device directed speech. It can be concluded that speech-based addressing of a technical system is a mainly conscious process including individual modifications of the speaking style.

Julia Krüger

Papers

How do we speak with ALEXA: Subjective and objective assessments of changes in speaking style between HC and HH conversations

Desiderata for the Design of Companion Systems

Making the System a Relational Partner: Users’ Ascriptions in Individualization-focused Interactions with Companion-systems

Eating disorder related research using Amazon Mechanical Turk (MTurk): Friend or foe?: Commentary on Burnette et al. (2021).

“Speech Melody and Speech Content Didn’t Fit Together”—Differences in Speech Behavior for Device Directed and Human Directed Interactions