Making a thinking-talking head

TLDR: The Thinking-Talking Head is described, an interdisciplinary project that sits between and draws upon engineering/computer science and behavioural/cognitive science; research and performance; implementation and evaluation; and how these head functions will be tuned and evaluated using various paradigms, including an imitation paradigm.

Abstract: This paper describes the Thinking-Talking Head; an interdisciplinary project that sits between and draws upon engineering/computer science and behavioural/cognitive science; research and performance; implementation and evaluation. The project involves collaboration between computer scientists, engineers, language technologists and cognitive scientists, and its aim is twofold (a) to create a 3-D computer animation of a human head that will interact in real time with human agents, and (b) to serve as a research platform to drive research in the contributing disciplines, and in talking head research in general. The thinkingtalking head will emulate elements of face-to-face conversation through speech (including intonation), gaze and gesture. So it must have an active sensorium that accurately reflects the properties of its immediate environment, and must be able to generate appropriate communicative signals to feedback to the interlocutor. Here we describe the current implementation and outline how we are tackling issues concerning both the outputs (synthetic voice, visual speech, facial expressiveness and naturalness) from and inputs (auditory-visual speech recognition, emotion recognition, auditory-visual speaker localization) to the head. We describe how these head functions will be tuned and evaluated using various paradigms, including an imitation paradigm.