Florian Hahn

Bio: Florian Hahn is an academic researcher from Bielefeld University. The author has contributed to research in topics: Gesture & Gesture recognition. The author has an hindex of 5, co-authored 15 publications receiving 208 citations.

Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications

Abstract: Communicating face-to-face, interlocutors frequently produce multimodal meaning packages consisting of speech and accompanying gestures. We discuss a systematically annotated speech and gesture corpus consisting of 25 route-and-landmark-description dialogues, the Bielefeld Speech and Gesture Alignment corpus (SaGA), collected in experimental face-to-face settings. We first describe the primary and secondary data of the corpus and its reliability assessment. Then we go into some of the projects carried out using SaGA demonstrating the wide range of its usability: on the empirical side, there is work on gesture typology, individual and contextual parameters influencing gesture production and gestures’ functions for dialogue structure. Speech-gesture interfaces have been established extending unification-based grammars. In addition, the development of a computational model of speech-gesture alignment and its implementation constitutes a research line we focus on.

Gestures Supporting Dialogue Structure and Interaction in the Bielefeld Speech and Gesture Alignment Corpus (SaGA)

Abstract: We report about gestures supporting dialogue structure and interaction in the Bielefeld Speech and Gesture Alignment corpus and provide a first classification of them based on Hahn and Rieser (2009-11). Numbers will be given on the poster.

The Bielefeld Speech and Gesture Alignment Corpus (SaGA)

Abstract: People communicate multimodally. Most prominently, they co-produce speech and gesture. How do they do that? Studying the interplay of both modalities has to be informed by empirically observed communication behavior. We present a corpus built of speech and gesture data gained in a controlled study. We describe 1) the setting underlying the data; 2) annotation of the data; 3) reliability evalution methods and results; and 4) applications of the corpus in the research domain of speech and gesture alignment.

Explaining Speech Gesture Alignment in MM Dialogue Using Gesture Typology

Abstract: The paper discusses how a gesture typology can be extracted form the Bielefeld Speech-And-GestureAlignment corpus (SAGA) making use of the annotated gesture morphology in the SAGA data. The SAGA corpus is briefly characterized. Using a portion of a MM dialogue, the interface between speech and gesture is shown focussing on the impact of gestures on lexical definitions. The interface demonstrates the need for working out types of gestures and specifying their semantics via a partial ontology. This is started setting up a “typological grid” for 400 gestures. It yields a hierarchy of n-dimensional single gestures and composites of them. A statistical analysis of the grid results is provided and evaluated with respect to the whole SAGA corpus. Finally, it is shown how the typological results are used in the specification of the speech-gesture interfaces for the MM dialogue.

Gesture Semantics Reconstruction Based on Motion Capturing and Complex Event Processing: a Circular Shape Example

Abstract: A fundamental problem in manual based gesture semantics reconstruction is the specification of preferred semantic concepts for gesture trajectories. This issue is complicated by problems human raters have annotating fast-paced three dimensional trajectories. Based on a detailed example of a gesticulated circular trajectory, we present a data-driven approach that covers parts of the semantic reconstruction by making use of motion capturing (mocap) technology. In our FA 3 ME framework we use a complex event processing approach to analyse and annotate multi-modal events. This framework provides grounds for a detailed description of how to get at the semantic concept of circularity observed in the data.

打磨Using Language,倡导新理念

Abstract: 人教版高中英语新课程教材中,语言运用（Using Language）是每个单元必不可少的部分,提供了围绕单元中心话题的听、说、读、写的综合性练习,是单元中心话题的延续和升华.如何设计Using Language部分的教学,使自己的教学模式既不落俗套,又能真正体现新课程标准所倡导的教学理念,正是广大一线英语教师一直努力探索的问题.

Hand and Mind: What Gestures Reveal about Thought

Abstract: Hand and Mind: What Gestures Reveal about Thought. David McNeill. Chicago and London: University of Chicago Press, 1992. 416 pp.

A Theory of Semiotics

Abstract: Foreword Note on graphic conventions 0. Introduction-Toward a Logic of Culture 0.1. Design for a semiotic theory 0.2. 'Semiotics': field or discipline? 0.3. Communication and/or signification 0.4. Political boundaries: the field 0.5. Natural boundaries: two definitions of semiotics 0.6. Natural boundaries: inference and signification 0.7. Natural boundaries the lower threshold 0.8. Natural boundaries: the upper threshold 0.9. Epistemological boundaries 1. Signification and Communication 1.1. An elementary communicational model 1.2. Systems and codes 1.3. The s-code as structure 1.4. Information, communication, signification 2. Theory of Codes 2.1. The sign-function 2.2. Expression and content 2.3. Denotation and connotation 2.4. Message and text 2.5 Content and referent 2.6. Meaning as cultural unit 2.7. The interpretant 2.8. The semantic system 2.9. The semantic markers and the sememe 2.10. The KF model 2.11. A revised semantic model 2.12. The model \"Q\" 2.13. The format of the semantic space 2.14. Overcoding and undercoding 2.15. The interplay of codes and the message as an open form 3. Theory of Sign Production 3.1. A general survey 3.2. Semiotic and factual statements 3.3. Mentioning 3.4 The prolem of a typology of signs 3.5. Critique of iconism 3.6. A typology of modes of production 3.7. The aesthetic text as invention 3.8. The rhetorical labor 3.9. Ideological code switching 4. The Subject of Semiotics References Index of authors Index of subjects

D64: a corpus of richly recorded conversational interaction

Abstract: In recent years there has been a substantial debate about the need for increasingly spontaneous, conversational corpora of spoken interaction that are not controlled or task directed. In parallel the need has arisen for the recording of multi-modal corpora which are not restricted to the audio domain alone. With a corpus that would fulfill both needs, it would be possible to investigate the natural coupling, not only in turn-taking and voice, but also in the movement of participants. In the following paper we describe the design and recording of such a corpus and we provide some illustrative examples of how such a corpus might be exploited in the study of dynamic interaction. The D64 corpus is a multimodal corpus recorded over two successive days. Each day resulted in approximately 4 h of recordings. In total five participants took part in the recordings of whom two participants were female and three were male. Seven video cameras were used of which at least one was trained on each participant. The Optitrack motion capture kit was used in order to enrich information. The D64 corpus comprises annotations on conversational involvement, speech activity and pauses as well as information of the average degree of change in the movement of participants.