A German viseme-set for automatic transcription of input text used for audio-visual speech synthesis.

TLDR: A German viseme inventory for visemically transcribing text according to phonetic transcribtion is introduced and an inventory of German visemo classes in a SAMPA-like labelling is worked out and a model for automatic visemic transcription of given input text is trained.

Abstract: In this paper, we introduce a German viseme inventory for visemically transcribing text according to phonetic transcribtion. A viseme set like the one presented in this work is essential for speech-driven audio-visual synthesis due to the fact that the selection of appropriate video segments is based on the visemically transcribed input text. For text-to-speech synthesis, a transcription of the input text into the phonemic representation is used, in order to avoid ambiguous meanings and to acquire the correct pronunciation of the underlying input text and to serve as labels in unitselection-based synthesis systems. Likewise, the visual synthesis requires a transcription that represents analogue to the phonemes the visual counterpart which is called viseme in related literature and which also serves as a unit label in our data-driven video-realistic audio-visual synthesis system. We worked out an inventory of German viseme classes in a SAMPA-like labelling and trained a model for automatic visemic transcription of given input text.