Viseme

About: Viseme is a research topic. Over the lifetime, 865 publications have been published within this topic receiving 17889 citations.

Overlapping of Speech Sounds

Abstract: THE piece of sound film reproduced in Fig. 1 shows the last part of the vowel and the first part of the consonant in a registration of ash. The vibratory bits that characterise the vowel can be traced in ever-diminishing strength to the third line in the reproduction. The mixture of regular and irregular vibrations that characterise the consonant can be traced back to the middle of the first line. The end of the vowel and the beginning of the consonant are seen to be overlapped.

Real-time speech driven talking avatar

Abstract: This paper presents a real-time speech driven talking avatar.Unlike most talking avatars in which the speech-synchronized facial animation is generated offline,this talking avatar is able to speak with live speech input.This life-like talking avatar has many potential applications in videophones,virtual conferences,audio/video chats and entertainment.Since phonemes are the smallest units of pronunciation,a real-time phoneme recognizer was built.The synchronization between the input live speech and the facial motion used a phoneme recognition and output algorithm.The coarticulation effects are included in a dynamic viseme generation algorithm to coordinate the facial animation parameters(FAPs) from the input phonemes.The MPEG-4 compliant avatar model is driven by the generated FAPs.Tests show that the avatar motion is synchronized and natural with MOS values of 3.42 and 3.5.

Visual-speech Synthesis of Exaggerated Corrective Feedback

Abstract: To provide more discriminative feedback for the second language (L2) learners to better identify their mispronunciation, we propose a method for exaggerated visual-speech feedback in computer-assisted pronunciation training (CAPT). The speech exaggeration is realized by an emphatic speech generation neural network based on Tacotron, while the visual exaggeration is accomplished by ADC Viseme Blending, namely increasing Amplitude of movement, extending the phone's Duration and enhancing the color Contrast. User studies show that exaggerated feedback outperforms non-exaggerated version on helping learners with pronunciation identification and pronunciation improvement.

Efficient lip-synch tool for 3D cartoon animation

Abstract: We propose a set of algorithms to efficiently make speech animation for 3D cartoon characters. Our prototype system is based on blendshapes, a linear interpolation technique, which is widely used in facial animation practice. In our system, a few base target shapes of the character, prerecorded voice, and its transcription are required as input. We describe a simple technique that amplifies the target shapes from few inputs using a generic database of viseme mouth shapes. We also introduce additional lip-synch editing parameters that allow designers to quickly tune the lip movements. Based on these, we implement our prototype system as a Maya plug-in. The demonstration movies created with this system illustrate well the practicality of our approach. Copyright © 2008 John Wiley & Sons, Ltd.