Patent
Methods and systems for synthesis of accurate visible speech via transformation of motion capture data
Reads0
Chats0
TLDR
In this article, a sequence of visemes, each associated with one or more phonemes are mapped onto a 3D target face, and concatentated with motion trajectories of a set facial points.Abstract:
The disclosure describes methods for synthesis of accurate visible speech using transformations of motion-capture data. Methods are provided for synthesis of visible speech in a three-dimensional face. A sequence of visemes, each associated with one or more phonemes, are mapped onto a three-dimensional target face, and concatentated. The sequence may include divisemes corresponding to pairwise sequences of phonemes, wherein the diviseme is comprised of motion trajectories of a set facial points. The sequence may also include multi-units corresponding to words and sequences of words. Various techniques involving mapping and concatenation are also addressed.read more
Citations
More filters
Patent
Real-time Animation for an Expressive Avatar
TL;DR: In this article, a process for providing real-time animation for a personalized cartoon avatar based on speech and motion data has been described, where the process links one or more predetermined phrases that represent emotional states to the one-or more animated models.
Patent
Rendered audiovisual communication
TL;DR: In this article, a model can be generated from the image information, and the model may be used to render audiovisual communication information from image and audio captured in real time.
Patent
Apparatus control based on visual lip share recognition
TL;DR: In this paper, an information processing apparatus that includes an image acquisition unit to acquire a temporal sequence of frames of image data, a detecting unit to detect a lip area and a lip image from each of the frames of the image data and a recognition unit to recognize a word based on the detected lip images of the lip areas, and a controller to control an operation at the information processing device based on a word recognized by the recognition unit is described.
Patent
Method and apparatus for providing natural facial animation
TL;DR: In this paper, an inter-viseme animation of 3D head model driven by speech recognition is calculated by applying limitations to the velocity and acceleration of a normalized parameter vector, each element of which may be mapped to animation node outputs of a 3D model based on mesh blending and weighted by a mix of key frames.
Patent
Using emoticons for contextual text-to-speech expressivity
TL;DR: This paper used emoticons identified from a source text to provide contextual text-to-speech expressivity, such as intonation, prosody, speed, pauses, and other expressivity characteristics.
References
More filters
PatentDOI
Method and apparatus for audio-visual speech detection and recognition
TL;DR: In this article, the authors propose a speech recognition technique for video and audio signals that consists of processing a video signal associated with an arbitrary content video source, processing an audio signal associated to the video signal, and recognizing at least a portion of the processed audio signal using at least the processed video signal to generate output signal representative of the audio signal.
Patent
Method and system for capturing and representing 3D geometry, color and shading of facial expressions and other animated objects
TL;DR: In this paper, a 3D model of a face and a series of deformations of the mesh are used to track motion of the face over time and establish a relationship between the model and texture.
Proceedings ArticleDOI
Translingual visual speech synthesis
TL;DR: This work presents a novel scheme to implement a language independent system for audio-driven facial animation given a speech recognition system for just one language, in this case, English.
Patent
Methods and devices for producing and using synthetic visual speech based on natural coarticulation
TL;DR: In this paper, a method of producing synthetic visual speech according to this invention includes receiving an input containing speech information, one or more visemes that correspond to the speech input are then identified.
Patent
Talking facial display method and apparatus
Tomaso Poggio,Antoine F. Ezzat +1 more
TL;DR: In this paper, a method and apparatus of converting input text into an audio-visual speech stream resulting in a talking face image enunciating the text is presented, which is then displayed in real time, thereby displaying photo-realistic talking face.