Viseme

About: Viseme is a research topic. Over the lifetime, 865 publications have been published within this topic receiving 17889 citations.

Development of a Speech Training System by Lip Movements

Abstract: Speech recognition technology is spreading with personal digital assistants such as smart phones However, we are concerned about the decline in the recognition rate at places with multiple voices and considerable noise Therefore, we have been studying a lip operation that would recognize the content of an utterance by reading from an image Based on this research, we created a database of utterances by Japanese television announcers and English teachers for utterance training in Japanese and English Furthermore, applying the technology we developed, we propose a method of utterance training using specific equipment First, we compared the student's utterance with data in the lip movement database Second, we evaluated the effectiveness of the utterance training equipment

An image-based instant message system and method for providing emotions expression

Abstract: A method of providing an image of emotions animation based on a text message, comprising: generating a phoneme event, a wave event or an index event based on the typed text message utilizing a text-to-speech engine; mapping a current phoneme stored via said phoneme event into a viseme according to a mapping table from phoneme to viseme; calculating the needed number of face/lip frames based on the length of current phoneme's wave data stored via said wave event; retrieving said needed number of face/lip frames from a model file based said viseme for output.

A Visual Speech Feature to Indentify the Speaking States from Video

Abstract: The paper proposes a kind of visual speech feature for the speaking mouth images from the video combining clues of the shape and local teeth texture. The geometric feature we proposed based on the computing the Euclidian distant between each the feature point around the inner and outer lip. The local texture with G and B components as baseline is employed to calculate the color moment to describe the visibility of teeth. The weighted fusion is used to combine the two features. The k-mean algorithm is utilized to analyze the feature performance according to evaluate the clustering results. The results show that with G and B color component to derive the local texture to model the teeth visibility are better than the others and our feature has higher ability to perceive the visemes than the PCA and geometric feature only.

Hidden markov models for visual speech synthesis with limited data

Abstract: This paper addresses a problem often encountered when estimating control points used in visual speech synthesis. First, Hidden Markov Models (HMMs) are estimated for each viseme present in stored video data. Second, models are generated for each triseme (a viseme in context with the previous and following visemes) in the training set. Next, a decision tree is used to cluster and relate states in the HMMs that are similar in a contextual and statistical sense. The tree is also used to estimate HMMs for any trisemes that are not present in the stored video data when control points for such trisemes are required for synthesizing the lip motion for a sentence. Finally, the HMMs are used to generate sequences of visual speech control points for those trisemes not occurring in the stored data. Comparisons of mouth shapes generated from the artificially generated control points and the control points estimated from video not used to train the HMMs indicate that the process estimated accurate control points for the trisemes tested. This paper thus establishes a useful method for synthesizing realistic audio-synchronized video facial features.