Viseme

About: Viseme is a research topic. Over the lifetime, 865 publications have been published within this topic receiving 17889 citations.

A Detailed Description of the AVOZES data corpus

Abstract: The AVOZES data corpus has recently been made publicly available for other interested researchers. It is the first publicly available audio-video speech data corpus for Australian English. It contains recordings from 20 speakers and the sequences provide both a systematic coverage of the phonemes and visemes of Australian English as well as some application-driven utterances. AVOZES is also the first audio-video speech data corpus with stereo-video recordings, which enable a more accurate measurement of geometric facial features.

Lipreading using a comparative machine learning approach

Abstract: Lipreading is the process of interpreting spoken word by observing lip movement. It plays a vital role in human communication and speech understanding, especially for hearing-impaired individuals. Automated lipreading approaches have recently been used in such applications as biometric identification, silent dictation, forensic analysis of surveillance camera capture, and communication with autonomous vehicles. However, lipreading is a difficult process that poses several challenges to human- and machine-based approaches alike. This is due to the large number of phonemes in human language that are visually represented by a smaller number of lip movements (visemes). Consequently, the same viseme may be used to represent several phonemes, which confuses any lipreader. In this paper, we present a detailed study of the machine learning approach for the real-time visual recognition of spoken words. Our focus on real-time performance is motivated by the recent trend of using lipreading in autonomous vehicles. In this paper, machine learning approaches are applied to recognize lip-reading and nine different classifiers has been implemented and tested, reporting their confusion matrices among different groups of words. The classification process went on more than one classifier but these three classifiers got the best results which are GradientBoosting, Support Vector Machine(SVM) and logistic regression with results 64.7%, 63.5% and 59.4% respectively.

Predicting word accuracy for the automatic speech recognition of non-native speech.

Abstract: We have developed an automated method that predicts the word accuracy of a speech recognition system for non-native speech, in the context of speaking proficiency scoring. A model was trained using features based on speech recognizer scores, function word distributions, prosody, background noise, and speaking fluency. Since the method was implemented for non-native speech, fluency features, which have been used for non-native speakers’ proficiency scoring, were implemented along with several feature groups used from past research. The fluency features showed promising performance by themselves, and improved the overall performance in tandem with other more traditional features. A model using stepwise regression achieved a correlation with word accuracy rates of 0.76, compared to a baseline of 0.63 using only confidence scores. A binary classifier for placing utterances in high-or low-word accuracy bins achieved an accuracy of 84%, compared to a majority class baseline of 64%.