IEEE Transactions on Pattern Analysis and Machine Intelligence

This paper proposes a concept of lip motion password (simply called lip-password hereinafter), which is composed of a password embedded in the lip movement and the underlying characteristic of lip motion. It provides a double security to a visual speaker verification system, where the speaker is verified by both of the private password information and the underlying behavioral biometrics of lip motions simultaneously. Accordingly, the target speaker saying the wrong password or an impostor who knows the correct password will be detected and rejected. To this end, we shall present a multi-boosted Hidden Markov model (HMM) learning approach to such a system. Initially, we extract a group of representative visual features to characterize each lip frame. Then, an effective lip motion segmentation algorithm is addressed to segment the lip-password sequence into a small set of distinguishable subunits. Subsequently, we integrate HMMs with boosting learning framework associated with a random subspace method and data sharing scheme to formulate a precise decision boundary for these subunits verification, featuring on high discrimination power. Finally, the lip-password, whether spoken by the target speaker with the pre-registered password or not, is identified based on all the subunit verification results learned from multi-boosted HMMs. The experimental results show that the proposed approach performs favorably compared with the state-of-the-art methods.

/pdf/learning-multi-boosted-hmms-for-lip-password-based-speaker-49dvbvjf6o.pdf

Learning Multi-Boosted HMMs for Lip-Password Based Speaker Verification

PART 1: THE ORIGINS AND DEVELOPMENT OF GEOMETRICAL KNOWLEDGE PART 2: ABSTRACT PROJECTIVE GEOMETRY

Algebraic Projective Geometry. By J. G. Semple and G. T. Kneebone. Pp. viii, 404. 35s. 1952. (Oxford University Press)

https://www.csc.kth.se/~hedvig/publications/specom_09.pdf

Audiovisual-to-articulatory inversion

This paper presents a visual passwords system to increase security. The system depends mainly on recognizing the speaker using the visual speech signal alone. The proposed scheme works in two stages: setting the visual password stage and the verification stage. At the setting stage the visual passwords system request the user to utter a selected password, a video recording of the user face is captured, and processed by a special words-based VSR system which extracts a sequence of feature vectors. In the verification stage, the same procedure is executed, the features will be sent to be compared with the stored visual password . The proposed scheme has been evaluated using a video database of 20 different speakers (10 females and 10 males), and 15 more males in another video database with different experiment sets. The evaluation has proved the system feasibility, with average error rate in the range of 7.63% to 20.51% at the worst tested scenario, and therefore, has potential to be a practical approach with the support of other conventional authentication methods such as the use of usernames and passwords.

Visual Passwords Using Automatic Lip Reading

We present a model for the lips using the active shape model (ASM) based on Bezier curves which can capture the dynamics of the lips efficiently. The curves are defined only by a few points that can be used as visual speech features. Using some measured feature points, we can recover the full model of the lips especially for those features which are difficult to be detected. Accurate detection of features of the lips is implemented using multiple independent feature templates.

Active shape lip modeling

In this paper we present an approach for lip motion analysis that can be used in conjunction with a person authentication system based on face recognition, to avoid attacks on the system using passive photographs. This work focuses on robustly tracking lips in gray scale images, which may be captured in the visible light or near infrared spectrum. We present an approach for locating the two lip corners in a face image. Then we extract suitable features from the mouth region to classify mouth states (visemes). The system shows a classification accuracy of above 85%. The temporal changes in the detected viseme classes can be used for detecting the imposter.

Real Time Lip Motion Analysis for a Person Authentication System using Near Infrared Illumination

In this work, a system of audio visual speech recognition will be presented. A new hybrid visual feature combination, which is suitable for audio -visual speech recognition was implemented. The features comprise both the shape and the appearance of lips, the dimensional reduction is applied using discrete cosine transform (DCT). A large visual speech database of the German language has been assembled, the German Audio -Visual Database (GAVD). The conducted experiments using only visual features resulted in a high recognition accuracy and improved the audio-visual speech recognition drastically.

A system for audio-visual speech recognition.

In this paper, we present our approach of face and lip detection, lip modeling, and tracking. A new lip model based on Bézier Curves is used to capture the dynamics of the lips efficiently. The model is defined only through few points which are modeled using the Active Shape Model (ASM). Accurate detection of lip details is implemented using multiple independent feature templates. The method detects tracks and model the lips online and robustly. The results of the lip detection are presented based on our collected data for German language. We also describe also some of the current and future work on audio video integration.

A System for Automatic Lip Reading.

In this paper, we present a technique for the extraction of the five main visemes produced in natural speech for German. The method belongs to the LDA (Linear Discriminant Analysis) family. The intensity, the edges, and the line segments are used to locate the lips automatically and for viseme classification. Using many features in the recognition maximizes the probability of recognition rate. The corners of the mouth are used in case of small rotation and scale. An experiment has been carried out on different people, to understand the part of the speech that the human being use. The people grouped the phonemes into five different visemes. The number of distinguished visemes is not the same for each speaker. Everyone express the speech in a different visemes. Good recognition rate has been achieved on different speaker.

Islam Shdaifat

Papers

Active shape lip modeling

Real Time Lip Motion Analysis for a Person Authentication System using Near Infrared Illumination

A system for audio-visual speech recognition.

A System for Automatic Lip Reading.

Viseme recognition using multiple feature matching.