In this work, a new instantaneous fundamental frequency extraction method is presented, with the attention especially focused on its robustness for pathological voices processing. It is based on the Ensemble Empirical Mode Decomposition (EEMD) algorithm, which is a completely data-driven method for signal decomposition into a sum of AM - FM components, called Intrinsic Mode Functions (IMFs) or modes. Our results show that the speech fundamental frequency can be captured in a single IMF. We also propose an algorithm for selecting the mode where the fundamental frequency can be found, based on the logarithm of the power of the IMFs. The instantaneous frequency is then extracted by means of well-known techniques. The behaviour of the proposed method is compared with other two ones (Robust Algorithm for Pitch Tracking -RAPT- and auto-correlation based algorithms), both in normal and pathological sustained vowels.

/pdf/a-new-algorithm-for-instantaneous-f-0-speech-extraction-if21lghd4s.pdf

A new algorithm for instantaneous F 0 speech extraction based on Ensemble Empirical Mode Decomposition

Empirical mode decomposition (EMD) is an algorithm for signal analysis recently introduced by Huang. It is a completely data-driven non-linear method for the decomposition of a signal into AM - FM components. In this paper two new EMD-based methods for the analysis and classification of pathological voices are presented. They are applied to speech signals corresponding to real and simulated sustained vowels. We first introduce a method that allows the robust extraction of the fundamental frequency of sustained vowels. Its determination is crucial for pathological voice analysis and diagnosis. This new method is based on the ensemble empirical mode decomposition (EEMD) algorithm and its performance is compared with others from the state of the art. As a second EMD-based tool, we explore spectral properties of the intrinsic mode functions and apply them to the classification of normal and pathological sustained vowels. We show that just using a basic pattern classification algorithm, the selected spectral features of only three modes are enough to discriminate between normal and pathological voices.

/pdf/pathological-voice-analysis-and-classification-based-on-3gtxrj4z7a.pdf

Pathological voice analysis and classification based on empirical mode decomposition

Yawn analysis with mouth occlusion detection

An elementary visual unit – the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector construction and an appropriate selection of the classifier were sought. The experimental research was conducted on the basis of a spoken corpus in which speech was represented both acoustically and visually. The extracted features represented three types: geometrical, textural and mixed ones. The features were processed employing the classification algorithms based on Hidden Markov Models and Sequential Minimal Optimization. Tests were carried out employing the processed video material recorded with English native speakers who read specially prepared list of commands. The obtained results are discussed in the paper.

/pdf/a-comparative-study-of-english-viseme-recognition-methods-20r14vlxpx.pdf

A comparative study of English viseme recognition methods and algorithms

Lip segmentation is a fundamental system component in a range of applications including: automatic lip reading, emotion recognition and biometric speaker identification. The first step in lip segmentation involves applying a colour transform to enhance the contrast between the lips and surrounding skin. However, there is much debate among researchers as to the best transform for this task. As such, this article presents the most comprehensive study to date by evaluating 33 colour transforms for lip segmentation: 21 channels from seven colour space models (RGB, HSV, YCbCr, YIQ, CIEXYZ, CIELUV and CIELAB) and 12 additional colour transforms (8 of which are designed specifically for lip segmentation). The colour transform comparison is extended to determine the best transform to segment the oral cavity. Histogram intersection and Otsu’s discriminant are used to quantify and compare the transforms. Results for lip–skin segmentation validate the experimental approach, as 11 of the top 12 transforms are used for lip segmentation in the literature. The necessity of selecting the correct transform is demonstrated by an increase in segmentation accuracy of up to three times. Hue-based transforms including pseudo hue and hue domain filtering perform the best for lip–skin segmentation, with the hue component of HSV achieving the greatest accuracy of 93.85 %. The a* component of CIELAB performs the best for lip–oral cavity segmentation, while pseudo hue and the LUX transform perform reasonably well for both lip–skin segmentation and lip–oral cavity segmentation.

Comparison of colour transforms used in lip segmentation algorithms

Most automatic speech recognition systems have concentrated exclusively on the acoustic speech signal, and therefore they are susceptible to acoustic noise. The benefits from visual speech cues have motivated significant interest in automatic lip-reading, which aims at improving automatic speech recognition by exploiting informative visual features of a speaker's mouth region, which means speaker lip motion stands out as the most linguistically visual feature. In this paper, we present a new improved robust lip location and tracking approach, aims at improving the lip-reading accuracy. Lip regions of interest are detected by a new method, combining with Intel Open source (OpenCV). In this new method, we analyze the distribution relationship between faces, eyes and mouth, and then the mouth region can be easily located. It can be proved as an effective method for lip tracking. In the subsequent step, color space is transferred to Lab from RGB color space, and a component of Lab color space is used for extracting lip segmentation and tracking lip region more accurately and efficiently from video sequences of a speaker's talking face in different lighting conditions, and with different lip shapes and head poses. Extensive experiments show that our proposed method can achieve superior performance to other similar lip tracking approaches, and then can be effectively integrated in lip-reading or visual speech recognition systems.

A real-time lip localization and tacking for lip reading

A modified pitch period detection (MPPD) algorithm is proposed in this paper. It is based on the redefinition of the instantaneous energy density level of Hilbert-Huang transform, called as frequency weighted energy density FIE(t). Two examples are employed to illuminate the physical meaning of this new definition. MPPD are used in detection experiments with 78 normal and 70 abnormal vowel /a:/ voices, the experimental results show that MPPD provides more robust performance and makes better results in normal voice pitch detection

A Novel Pitch Period Detection Algorithm Bases on HHT with Application to Normal and Pathological Voice

This paper presents a method to extract the outer contour of lip based on manifold without lip model. As we all know, the red exclusion and color transform methods are used widely to extract the lip for their simplicity. Lots of modified method and some clustering method are proposed too, but most of these methods are based on the distribution of color difference or correlation between lip and skin. They can only extract the raw lip area not the contour of the lip. The edge of the lip is very important for the correction of extracted features, the quality of these features determine the application related to lip, such as lip reading and speaker identification. Based on the fact that the mouth area has high edge content and the change in lip contour is bigger than in other area, this paper uses the change of color components and based it on the frame of manifold to get the edge of the lip. The experimental results show that the proposed method is more effective in extraction outer contour of lip and more robust than red exclusion.

Lip Contour Extraction Based on Manifold

A cross octagonal search algorithm (COSA) is proposed in this paper to perform block motion estimation in video sequence compression. In contrast with the classical fast block motion algorithm, cross and octagonal search pattern is adopted in search process in this algorithm. Based on the characteristic of motion vector's cross center biased distribution, the correlation of the ending points in the searching pattern can be reduced. So in this paper octagonal pattern is used and cross pattern is used in COSA to reduce the number of searching points. The experimental results show that the cross octagonal search algorithm performs better than HEXBS and hybrid unsymmetrical cross multi-hexagon-grid search (UMHexagonS) which is adopted by joint modal for near four years and still be used in the latest version JM9.6.

Liang Yaling

Papers

A real-time lip localization and tacking for lip reading

A Novel Pitch Period Detection Algorithm Bases on HHT with Application to Normal and Pathological Voice

Lip Contour Extraction Based on Manifold

A cross octagonal search algorithm for fast block motion estimation