Extraction of discriminative features from salient facial patches plays a vital role in effective facial expression recognition. The accurate detection of facial landmarks improves the localization of the salient patches on face images. This paper proposes a novel framework for expression recognition by using appearance features of selected facial patches. A few prominent facial patches, depending on the position of facial landmarks, are extracted which are active during emotion elicitation. These active patches are further processed to obtain the salient patches which contain discriminative features for classification of each pair of expressions, thereby selecting different facial patches as salient for different pair of expression classes. One-against-one classification method is adopted using these features. In addition, an automated learning-free facial landmark detection technique has been proposed, which achieves similar performances as that of other state-of-art landmark detection methods, yet requires significantly less execution time. The proposed method is found to perform well consistently in different resolutions, hence, providing a solution for expression recognition in low resolution images. Experiments on CK+ and JAFFE facial expression databases show the effectiveness of the proposed system.

Automatic Facial Expression Recognition Using Features of Salient Facial Patches

Automatic facial expression recognition has many potential applications in different areas of human computer interaction. However, they are not yet fully realized due to the lack of an effective facial feature descriptor. In this paper, we present a new appearance-based feature descriptor, the local directional pattern (LDP), to represent facial geometry and analyze its performance in expression recognition. An LDP feature is obtained by computing the edge response values in 8 directions at each pixel and encoding them into an 8 bit binary number using the relative strength of these edge responses. The LDP descriptor, a distribution of LDP codes within an image or image patch, is used to describe each expression image. The effectiveness of dimensionality reduction techniques, such as principal component analysis and AdaBoost, is also analyzed in terms of computational cost saving and classification accuracy. Two well-known machine learning methods, template matching and support vector machine, are used for classification using the Cohn-Kanade and Japanese female facial expression databases. Better classification accuracy shows the superiority of LDP descriptor against other appearance-based feature descriptors.

https://onlinelibrary.wiley.com/doi/pdf/10.4218/etrij.10.1510.0132

Robust Facial Expression Recognition Based on Local Directional Pattern

Video-based human activity recognition systems have potential contributions to various applications such as smart homes and healthcare services. In this work, we present a novel depth video-based translation and scaling invariant human activity recognition (HAR) system utilizing R transformation of depth silhouettes. To perform HAR in indoor settings, an invariant HAR method is critical to freely perform activities anywhere in a camera view without translation and scaling problems of human body silhouettes. We obtain such invariant features via R transformation on depth silhouettes. Furthermore, in R transforming depth silhouettes, shape information of human body reflected in depth values is encoded into the features. In R transformation, 2D feature maps are computed first through Radon transform of each depth silhouette followed by computing 1D feature profile through R transform to get the translation and scaling invariant features. Then, we apply Principle Component Analysis (PCA) for dimension reduction and Linear Discriminant Analysis (LDA) to make the features more prominent, compact and robust. Finally, Hidden Markov Models (HMMs) are used to train and recognize different human activities. Our proposed system shows superior recognition rate over the conventional approaches, reaching up to the mean recognition rate of 93.16% for six typical human activities whereas the conventional PC and IC-based depth silhouettes achieved only 74.83% and 86.33% ,while binary silhouettes-based R transformation approach achieved 67.08% respectively. Our experimental results show that the proposed method is robust, reliable, and efficient in recognizing the daily human activities.

Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home

This paper introduces an accurate and robust facial expression recognition (FER) system. For feature extraction, the proposed FER system employs stepwise linear discriminant analysis (SWLDA). SWLDA focuses on selecting the localized features from the expression frames using the partial $\boldsymbol {F}$ -test values, thereby reducing the within class variance and increasing the low between variance among different expression classes. For recognition, the hidden conditional random fields (HCRFs) model is utilized. HCRF is capable of approximating a complex distribution using a mixture of Gaussian density functions. To achieve optimum results, the system employs a hierarchical recognition strategy. Under these settings, expressions are divided into three categories based on parts of the face that contribute most toward an expression. During recognition, at the first level, SWLDA and HCRF are employed to recognize the expression category; whereas, at the second level, the label for the expression within the recognized category is determined using a separate set of SWLDA and HCRF, trained just for that category. In order to validate the system, four publicly available data sets were used, and a total of four experiments were performed. The weighted average recognition rate for the proposed FER approach was 96.37% across the four different data sets, which is a significant improvement in contrast to the existing FER methods.

Human Facial Expression Recognition Using Stepwise Linear Discriminant Analysis and Hidden Conditional Random Fields

Facial expression recognition (FER) from video is an essential research area in the field of human computer interfaces (HCI). In this work, we present a new method to recognize several facial expressions from time sequential facial expression images. To produce robust facial expression features, enhanced independent component analysis (EICA) is utilized to extract locally independent component (IC) features which are further classified by Fisher linear discriminant analysis (FLDA). Using these features, discrete hidden Markov models (HMMs) are utilized to model different facial expressions such as joy, anger, and sad. Performance of our proposed FER system is compared against four other conventional feature extraction approaches (i.e., PCA, PCA-FLDA, ICA, and EICA) in conjunction with the same HMM scheme. The experimental results using the Cohn-Kanade database of facial expression videos show that our proposed system yields much improved recognition rate reaching the mean recognition rate of 93.23% whereas the conventional methods yield 82.92% at best.

An enhanced independent component-based human facial expression recognition from video

In proactive computing, human activity recognition from image sequences is an active research area. In this paper, a novel human activity recognition method is proposed, which utilizes Independent Component Analysis (ICA) for activity shape information extraction from image sequences and Hidden Markov Model (HMM) for recognition. Various human activities are represented by shape feature vectors from the sequence of activity shape images via ICA. Based on these features, each HMM is trained and activity recognition is achieved by the trained HMMs of different activities. Our recognition performance has been compared to the conventional method where Principal Component Analysis (PCA) is typically used to derive activity shape features. Our results show that superior recognition is achieved with the proposed method especially for activities (e.g., skipping) that cannot be easily recognized by the conventional method. Furthermore, by employing Linear Discriminant Analysis (LDA) on IC features, the recognition results further improved significantly in the recognition performance.

Independent shape component-based human activity recognition via Hidden Markov Model

In this paper, a novel human activity recognition method is proposed which utilizes independent components of activity shape information from image sequences and Hidden Markov Model (HMM) for recognition. Activities are represented by feature vectors from Independent Component Analysis (ICA) on video images and based on these features, recognition is achieved by trained HMMs of activities. Our recognition performance has been compared to the conventional method where Principle Component Analysis (PCA) is typically used to derive activity shape features. Our results show that superior recognition is achieved with our proposed method especially for activities (e.g., skipping) that cannot be easily recognized by the conventional method.

Shape-Based Human Activity Recognition Using Independent Component Analysis and Hidden Markov Model

Facial expression recognition is an essential research area in the field of Human Computer Interface. In this work, we present a spatiotemporal approach of facial expression recognition using image sequences. The system proposed in this paper describes fisher independent component analysis as a feature extractor where the higher order moment classification method (i.e., independent component analysis) is augmented with fisher linear discriminant. This procedure is simply abbreviated as FICA and produces the shape based spatial facial expression features. For recognition, we have utilized Hidden Markov Model (HMM) to learn the obtained spatial features with the temporal dynamics in six different expressions. Our proposed approach for the first time deals with sequential images of emotion-specific facial expression data analyzed with FICA and recognized with HMM. Performance of our proposed system has been compared with four conventional approaches where principal component analysis, generic independent component analysis, enhanced independent component analysis, and augmented principal component analysis with linear discriminant analysis are utilized for feature extraction. Our preliminary results show that our proposed system yields much improved recognition rates reaching the mean recognition rate of 92.85%.

spatiotemporal human facial expression recognition using fisher independent component analysis and Hidden Markov Model

In proactive computing, human activity recognition from image sequences is an active research area. This paper presents a novel approach of human activity recognition based on Linear Discriminant Analysis (LDA) of Independent Component (IC) features from shape information. With extracted features, Hidden Markov Model (HMM) is applied for training and recognition. The recognition performance using LDA of IC features has been compared to other approaches including Principle Component Analysis (PCA), LDA of PC, and ICA. The preliminary results show much improved performance in the recognition rate with our proposed method.

J. J. Lee

Papers

An enhanced independent component-based human facial expression recognition from video

Independent shape component-based human activity recognition via Hidden Markov Model

Shape-Based Human Activity Recognition Using Independent Component Analysis and Hidden Markov Model

spatiotemporal human facial expression recognition using fisher independent component analysis and Hidden Markov Model

Independent Component feature-based human activity recognition via Linear Discriminant Analysis and Hidden Markov Model