A new technique for facial expression recognition is proposed, which uses the two-dimensional (2D) discrete cosine transform (DCT) over the entire face image as a feature detector and a constructive one-hidden-layer feedforward neural network as a facial expression classifier. An input-side pruning technique, proposed previously by the authors, is also incorporated into the constructive learning process to reduce the network size without sacrificing the performance of the resulting network. The proposed technique is applied to a database consisting of images of 60 men, each having five facial expression images (neutral, smile, anger, sadness, and surprise). Images of 40 men are used for network training, and the remaining images of 20 men are used for generalization and testing. Confusion matrices calculated in both network training and generalization for four facial expressions (smile, anger, sadness, and surprise) are used to evaluate the performance of the trained network. It is demonstrated that the best recognition rates are 100% and 93.75% (without rejection), for the training and generalizing images, respectively. Furthermore, the input-side weights of the constructed network are reduced by approximately 30% using our pruning method. In comparison with the fixed structure back propagation-based recognition methods in the literature, the proposed technique constructs one-hidden-layer feedforward neural network with fewer number of hidden units and weights, while simultaneously provide improved generalization and recognition performance capabilities.

Facial expression recognition using constructive feedforward neural networks

This paper describes a method to detect smiles and laughter sounds from the video of natural dialogue. A smile is the most common facial expression observed in a dialogue. Detecting a user's smiles and laughter sounds can be useful for estimating the mental state of the user of a spoken-dialogue-based user interface. In addition, detecting laughter sound can be utilized to prevent the speech recognizer from wrongly recognizing the laughter sound as meaningful words. In this paper, a method to detect smile expression and laughter sound robustly by combining an image-based facial expression recognition method and an audio-based laughter sound recognition method. The image-based method uses a feature vector based on feature point detection from face images. The method could detect smile faces by more than 80% recall and precision rate. A method to combine a GMM-based laughter sound recognizer and the image-based method could improve the accuracy of detection of laughter sounds compared with methods that use image or sound only. As a result, more than 70% recall and precision rate of laughter sound detection was obtained from the natural conversation videos

Smile and laughter recognition using speech processing and face recognition from conversation video

Narrowband active noise control (ANC) systems have many real-life applications where the noise signals generated by rotating machines are modeled as sinusoidal signals in additive noise. However, when the timing signal sensor, such as a tachometer used to identify the signal frequencies, and the cosine wave generator contain errors, the frequencies of reference sinusoids fed to each ANC channel will then be different from the real primary noise signal frequencies. This difference is referred to as frequency mismatch (FM). In this paper, through extensive simulations, we first demonstrate that the performance capabilities of a conventional parallel form narrowband ANC system using the filtered-x LMS (FxLMS) algorithm degrades significantly even for an FM as small as 1 %. Convergence of the algorithm in the mean sense is also analytically investigated for a better understanding of its performance degradation. Next, we propose a new narrowband ANC system that successfully compensates for the performance degradations due to the FM. The amplitude/phase adjustment of reference sinusoids and the FM mitigations in the proposed system are performed simultaneously in a harmonic fashion such that the influence of the FM can be removed almost completely. Simulations as well as application to a real noise signal are provided to demonstrate the effectiveness of the proposed new system

A New Robust Narrowband Active Noise Control System in the Presence of Frequency Mismatch

A part-based approach for spontaneous expression recognition using audio-visual feature and deep convolution neural network (DCNN) is proposed. The ability of convolution neural network to handle variations in translation and scale is exploited for extracting visual features. The sub-regions, namely, eye and mouth parts extracted from the video faces are given as an input to the deep CNN (DCNN) inorder to extract convnet features. The audio features, namely, voice-report, voice intensity, and other prosodic features are used to obtain complementary information useful for classification. The confidence scores of the classifier trained on different facial parts and audio information are combined using different fusion rules for recognizing expressions. The effectiveness of the proposed approach is demonstrated on acted facial expression in wild (AFEW) dataset.

Spontaneous Facial Expression Recognition: A Part Based Approach

Performance of the conventional broadband active noise control (ANC) system may degrade severely if its primary and reference noise signals contain both wideband and narrowband components simultaneously. In this letter, we propose a new feedforward hybrid ANC system capable of reducing such primary noise signals. First, typical simulation results are provided to show the performance deterioration of the conventional system in the presence of mixture of wideband and narrowband components. Next, a new hybrid ANC system is proposed to tackle the problem. The new system consists of three subsystems, i.e., a sinusoidal noise canceller, a broadband and a narrowband ANC subsystem, which work in harmony. Extensive simulations are conducted to demonstrate the effectiveness of the proposed system.

A New Feedforward Hybrid Active Noise Control System

The recognition of human facial expressions has been expected to have applications in various fields, such as psychology, engineering, and so forth, and many techniques have been proposed to date. However, most of the techniques are based on the motions of local feature-bearing blocks, such as the eyes and mouth, which are supposed to be closely associated with facial expressions. It is thus required to segment these blocks from the face image and to track their motions in real-time applications, resulting in high complexity. Furthermore, not all information on the motions of the facial expressions has been utilized. In this paper, a new recognition technique is proposed, which uses the 2D DCT of the entire facial image and a neural network. The segmentation of local feature-bearing blocks is not needed. The differences of the DCT coefficients in the lower frequency area between neutral and expression-bearing images are given to a neural network to realize a mapping into the facial expression space. The new technique has been applied to a database of normalized facial images of 60 persons, with images of 40 persons used for network training and the remaining images for testing. The recognition rate is 100% for the training images. A maximum recognition rate of 95% is achieved for the testing images. © 1999 Scripta Technica, Electron Comm Jpn Pt 3, 82(7): 1–11, 1999

Recognition of facial expressions using 2d dct and neural network

Real-valued LMS Fourier analyzer for sinusoidal signals in additive noise

Facial Expression Recognition Using Vector Matching of Spacial Frequency Components

https://www.jstage.jst.go.jp/article/ieejeiss1987/118/1/118_1_29/_pdf

Yegui Xiao

Papers

Recognition of facial expressions using 2d dct and neural network

Real-valued LMS Fourier analyzer for sinusoidal signals in additive noise

Facial Expression Recognition Using Vector Matching of Spacial Frequency Components

Adaptive Frequency Estimation by Variable Sampling and Synchronous Addition / Subtraction Processing