Communication without words

Human facial emotion recognition (FER) has attracted the attention of the research community for its promising applications. Mapping different facial expressions to the respective emotional states are the main task in FER. The classical FER consists of two major steps: feature extraction and emotion recognition. Currently, the Deep Neural Networks, especially the Convolutional Neural Network (CNN), is widely used in FER by virtue of its inherent feature extraction mechanism from images. Several works have been reported on CNN with only a few layers to resolve FER problems. However, standard shallow CNNs with straightforward learning schemes have limited feature extraction capability to capture emotion information from high-resolution images. A notable drawback of the most existing methods is that they consider only the frontal images (i.e., ignore profile views for convenience), although the profile views taken from different angles are important for a practical FER system. For developing a highly accurate FER system, this study proposes a very Deep CNN (DCNN) modeling through Transfer Learning (TL) technique where a pre-trained DCNN model is adopted by replacing its dense upper layer(s) compatible with FER, and the model is fine-tuned with facial emotion data. A novel pipeline strategy is introduced, where the training of the dense layer(s) is followed by tuning each of the pre-trained DCNN blocks successively that has led to gradual improvement of the accuracy of FER to a higher level. The proposed FER system is verified on eight different pre-trained DCNN models (VGG-16, VGG-19, ResNet-18, ResNet-34, ResNet-50, ResNet-152, Inception-v3 and DenseNet-161) and well-known KDEF and JAFFE facial image datasets. FER is very challenging even for frontal views alone. FER on the KDEF dataset poses further challenges due to the diversity of images with different profile views together with frontal views. The proposed method achieved remarkable accuracy on both datasets with pre-trained models. On a 10-fold cross-validation way, the best achieved FER accuracies with DenseNet-161 on test sets of KDEF and JAFFE are 96.51% and 99.52%, respectively. The evaluation results reveal the superiority of the proposed FER system over the existing ones regarding emotion detection accuracy. Moreover, the achieved performance on the KDEF dataset with profile views is promising as it clearly demonstrates the required proficiency for real-life applications.

https://www.mdpi.com/2079-9292/10/9/1036/pdf

Facial Emotion Recognition Using Transfer Learning in the Deep CNN

In marketing research, one of the most exciting, innovative, and promising trends is quantification of customer interest. This paper presents a deep learning-based system for monitoring customer behavior specifically for detection of interest. The proposed system first measures customer attention through head pose estimation. For those customers whose heads are oriented toward the advertisement or the product of interest, the system further analyzes the facial expressions and reports customers’ interest. The proposed system starts by detecting frontal face poses; facial components important for facial expression recognition are then segmented and an iconized face image is generated; finally, facial expressions are analyzed using the confidence values of obtained iconized face image combined with the raw facial images. This approach fuses local part-based features with holistic facial information for robust facial expression recognition. With the proposed processing pipeline, using a basic imaging device, such as a webcam, head pose estimation, and facial expression recognition is possible. The proposed pipeline can be used to monitor emotional response of focus groups to various ideas, pictures, sounds, words, and other stimuli.

Deep learning-based face analysis system for monitoring customer interest

Various myoelectric prostheses controlled by electromyography (EMG) signals have been developed However, there have been few studies that provide fast and accurate methods to predict handgrip force from EMG signals Rapid and precise handgrip force prediction is required, especially for the real-time control system of myoelectric prostheses In this study, extreme learning machine (ELM) is applied to predict handgrip force from surface EMG signals of forearm muscles Furthermore, ELM is compared with support vector machine (SVM) and multiple nonlinear regression (MNLR) The below 10 % of the surface EMG and handgrip force signals were cut away, and then the root mean square feature extracted from the modified surface EMG signals was taken as input vector for these three kinds of predicting mechanisms For the testing dataset, ELM achieved a slightly larger root mean squared error than SVM did and a smaller one than MNLR did Meanwhile, all three methods showed high correlation coefficients For the total processing time, ELM and MNLR consumed much less time than SVM did Experimental results demonstrate that ELM possesses a relatively good accuracy and little consumed time, although SVM is effective for handgrip force estimation in terms of accuracy Overall, ELM has a promising potential for predicting handgrip force rapidly and precisely

Modified EMG-based handgrip force prediction using extreme learning machine

In cloud manufacturing (CMfg) environments, an increasing number of manufacturing enterprises outsource manufacturing activities to subcontractors that are more professional to focus on the development of their core business. Volume, diversity, and variety are the typical characteristics of outsourcing services for cement equipment manufacturing enterprises (CEMEs). To address the new problem of service discovery and combinatorial optimization of outsourcing resources (COOR), a novel heuristic approach is investigated in this paper. First, a clustering and searching model of a web outsourcing service based on Ontology Web Language for Services (OWL-S) and an artificial neural network (ANN) is established. Then, an improved shuffled frog leaping algorithm (SFLA) is developed to solve the COOR problem. Finally, an investigation and comparative experiments based on a group of cement equipment manufacturing companies is presented. The experimental results show that the proposed method is preferable and is more efficient for solving large-scale problems in a CMfg environment.

An outsourcing service selection method using ANN and SFLA algorithms for cement equipment manufacturing enterprises in cloud manufacturing

This paper develops a regularized discriminant analysis (RDA)-based boosting algorithm, and its application of the facial emotion recognition. The RDA-based boosting algorithm uses RDA as a learning rule in the boosting algorithm. The RDA combines strengths of linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). It solves small sample size and ill-posed problems suffered from QDA and LDA through a regularization technique. Additionally, this study uses a particle swarm optimization algorithm to estimate optimal parameters in RDA. In this work, the proposed RDA-based boosting is used in the facial emotion recognition, and achieves a good performance. In the facial emotion recognition, contourlet features are extracted and followed by an entropy criterion to select the informative contourlet features which is a subset of informative and non-redundant contourlet features. Experiment results demonstrate that the proposed RDA-based boosting can accurately and robustly recognize facial emotions.

Po-Chiang Lin

Papers

An improved boosting algorithm and its application to facial emotion recognition