Facial expression recognition method with multi-label distribution learning for non-verbal behavior understanding in the classroom

Sign language recognition is one of the most challenging applications in machine learning and human-computer interaction. Many researchers have developed classification models for different sign languages such as English, Arabic, Japanese, and Bengali; however, no significant research has been done on the general-shape performance for different datasets. Most research work has achieved satisfactory performance with a small dataset. These models may fail to replicate the same performance for evaluating different and larger datasets. In this context, this paper proposes a novel method for recognizing Bengali sign language (BSL) alphabets to overcome the issue of generalization. The proposed method has been evaluated with three benchmark datasets such as `38 BdSL’, `KU-BdSL’, and `Ishara-Lipi’. Here, three steps are followed to achieve the goal: segmentation, augmentation, and Convolutional neural network (CNN) based classification. Firstly, a concatenated segmentation approach with YCbCr, HSV and watershed algorithm was designed to accurately identify gesture signs. Secondly, seven image augmentation techniques are selected to increase the training data size without changing the semantic meaning. Finally, the CNN-based model called BenSignNet was applied to extract the features and classify purposes. The performance accuracy of the model achieved 94.00%, 99.60%, and 99.60% for the BdSL Alphabet, KU-BdSL, and Ishara-Lipi datasets, respectively. Experimental findings confirmed that our proposed method achieved a higher recognition rate than the conventional ones and accomplished a generalization property in all datasets for the BSL domain.

/pdf/bensignnet-bengali-sign-language-alphabet-recognition-using-2ak8qzpc.pdf

BenSignNet: Bengali Sign Language Alphabet Recognition Using Concatenated Segmentation and Convolutional Neural Network

Objects that occupy a small portion of an image or a frame contain fewer pixels and contains less information. This makes small object detection a challenging task in computer vision. In this paper, an improved Single Shot multi-box Detector based on feature fusion and dilated convolution (FD-SSD) is proposed to solve the problem that small objects are difficult to detect. The proposed network uses VGG-16 as the backbone network, which mainly includes a multi-layer feature fusion module and a multi-branch residual dilated convolution module. In the multi-layer feature fusion module, the last two layers of the feature map are up-sampled, and then they are concatenated at the channel level with the shallow feature map to enhance the semantic information of the shallow feature map. In the multi-branch residual dilated convolution module, three dilated convolutions with different dilated ratios based on the residual network are combined to obtain the multi-scale context information of the feature without losing the original resolution of the feature map. In addition, deformable convolution is added to each detection layer to better adapt to the shape of small objects. The proposed FD-SSD achieved 79.1% mAP and 29.7% mAP on PASCAL VOC2007 dataset and MS COCO dataset respectively. Experimental results show that FD-SSD can effectively improve the utilization of multi-scale information of small objects, thus significantly improve the effect of the small object detection.

FD-SSD: An improved SSD object detection algorithm based on feature fusion and dilated convolution

• A Gaussian mixed distribution label-based model is proposed for students’ head pose estimation. • The model can learn the discriminative facial features in each angle image. • Experimental results show that the GMDL model outperforms other existing models. Students’ head pose estimation is a very difficult task since the training data is insufficient for many head pose angles. In this study, we consider each head pose image as a Gaussian mixed distribution other than the traditional hard label, in which the adjacent head pose images can provide supplementary information for the target image. Specifically, the Gaussian mixed distribution covers the current head pose image and its adjacent 8 head pose images. Each label of head pose image describes the similar degree between the current image and its adjacent head pose images. Then, a novel network architecture is proposed by constructing the Gaussian mixed distribution which learns more discriminative facial features. The extensive evaluations on two public HPE databases show that the proposed GMDL model obtains the better performance compared with the conventional algorithms. In practice, the proposed model can be utilized to estimate learners’ head pose angle for attention understanding in the instruction and learning scenarios. 

GMDL: Toward precise head pose estimation via Gaussian mixed distribution learning for students’ attention understanding

With the development of Internet of Things technology, Channel State Information (CSI) based human activity recognition (HAR) plays an important role in Human-Computer Interaction and achieves considerable advancements over recent years. However, when the trained model is applied to recognize new activity categories or recognizing new users in new scenarios, the recognition performance of general methods will dramatically decline. And re-collecting adequate new activity categories’ samples to train the HAR model to adapt to the new situation will consume a lot of time and human effort. To overcome this challenge, we propose a framework, Augment Few Shot Learning-based Human Activity Recognition (AFSL-HAR), which can achieve significant performance in recognizing new categories through a small amount of samples to fine-tune the model parameters and avoid retraining the network from scratch again. And besides, in order to improve the robustness of AFSL-HAR, we design a Feature Wasserstein Generative Adversarial Network (FWGAN) module, which can synthesize diverse samples to help the recognition model learn more sharper classification boundaries. Specifically, the FWGAN module incorporates a feature extractor to realize converging with a fewer number of training samples, and takes an improved discriminator to enhance system performance. The experimental results demonstrate that AFSL-HAR can achieve accuracy of 98.9% and 94.7% when recognizing new activities using few samples for each category on the public data set and self-made data set, respectively.

Hanwen Nie

Papers

Anisotropic angle distribution learning for head pose estimation and attention understanding in human-computer interaction

On automatic recommender algorithm with regularized convolutional neural network and IR technology in the self-regulated learning process