Sign language recognition is one of the most challenging applications in machine learning and human-computer interaction. Many researchers have developed classification models for different sign languages such as English, Arabic, Japanese, and Bengali; however, no significant research has been done on the general-shape performance for different datasets. Most research work has achieved satisfactory performance with a small dataset. These models may fail to replicate the same performance for evaluating different and larger datasets. In this context, this paper proposes a novel method for recognizing Bengali sign language (BSL) alphabets to overcome the issue of generalization. The proposed method has been evaluated with three benchmark datasets such as `38 BdSL’, `KU-BdSL’, and `Ishara-Lipi’. Here, three steps are followed to achieve the goal: segmentation, augmentation, and Convolutional neural network (CNN) based classification. Firstly, a concatenated segmentation approach with YCbCr, HSV and watershed algorithm was designed to accurately identify gesture signs. Secondly, seven image augmentation techniques are selected to increase the training data size without changing the semantic meaning. Finally, the CNN-based model called BenSignNet was applied to extract the features and classify purposes. The performance accuracy of the model achieved 94.00%, 99.60%, and 99.60% for the BdSL Alphabet, KU-BdSL, and Ishara-Lipi datasets, respectively. Experimental findings confirmed that our proposed method achieved a higher recognition rate than the conventional ones and accomplished a generalization property in all datasets for the BSL domain.

/pdf/bensignnet-bengali-sign-language-alphabet-recognition-using-2ak8qzpc.pdf

BenSignNet: Bengali Sign Language Alphabet Recognition Using Concatenated Segmentation and Convolutional Neural Network

Objects that occupy a small portion of an image or a frame contain fewer pixels and contains less information. This makes small object detection a challenging task in computer vision. In this paper, an improved Single Shot multi-box Detector based on feature fusion and dilated convolution (FD-SSD) is proposed to solve the problem that small objects are difficult to detect. The proposed network uses VGG-16 as the backbone network, which mainly includes a multi-layer feature fusion module and a multi-branch residual dilated convolution module. In the multi-layer feature fusion module, the last two layers of the feature map are up-sampled, and then they are concatenated at the channel level with the shallow feature map to enhance the semantic information of the shallow feature map. In the multi-branch residual dilated convolution module, three dilated convolutions with different dilated ratios based on the residual network are combined to obtain the multi-scale context information of the feature without losing the original resolution of the feature map. In addition, deformable convolution is added to each detection layer to better adapt to the shape of small objects. The proposed FD-SSD achieved 79.1% mAP and 29.7% mAP on PASCAL VOC2007 dataset and MS COCO dataset respectively. Experimental results show that FD-SSD can effectively improve the utilization of multi-scale information of small objects, thus significantly improve the effect of the small object detection.

FD-SSD: An improved SSD object detection algorithm based on feature fusion and dilated convolution

• A Gaussian mixed distribution label-based model is proposed for students’ head pose estimation. • The model can learn the discriminative facial features in each angle image. • Experimental results show that the GMDL model outperforms other existing models. Students’ head pose estimation is a very difficult task since the training data is insufficient for many head pose angles. In this study, we consider each head pose image as a Gaussian mixed distribution other than the traditional hard label, in which the adjacent head pose images can provide supplementary information for the target image. Specifically, the Gaussian mixed distribution covers the current head pose image and its adjacent 8 head pose images. Each label of head pose image describes the similar degree between the current image and its adjacent head pose images. Then, a novel network architecture is proposed by constructing the Gaussian mixed distribution which learns more discriminative facial features. The extensive evaluations on two public HPE databases show that the proposed GMDL model obtains the better performance compared with the conventional algorithms. In practice, the proposed model can be utilized to estimate learners’ head pose angle for attention understanding in the instruction and learning scenarios. 

GMDL: Toward precise head pose estimation via Gaussian mixed distribution learning for students’ attention understanding

With the development of Internet of Things technology, Channel State Information (CSI) based human activity recognition (HAR) plays an important role in Human-Computer Interaction and achieves considerable advancements over recent years. However, when the trained model is applied to recognize new activity categories or recognizing new users in new scenarios, the recognition performance of general methods will dramatically decline. And re-collecting adequate new activity categories’ samples to train the HAR model to adapt to the new situation will consume a lot of time and human effort. To overcome this challenge, we propose a framework, Augment Few Shot Learning-based Human Activity Recognition (AFSL-HAR), which can achieve significant performance in recognizing new categories through a small amount of samples to fine-tune the model parameters and avoid retraining the network from scratch again. And besides, in order to improve the robustness of AFSL-HAR, we design a Feature Wasserstein Generative Adversarial Network (FWGAN) module, which can synthesize diverse samples to help the recognition model learn more sharper classification boundaries. Specifically, the FWGAN module incorporates a feature extractor to realize converging with a fewer number of training samples, and takes an improved discriminator to enhance system performance. The experimental results demonstrate that AFSL-HAR can achieve accuracy of 98.9% and 94.7% when recognizing new activities using few samples for each category on the public data set and self-made data set, respectively.

Robust CSI-Based Human Activity Recognition With Augment Few Shot Learning

Overview of behavior recognition based on deep learning

Head pose estimation suffers from several problems, including low pose tolerance under different disturbances and ambiguity arising from common head pose representation. In this study, a robust three-branch model with triplet module and matrix Fisher distribution module is proposed to address these problems. Based on metric learning, the triplet module employs triplet architecture and triplet loss. It is implemented to maximize the distance between embeddings with different pose pairs and minimize the distance between embeddings with same pose pairs. It can learn a highly discriminate and robust embedding related to head pose. Moreover, the rotation matrix instead of Euler angle and unit quaternion is utilized to represent head pose. An exponential probability density model based on the rotation matrix (referred to as the matrix Fisher distribution) is developed to model head rotation uncertainty. The matrix Fisher distribution can further analyze the head pose, and its maximum likelihood obtained using singular value decomposition provides enhanced accuracy. Extensive experiments executed over AFLW2000 and BIWI datasets demonstrate that the proposed model achieves state-of-the-art performance in comparison with traditional methods.

MFDNet: Collaborative Poses Perception and Matrix Fisher Distribution for Head Pose Estimation

Recommendation accuracy is a fundamental problem in the quality of the recommendation system. In this paper, we propose an efficient deep matrix factorization with review feature learning for the industrial recommender system (EDMF). Two characteristics in user's review are revealed. First, interactivity between the user and the item, which can also be considered as the former's scoring behavior on the latter, is exploited in a review. Second, the review is only a partial description of the user's preferences for the item, which is revealed as the sparsity property. Specifically, in the first characteristic, EDMF extracts the interactive features of onefold review by convolutional neural networks with word attention mechanism. Subsequently, L0 norm is leveraged to constrain the review considering that the review information is a sparse feature, which is the second characteristic. Furthermore, the loss function is constructed by maximum a posteriori estimation theory, where the interactivity and sparsity property are converted as two prior probability functions. Finally, the alternative minimization algorithm is introduced to optimize the loss functions. Experimental results on several datasets demonstrate that the proposed methods, which show good industrial conversion application prospects, outperform the state-of-the-art methods in terms of effectiveness and efficiency.

Jiazhang Wang

Papers

MFDNet: Collaborative Poses Perception and Matrix Fisher Distribution for Head Pose Estimation

EDMF: Efficient Deep Matrix Factorization with Review Feature Learning for Industrial Recommender System

Multi-perspective social recommendation method with graph representation learning