A new hybrid deep learning model for human action recognition

This paper addresses the recognitions of human actions in videos Human action recognition can be seen as the automatic labeling of a video according to the actions occurring in it It has become one of the most challenging and attractive problems in the pattern recognition and video classification fields The problem itself is difficult to solve by traditional video processing methods because of several challenges such as the background noise, sizes of subjects in different videos, and the speed of actions Derived from the progress of deep learning methods, several directions are developed to recognize a human action from a video, such as the long-short-term memory (LSTM)-based model, two-stream convolutional neural network (CNN) model, and the convolutional 3D modelIn this paper, we focus on the two-stream structure The traditional two-stream CNN network solves the problem that CNNs do not have satisfactory performance on temporal features By training a temporal stream, which uses the optical flow as the input, a CNN can have the ability to extract temporal features However, the optical flow only contains limited temporal information because it only records the movements of pixels on the x-axis and the y-axis Therefore, we attempt to design and implement a new two-stream model by using an LSTM-based model in its spatial stream to extract both spatial and temporal features in RGB frames In addition, we implement a DenseNet in the temporal stream to improve the recognition accuracy This is in-contrast to traditional approaches which typically utilize the spatial stream for extracting only spatial features The quantitative evaluation and experiments are conducted on the UCF-101 dataset, which is a well-developed public video dataset For the temporal stream, we choose the optical flow of UCF-101 Images in the optical flow are provided by the Graz University of Technology The experimental result shows that the proposed method outperforms the traditional two-stream CNN method with an accuracy of at least 3% For both spatial and temporal streams, the proposed model also achieves higher recognition accuracies In addition, compared with the state of the art methods, the new model can still have the best recognition performance

/pdf/improved-two-stream-model-for-human-action-recognition-22pqo1q8bq.pdf

Improved two-stream model for human action recognition

The recognition of human activities is usually considered to be a simple procedure. Problems occur in complex scenes involving high speeds. Activity prediction using Artificial Intelligence (AI) by numerical analysis has attracted the attention of several researchers. Human activities are an important challenge in various fields. There are many great applications in this area, including smart homes, assistive robotics, human–computer interactions, and improvements in protection in several areas such as security, transport, education, and medicine through the control of falling or aiding in medication consumption for elderly people. The advanced enhancement and success of deep learning techniques in various computer vision applications encourage the use of these methods in video processing. The human presentation is an important challenge in the analysis of human behavior through activity. A person in a video sequence can be described by their motion, skeleton, and/or spatial characteristics. In this paper, we present a novel approach to human activity recognition from videos using the Recurrent Neural Network (RNN) for activity classification and the Convolutional Neural Network (CNN) with a new structure of the human skeleton to carry out feature presentation. The aims of this work are to improve the human presentation through the collection of different features and the exploitation of the new RNN structure for activities. The performance of the proposed approach is evaluated by the RGB-D sensor dataset CAD-60. The experimental results show the performance of the proposed approach through the average error rate obtained (4.5%).

https://www.mdpi.com/1424-8220/20/17/4944/pdf

Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model.

Automatic interpretation of human actions from realistic videos attracts increasing research attention owing to its growing demand in real-world deployments such as biometrics, intelligent robotics, and surveillance. In this research, we propose an ensemble model of evolving deep networks comprising Convolutional Neural Networks (CNNs) and bidirectional Long Short-Term Memory (BLSTM) networks for human action recognition. A swarm intelligence (SI)-based algorithm is also proposed for identifying the optimal hyper-parameters of the deep networks. The SI algorithm plays a crucial role for determining the BLSTM network and learning configurations such as the learning and dropout rates and the number of hidden neurons, in order to establish effective deep features that accurately represent the temporal dynamics of human actions. The proposed SI algorithm incorporates hybrid crossover operators implemented by sine, cosine, and tanh functions for multiple elite offspring signal generation, as well as geometric search coefficients extracted from a three-dimensional super-ellipse surface. Moreover, it employs a versatile search process led by the yielded promising offspring solutions to overcome stagnation. Diverse CNN–BLSTM networks with distinctive hyper-parameter settings are devised. An ensemble model is subsequently constructed by aggregating a set of three optimized CNN–BLSTM​ networks based on the average prediction probabilities. Evaluated using several publicly available human action data sets, our evolving ensemble deep networks illustrate statistically significant superiority over those with default and optimal settings identified by other search methods. The proposed SI algorithm also shows great superiority over several other methods for solving diverse high-dimensional unimodal and multimodal optimization functions with artificial landscapes.

Intelligent human action recognition using an ensemble model of evolving deep networks with swarm-based optimization.

From the past few decades, Human activity recognition (HAR) is one of the vital research areas in computer vision in which much research is ongoing. The researcher’s focus is shifting towards this area due to its vast range of real-life applications to assist in daily living. Therefore, it is necessary to validate its performance on standard benchmark datasets and state-of-the-art systems before applying it in real-life applications. The primary objective of this Systematic Literature Review (SLR) is to collect existing research on video-based human activity recognition, summarize, and analyze the state-of-the-art deep learning architectures regarding various methodologies, challenges, and issues. The top five scientific databases (such as ACM, IEEE, ScienceDirect, SpringerLink, and Taylor & Francis) are accessed to accompany this systematic study by summarizing 70 different research articles on human activity recognition after critical review. Human activity recognition in videos is a challenging problem due to its diverse and complex nature. For accurate video classification, extraction of both spatial and temporal features from video sequences is essential. Therefore, this SLR focuses on reviewing the recent advancements in stratified self-deriving feature-based deep learning architectures. Furthermore, it explores various deep learning techniques available for HAR, challenges researchers to face to build a robust model, and state-of-the-art datasets used for evaluation. This SLR intends to provide a baseline for video-based human activity recognition research while emphasizing several challenges regarding human activity recognition accuracy in video sequences using deep neural architectures.

/pdf/analysis-of-deep-neural-networks-for-human-activity-40kl422ooq.pdf

Analysis of Deep Neural Networks for Human Activity Recognition in Videos—A Systematic Literature Review

Traditional convolution neural networks have achieved great success in human action recognition. However, it is challenging to establish effective associations between different human bone nodes to capture detailed information. In this paper, we propose a dual attention-guided multiscale dynamic aggregate graph convolution neural network (DAG-GCN) for skeleton-based human action recognition. Our goal is to explore the best correlation and determine high-level semantic features. First, a multiscale dynamic aggregate GCN module is used to capture important semantic information and to establish dependence relationships for different bone nodes. Second, the higher level semantic feature is further refined, and the semantic relevance is emphasized through a dual attention guidance module. In addition, we exploit the relationship of joints hierarchically and the spatial temporal correlations through two modules. Experiments with the DAG-GCN method result in good performance on the NTU-60-RGB+D and NTU-120-RGB+D datasets. The accuracy is 95.76% and 90.01%, respectively, for the cross (X)-View and X-Subon the NTU60dataset.

Dual Attention-Guided Multiscale Dynamic Aggregate Graph Convolutional Networks for Skeleton-Based Human Action Recognition

In recent years, deep Convolutional neural networks(CNNs) have made fantastic progress in static image recognition, but the ability to model motion information on behavioral video is weak. Therefore, our paper put forward a new time transition layer that models variable temporal convolution kernel depths. We embed this new Hybrid Model in our proposed 3D CNN. We extend the DenseNet architecture with 3D filters and pooling kernels. It will take time as training a 3D convolutional neural network requires a large number of tagged data sets to start training from the input. Therefore, the focus of this paper is on simple and effective technique of passing 2D convolutional neural network pre-trained data to a randomly initialized 3D convolutional neural network for stable weight initialization, where we can still achieve our experimental results by appropriately reducing the number of 3D convolutional neural network training samples. Experiments show that the network can make a more accurate classification of behavioral video, identify it in the UCF-101 database, and compare it with other classical algorithms that have appeared in recent years. The results reflect the superiority of the algorithm.

Human Motion Recognition Based on Improved 3-Dimensional Convolutional Neural Network

/pdf/human-action-recognition-based-on-global-silhouette-and-cnhl21ohos.pdf

Human Action Recognition Based on Global Silhouette and Local Optical Flow

Aiming at the problem of complex feature extraction and low accuracy in human action recognition, this paper proposed a network structure combining batch normalization algorithm with GoogLeNet network model. Applying Batch Normalization idea in the field of image classification to action recognition field, it improved the algorithm by normalizing the network input training sample by mini-batch. For convolutional network, RGB image was the spatial input, and stacked optical flows was the temporal input. Then, it fused the spatio-temporal networks to get the final action recognition result. It trained and evaluated the architecture on the standard video actions benchmarks of UCF101 and HMDB51, which achieved the accuracy of 93.42% and 67.82%. The results show that the improved convolutional neural network has a significant improvement in improving the recognition rate and has obvious advantages in action recognition.

http://koreascience.or.kr:80/article/JAKO202024852036271.pdf

Human Motion Recognition Based on Spatio-temporal Convolutional Neural Network

The choice of the motion features influences the result of the human action recognition method directly. Many factors often influence the single feature differently, such as appearance of the human body, environment and video camera. So the accuracy of action recognition is restricted. On the bases of studying the representation and recognition of human actions, and giving fully consideration to the advantages and disadvantages of different features, the Dense Trajectories(DT) algorithm is a very classic algorithm in the field of behavior recognition feature extraction, but there are some defects in the use of optical flow images. In this paper, we will use the improved Dense Trajectories(iDT) algorithm to optimize and extract the optical flow features in the movement of human action, then we will combined with Support Vector Machine methods to identify human behavior, and use the image in the KTH database for training and testing..

https://www.koreascience.or.kr:443/article/JAKO201814955686285.pdf

Zeyuan Hu

Papers

Dual Attention-Guided Multiscale Dynamic Aggregate Graph Convolutional Networks for Skeleton-Based Human Action Recognition

Human Motion Recognition Based on Improved 3-Dimensional Convolutional Neural Network

Human Action Recognition Based on Global Silhouette and Local Optical Flow

Human Motion Recognition Based on Spatio-temporal Convolutional Neural Network

Improved DT Algorithm Based Human Action Features Detection