Human action recognition techniques have gained significant attention among next-generation technologies due to their specific features and high capability to inspect video sequences to understand human actions. As a result, many fields have benefited from human action recognition techniques. Deep learning techniques played a primary role in many approaches to human action recognition. The new era of learning is spreading by transfer learning. Accordingly, this study’s main objective is to propose a framework with three main phases for human action recognition. The phases are pre-training, preprocessing, and recognition. This framework presents a set of novel techniques that are three-fold as follows, (i) in the pre-training phase, a standard convolutional neural network is trained on a generic dataset to adjust weights; (ii) to perform the recognition process, this pre-trained model is then applied to the target dataset; and (iii) the recognition phase exploits convolutional neural network and long short-term memory to apply five different architectures. Three architectures are stand-alone and single-stream, while the other two are combinations between the first three in two-stream style. Experimental results show that the first three architectures recorded accuracies of 83.24%, 90.72%, and 90.85%, respectively. The last two architectures achieved accuracies of 93.48% and 94.87%, respectively. Moreover, The recorded results outperform other state-of-the-art models in the same field.

/pdf/human-action-recognition-based-on-transfer-learning-approach-2acvz4wq9x.pdf

Human Action Recognition Based on Transfer Learning Approach

Sporadic falls, due to the lack of balance and other factors, are some of the complications that elderly people might experience more frequently than others. Accordingly, as there is a high probability of these events causing major health casualties, such as bone breaking or head clots, studies have been monitoring these falls to rapidly assist the victim. In this work, we propose and evaluate a multi-stream learning model based on convolutional neural networks using high-level handcrafted features as input in order to cope with this situation. Therefore, our approach consists of extracting high-level handcrafted features, for instance, human pose estimation and optical flow, and using each one as an input for a distinct VGG-16 classifier. In addition, these experiments are able to showcase what features can be used in fall detection. The results have shown that by assembling our directed input learners, our approach outperforms, in terms of accuracy and sensitivity rates, to other similar tested methods found in literature.

Multi-Stream Deep Convolutional Network Using High-Level Features Applied to Fall Detection in Video Sequences

Action recognition requires modelling the interactions between either human & human or human & objects. Re-cently, graph convolutional neural networks (GCNs) are exploited to effectively capture the structure of action by modelling the relationship among entities present in a video. However, most of the approaches depend on the effectiveness of object detection frameworks to detect the entities. In this paper, we propose a graph-based framework for action recognition to model the spatio-temporal interactions among the entities in a video without any object-level supervision. First, we obtain the salient space-time interest points (STIP) that contain rich information about the significant local variations in space and time by using the Harris 3D detector. In order to incorporate the local appearance and motion information of the entities, either low-level or deep features are extracted around these STIPs. Next, we build a graph by considering the extracted STIPs as nodes and are connected by spatial edges and temporal edges. These edges are determined based on a membership function that measures the similarity of entities associated with the STIPs. Finally, GCN is employed on the given graph to provide reasoning among different entities present in a video. We evaluate our method on three widely used datasets, namely, UCF-101, HMDB-51, SSV2 to demonstrate the efficacy of the proposed approach. 

STIP-GCN: Space-time interest points graph convolutional network for action recognition

Action recognition requires modelling the interactions between either human & human or human & objects. Re-cently, graph convolutional neural networks (GCNs) are exploited to effectively capture the structure of action by modelling the relationship among entities present in a video. However, most of the approaches depend on the effectiveness of object detection frameworks to detect the entities. In this paper, we propose a graph-based framework for action recognition to model the spatio-temporal interactions among the entities in a video without any object-level supervision. First, we obtain the salient space-time interest points (STIP) that contain rich information about the significant local variations in space and time by using the Harris 3D detector. In order to incorporate the local appearance and motion information of the entities, either low-level or deep features are extracted around these STIPs. Next, we build a graph by considering the extracted STIPs as nodes and are connected by spatial edges and temporal edges. These edges are determined based on a membership function that measures the similarity of entities associated with the STIPs. Finally, GCN is employed on the given graph to provide reasoning among different entities present in a video. We evaluate our method on three widely used datasets, namely, UCF-101, HMDB-51, SSV2 to demonstrate the efficacy of the proposed approach.

YOLO V3 + VGG16-based automatic operations monitoring and analysis in a manufacturing workshop under Industry 4.0

Despite the significant progress of Deep Learning models on the image classification task, it still needs enhancements for the Human Action Recognition task. In this work, we propose to extract horizontal and vertical Visual Rhythms as well as their data augmentations as video features. The data augmentation is driven by crops extracted from the symmetric extension of the time dimension, preserving the video frame rate, which is essential to keep motion patterns. The crops provide a 2D representation of the video volume matching the fixed input size of a 2D Convolutional Neural Network. In addition, multiple crops with stride guarantee coverage of the entire video. We verified that the combination of horizontal and vertical directions leads do better results than previous methods. A multi-stream strategy combining RGB and Optical Flow information is modified to include the additional spatiotemporal streams: one for the horizontal Symmetrically Extended Visual Rhythm (SEVR), and another for the vertical one. Results show that our method achieves accuracy rates close to the state of the art on the challenging UCF101 and HMDB51 datasets. Furthermore, we assessed the impact of data augmentations methods for Human Action Recognition and verified an increase of 10% for

Multi-stream Architecture with Symmetric Extended Visual Rhythms for Deep Learning Human Action Recognition.

Nos ultimos anos, os avancos em Aprendizado Profundo revolucionaram diversas sub-
areas da Visao Computacional, incluindo o Rastreamento de Objetos Visuais. Um tipo especial
de rede neural profunda, a Rede Neural Siamesa, chamou a atencao da comunidade
especializada em rastreamento. Ela possui baixo custo computacional e alta efi cacia para
comparar a similaridade entre objetos. Atualmente, a comunidade cienti ca atingiu resultados
notaveis ao aplicar tais redes ao problema de Rastreamento de Objetos Visuais.
No entanto, observou-se que limitacoes dessa rede neural impactam negativamente no rastreamento.
Superou-se o problema ao se obter um novo descritor para referencia do objeto
combinando descritores passados fornecidos pelo rastreador. Em particular, foi proposto
a combinacao de sinal de descritores em blocos de memorias de longo e de curto prazo,
os quais representam a primeira e a mais recente aparencia do objeto, respectivamente.
Um descritor nal e gerado a partir desses blocos de memoria, o qual o rastreador usa
como referencia. Este trabalho enfatizou-se na obtencao de um metodo para calcular um
banco de ltros otimizado atraves do uso de um algoritmo genetico. O banco de ltros
e utilizado entao para gerar a saida da memoria de curto prazo. De acordo com experimentos
realizados na base de dados OTB, esta proposta apresenta ganhos em comparacao
com a proposta original da SiamFC. Considerando a metrica area abaixo da curva, ha
ganhos de 7.4% e 3.0% para os gra cos de precisao e sucesso, respectivamente, tornando
este trabalho comparavel a metodos do estato da arte.

/pdf/filter-learning-from-deep-descriptors-of-a-fully-2mlexp9xvn.pdf

Hemerson Tacon

Papers

Multi-stream Architecture with Symmetric Extended Visual Rhythms for Deep Learning Human Action Recognition.

Filter Learning from Deep Descriptors of a Fully Convolutional Siamese Network for Tracking in Videos.