UntrimmedNets for Weakly Supervised Action Recognition and Detection
Limin Wang,Yuanjun Xiong,Dahua Lin,Luc Van Gool +3 more
- pp 6402-6411
TLDR
This paper presents a new weakly supervised architecture, called UntrimmedNet, which is able to directly learn action recognition models from untrimmed videos without the requirement of temporal annotations of action instances.Abstract:
Current action recognition methods heavily rely on trimmed videos for model training. However, it is expensive and time-consuming to acquire a large-scale trimmed video dataset. This paper presents a new weakly supervised architecture, called UntrimmedNet, which is able to directly learn action recognition models from untrimmed videos without the requirement of temporal annotations of action instances. Our UntrimmedNet couples two important components, the classification module and the selection module, to learn the action models and reason about the temporal duration of action instances, respectively. These two components are implemented with feed-forward networks, and UntrimmedNet is therefore an end-to-end trainable architecture. We exploit the learned models for action recognition (WSR) and detection (WSD) on the untrimmed video datasets of THUMOS14 and ActivityNet. Although our UntrimmedNet only employs weak supervision, our method achieves performance superior or comparable to that of those strongly supervised approaches on these two datasets.read more
Citations
More filters
Posted Content
SmallBigNet: Integrating Core and Contextual Views for Video Classification
TL;DR: In this article, the authors proposed a small-big network with the cooperation of small and big views to learn more robust and discriminative spatio-temporal representations for video classification, which improves model compactness as well as alleviates overfitting.
Journal ArticleDOI
Uncertainty Guided Collaborative Training for Weakly Supervised and Unsupervised Temporal Action Localization
TL;DR: A novel Uncertainty Guided Collaborative Training (UGCT) strategy to alleviate the problem of weakly supervised and unsupervised temporal action localization, which mainly includes an online pseudo label generation module, in which the RGB and FLOW streams work collaboratively to learn from each other.
Action recognition from RGB-D data
TL;DR: To exploit the joint information in multi-modal features arising from heterogeneous sources (RGB, depth), it is proposed to cooperatively train a single ConvNet on both RGB features and depth features, and deeply aggregate the two modalities to achieve robust action recognition.
Journal Article
Recent Progress in Appearance-based Action Recognition
TL;DR: A comprehensive review of recent achievements in appearance-based action recognition is provided and several dozens of related research papers are summarized and discussed, which can be roughly divided into four categories according to different appearance modelling strategies.
Proceedings ArticleDOI
Category-Level Multi-Attention based Boundary Refinement for Action Detection
TL;DR: This paper focuses on a weakly supervised setting using only video-level labels to address the temporal action localization task and proposes a category-level multi-attention based method to provide a comprehensive representation for each action category and obtain much more precise temporal intervals.
References
More filters
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Proceedings ArticleDOI
ImageNet: A large-scale hierarchical image database
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Proceedings Article
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe,Christian Szegedy +1 more
TL;DR: Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
Journal ArticleDOI
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Related Papers (5)
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Joao Carreira,Andrew Zisserman +1 more