Open AccessPosted Content
Convolutional Two-Stream Network Fusion for Video Action Recognition
TLDR
In this paper, a spatial and temporal network can be fused at the last convolution layer without loss of performance, but with a substantial saving in parameters, and furthermore, pooling of abstract convolutional features over spatiotemporal neighbourhoods further boosts performance.Abstract:
Recent applications of Convolutional Neural Networks (ConvNets) for human action recognition in videos have proposed different solutions for incorporating the appearance and motion information. We study a number of ways of fusing ConvNet towers both spatially and temporally in order to best take advantage of this spatio-temporal information. We make the following findings: (i) that rather than fusing at the softmax layer, a spatial and temporal network can be fused at a convolution layer without loss of performance, but with a substantial saving in parameters; (ii) that it is better to fuse such networks spatially at the last convolutional layer than earlier, and that additionally fusing at the class prediction layer can boost accuracy; finally (iii) that pooling of abstract convolutional features over spatiotemporal neighbourhoods further boosts performance. Based on these studies we propose a new ConvNet architecture for spatiotemporal fusion of video snippets, and evaluate its performance on standard benchmarks where this architecture achieves state-of-the-art results.read more
Citations
More filters
Proceedings ArticleDOI
Weakly Supervised Temporal Action Localization Using Deep Metric Learning
Ashraful Islam,Richard J. Radke +1 more
TL;DR: Li et al. as discussed by the authors proposed a weakly supervised temporal action localization method that only requires video-level action instances as supervision during training, and jointly optimize a balanced binary cross-entropy loss and a metric loss using a standard backpropagation algorithm.
Proceedings ArticleDOI
Driver Distraction Recognition using 3D Convolutional Neural Networks
TL;DR: This research has derived a benefit from temporal information by using a 3D convolutional neural network and optical flow to improve the driver distraction monitoring task and has shown that fine-tuning a pertained network on Kinetics dataset for learning driver action achieves a detection accuracy on State Farm dataset, which outperforms other methods on the same dataset.
Book ChapterDOI
Learning Discriminative Video Representations Using Adversarial Perturbations
Jue Wang,Anoop Cherian +1 more
TL;DR: This paper first generates adversarial noise adapted to a well-trained deep model for per-frame video recognition, then develops a binary classification problem that learns a set of discriminative hyperplanes – as a subspace – that will separate the two bags from each other.
Proceedings ArticleDOI
Modeling Sub-Event Dynamics in First-Person Action Recognition
TL;DR: A new dataset collected from YouTube is introduced which has a larger number of classes and a greater diversity of capture conditions thereby more closely depicting real-world challenges in first-person video analysis.
Proceedings Article
Action Recognition With Coarse-to-Fine Deep Feature Integration and Asynchronous Fusion
TL;DR: Zhang et al. as discussed by the authors proposed a coarse-to-fine network which extracts shared deep features at different action class granularities and progressively integrates them to obtain a more accurate feature representation for input actions.
References
More filters
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Proceedings ArticleDOI
Going deeper with convolutions
Christian Szegedy,Wei Liu,Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich +8 more
TL;DR: Inception as mentioned in this paper is a deep convolutional neural network architecture that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
Proceedings Article
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe,Christian Szegedy +1 more
TL;DR: Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.