We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.

/pdf/microsoft-coco-common-objects-in-context-55ihxq1zt5.pdf

Microsoft COCO: Common Objects in Context

We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is less likely to predict false positives on background. Finally, YOLO learns very general representations of objects. It outperforms other detection methods, including DPM and R-CNN, when generalizing from natural images to other domains like artwork.

/pdf/you-only-look-once-unified-real-time-object-detection-3gan6roe1o.pdf

You Only Look Once: Unified, Real-Time Object Detection

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

/pdf/rich-feature-hierarchies-for-accurate-object-detection-and-22fvjlggyg.pdf

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But pyramid representations have been avoided in recent object detectors that are based on deep convolutional networks, partially because they are slow to compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.

/pdf/feature-pyramid-networks-for-object-detection-4b1j4aq4vv.pdf

Feature Pyramid Networks for Object Detection

Given two or more video sequences containing similar human activities (running, jumping, etc.), we want to devise a method which extracts spatio-temporal signatures associated with these activities, compares these signatures, and aligns key positions from different videos. We introduce a method which, in conjunction with a number of hypotheses, allows the analysis of the motion of specific body parts and extracts their 2D (image plane) time-varying trajectories as well as their 3D trajectories. Two such trajectories recovered from two different videos have different characteristics. We develop a curve registration technique which consists of estimating a transformation which maps one time-basis (of the first curve) onto another time-basis (the second curve). We also analyse in depth the conditions under which such curve registration techniques are valid. Finally, we show results with two similar athletic events performed by two different athletes.

https://hal.inria.fr/inria-00590163/document

Indexing key positions between multiple videos

Software - Histogram of Oriented Gradient Object Detection Toolkit

Systems and methods are disclosed for gesture-initiated actions in videoconferences In one implementation, a processing device receives one or more content streams as part of a communication session The processing device identifies, within the one or more content streams, a request for feedback The processing device processes, based on an identification of a request for feedback within the one of the plurality of content streams, the one or more content streams to identify a presence of one or more gestures within at least one of the one or more content streams The processing device initiates, based on an identification of the presence of one or more gestures within at least one of the one or more content streams, an action with respect to the communication session

Gesture-initiated actions in videoconferences

A method and apparatus for the detection of local image structures represented as clusters in a joint-spatial range domain where the method comprises receiving an input image made having one or more clusters in a joint-spatial range domain, and each of the one or more clusters having a corresponding mode. Receiving a set of analysis matrices and selecting through each one of the analysis matrices. Using the selected analysis matrix to partition the input image into the one or more clusters and their corresponding modes, and computing a mean, μ, and a local covariance matrix Σ for each of the corresponding modes of each of the one or more clusters. Selecting at least one of the one or more clusters, where each selected cluster has a stable mean and stable covariance matrix across the set of analysis matrices, whereby each of the selected clusters is indicative of a local image structure.

Multiscale detection of local image structures

La construction d’un panorama est possible à partir d’une séquence vidéo provenant d’une caméra subissant un mouvement de rotation pure. Un panorama constitue une représentation “minimale” de la séquence vidéo. Les approches existantes permettent de traiter efficacement les séquences vidéo de scènes statiques. Dans ce travail, nous faisons l’hypothèse que la scène observée est constituée d’un fond statique et d’une couche dynamique. Nous proposons un cadre pour la construction de panoramas de mouvement, permettant de représenter de telles séquences de manière efficace.

Navneet Dalal

Papers

Indexing key positions between multiple videos

Software - Histogram of Oriented Gradient Object Detection Toolkit

Gesture-initiated actions in videoconferences

Multiscale detection of local image structures

Des séquences vidéo aux panoramas de mouvement