Recent advances in convolutional neural networks

The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object’s appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.

/pdf/fully-convolutional-siamese-networks-for-object-tracking-y39l8kt2ze.pdf

Fully-Convolutional Siamese Networks for Object Tracking

We tackle the problem of large scale visual place recognition, where the task is to quickly and accurately recognize the location of a given query photograph. We present the following three principal contributions. First, we develop a convolutional neural network (CNN) architecture that is trainable in an end-to-end manner directly for the place recognition task. The main component of this architecture, NetVLAD, is a new generalized VLAD layer, inspired by the "Vector of Locally Aggregated Descriptors" image representation commonly used in image retrieval. The layer is readily pluggable into any CNN architecture and amenable to training via backpropagation. Second, we develop a training procedure, based on a new weakly supervised ranking loss, to learn parameters of the architecture in an end-to-end manner from images depicting the same places over time downloaded from Google Street View Time Machine. Finally, we show that the proposed architecture significantly outperforms non-learnt image representations and off-the-shelf CNN descriptors on two challenging place recognition benchmarks, and improves over current state of-the-art compact image representations on standard image retrieval benchmarks.

/pdf/netvlad-cnn-architecture-for-weakly-supervised-place-jj18zqn7c8.pdf

NetVLAD: CNN Architecture for Weakly Supervised Place Recognition

The field of object detection has made significant advances riding on the wave of region-based ConvNets, but their training procedure still includes many heuristics and hyperparameters that are costly to tune. We present a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region-based ConvNet detectors. Our motivation is the same as it has always been – detection datasets contain an overwhelming number of easy examples and a small number of hard examples. Automatic selection of these hard examples can make training more effective and efficient. OHEM is a simple and intuitive algorithm that eliminates several heuristics and hyperparameters in common use. But more importantly, it yields consistent and significant boosts in detection performance on benchmarks like PASCAL VOC 2007 and 2012. Its effectiveness increases as datasets become larger and more difficult, as demonstrated by the results on the MS COCO dataset. Moreover, combined with complementary advances in the field, OHEM leads to state-of-the-art results of 78.9% and 76.3% mAP on PASCAL VOC 2007 and 2012 respectively.

/pdf/training-region-based-object-detectors-with-online-hard-3xv7sk7ef9.pdf

Training Region-Based Object Detectors with Online Hard Example Mining

The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object's appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.

Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

/pdf/discriminative-learning-of-deep-convolutional-feature-point-qbh8u316j4.pdf

Discriminative Learning of Deep Convolutional Feature Point Descriptors

We propose a real-time, robust to outliers and accurate solution to the Perspective-n-Point (PnP) problem. The main advantages of our solution are twofold: first, it in- tegrates the outlier rejection within the pose estimation pipeline with a negligible computational overhead, and sec- ond, its scalability to arbitrarily large number of correspon- dences. Given a set of 3D-to-2D matches, we formulate pose estimation problem as a low-rank homogeneous sys- tem where the solution lies on its 1D null space. Outlier correspondences are those rows of the linear system which perturb the null space and are progressively detected by projecting them on an iteratively estimated solution of the null space. Since our outlier removal process is based on an algebraic criterion which does not require computing the full-pose and reprojecting back all 3D points on the image plane at each step, we achieve speed gains of more than 100× compared to RANSAC strategies. An extensive exper- imental evaluation will show that our solution yields accu- rate results in situations with up to 50% of outliers, and can process more than 1000 correspondences in less than 5ms.

/pdf/very-fast-solution-to-the-pnp-problem-with-algebraic-outlier-2n932biqm0.pdf

Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection

In this paper we propose a novel framework for learning local image descriptors in a discriminative manner For this purpose we explore a siamese architecture of Deep Convolutional Neural Networks (CNN), with a Hinge embedding loss on the L2 distance between descriptors Since a siamese architecture uses pairs rather than single image patches to train, there exist a large number of positive samples and an exponential number of negative samples We propose to explore this space with a stochastic sampling of the training set, in combination with an aggressive mining strategy over both the positive and negative samples which we denote as "fracking" We perform a thorough evaluation of the architecture hyper-parameters, and demonstrate large performance gains compared to both standard CNN learning strategies, hand-crafted image descriptors like SIFT, and the state-of-the-art on learned descriptors: up to 25x vs SIFT and 15x vs the state-of-the-art in terms of the area under the curve (AUC) of the Precision-Recall curve

Fracking Deep Convolutional Image Descriptors.

Trabajo presentado a la 25th British Machine Vision Conference (BMVC), celebrada en Nottingham (UK) del 1 al 5 de septiembre de 2014-- Este item (excepto textos e imagenes no creados por el autor) esta sujeto a una licencia de Creative Commons: Attribution-NonCommercial-NoDerivs 30 Spain

http://www.bmva.org/bmvc/2014/files/paper065.pdf

Leveraging feature uncertainty in the PnP problem

Millimetric Waves Images (MMW) are becoming more and more useful in the passive detection of threaten objects based on plastic substances as explosives or sharp/cutting weapons. Our goal is to achieve segmentation of the body and concealed threats dealing with the inherent problems of this type of images: noise, low resolution and intensity inhomogeneity. In this work we present the results of applying Iterative Steering Kernel Regression (ISKR) method for denoising and Local Binary Fitting (LBF) for segmentation in order to correctly segment bodies and threats over a database of 29 MMW images. These methods, which had not been tested in the literature with these type of images, are compared with previously applied state of the art methods. Experimental results show that the use of the proposed methods in MMW images improve the results that had been obtained before.

Luis Ferraz

Papers

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection

Fracking Deep Convolutional Image Descriptors.

Leveraging feature uncertainty in the PnP problem

Concealed object detection and segmentation over millimetric waves images