The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection.

This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history of the challenge, and proposes directions for future improvement and extension.

/pdf/the-pascal-visual-object-classes-voc-challenge-ugvl0p2h51.pdf

The Pascal Visual Object Classes (VOC) Challenge

We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/ .

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

This paper contributes a new high quality dataset for person re-identification, named "Market-1501". Generally, current datasets: 1) are limited in scale, 2) consist of hand-drawn bboxes, which are unavailable under realistic settings, 3) have only one ground truth and one query image for each identity (close environment). To tackle these problems, the proposed Market-1501 dataset is featured in three aspects. First, it contains over 32,000 annotated bboxes, plus a distractor set of over 500K images, making it the largest person re-id dataset to date. Second, images in Market-1501 dataset are produced using the Deformable Part Model (DPM) as pedestrian detector. Third, our dataset is collected in an open system, where each identity has multiple images under each camera. As a minor contribution, inspired by recent advances in large-scale image search, this paper proposes an unsupervised Bag-of-Words descriptor. We view person re-identification as a special task of image search. In experiment, we show that the proposed descriptor yields competitive accuracy on VIPeR, CUHK03, and Market-1501 datasets, and is scalable on the large-scale 500k dataset.

/pdf/scalable-person-re-identification-a-benchmark-iiee63lja2.pdf

Scalable Person Re-identification: A Benchmark

We propose a new method for learning to segment objects in images. This method is based on a latent variables model used for representing images and objects, inspired by the LDA model. Like the LDA model, our model is capable of automatically discovering which visual information comes from which object. We extend LDA by considering that images are made of multiple overlapping regions, treated as distinct documents, giving more chance to small objects to be discovered. This model is extremely well suited for assigning image patches to objects (even if they are small), and therefore for segmenting objects. We apply this method on objects belonging to categories with high intra-class variations and strong viewpoint changes.

https://hal.inria.fr/inria-00548681/document

CATEGORY LEVEL OBJECT SEGMENTATION - Learning to Segment Objects with Latent Aspect Models

There is a general trend in recent methods to use image regions (i.e. super-pixels) obtained in an unsupervised way to enhance the semantic image segmentation task. This paper proposes a detailed study on the role and the benefit of using these regions, at different steps of the segmentation process. For the purpose of this benchmark, we propose a simple system for semantic segmentation that uses a hierarchy of regions. A patch based system with similar settings is compared, which allows us to evaluate the contribution of each component of the system. Both systems are evaluated on the standard MSRC-21 dataset and obtain competitive results. We show that the proposed region based system can achieve good results without any complex regularization, while its patch based counterpart becomes competitive when using image prior and regularization methods. The latter benefit more from a CRF based regularization, yielding to state-of-the-art results with simple constraints based only on the leaf regions exploited in the pairwise potential.

On the use of regions for semantic image segmentation

This paper presents an approach to segment unseen objects of known categories. At the heart of the approach lies a probabilistic model of images which captures local appearance of objects through a bag-of-words representation. Bag-of-words models have been very successful for image categorization; however, as they model objects as loose collections of small image patches, they can not accurately predict object boundaries. On the other hand, Markov Random Fields (MRFs), which are often used in many low-level application for general purpose image segmentation, do incorporate the spatial layout of images. Yet, as they are usually based on very local image evidence they fail to capture larger scale structures needed to recognize object categories under large appearance variations. The main contribution of this article is to combine the advantages of both approaches into a single probabilistic model. First, a mechanism based on a bag-of-words representation produces object recognition and localization at a rough spatial resolution. Second, a MRF component enforces precise object boundaries, guided by local image cues (color, texture, and edges) and by long-distance dependencies. Gibbs sampling is used to infer the model parameters and the object segmentation. The proposed method successfully segments object categories, despite highly varying appearances, cluttered backgrounds and large viewpoint changes. Through a series of experiments, we emphasize the strength as well as the limitation of our model. First, we evaluate the results of several strategies for building the visual vocabulary. Second, we show how it is possible to combine strong labeling (segmented images) with weak labeling (images annotated with bounding boxes), in order to limit the amount of training data needed to learn the model. Third, we study the influence of the initialization on the model estimation. Last, we present extensive experiments on four different image databases, including the challenging Pascal VOC 2007 dataset on which we obtain state-of-the art results.

Category Level Object Segmentation by Combining Bag-of-words Models and Markov Random Fields

We consider the problem of training a deep neural network on a given classification task, e.g., ImageNet-1K (IN1K), so that it excels at that task as well as at other (future) transfer tasks. These two seemingly contradictory properties impose a trade-off between improving the model’s generalization while maintaining its performance on the original task. Models trained with self-supervised learning (SSL) tend to generalize better than their supervised counterparts for transfer learning; yet, they still lag behind supervised models on IN1K. In this paper, we propose a supervised learning setup that leverages the best of both worlds. We enrich the common supervised training framework using two key components of recent SSL models: multi-scale crops for data augmentation and the use of an expendable projector head. We replace the last layer of class weights with class prototypes computed on the fly using a memory bank. We show that these three improvements lead to a more favorable trade-off between the IN1K training task and 13 transfer tasks. Over all the explored configurations, we single out two models: t-ReX that achieves a new state of the art for transfer learning and outperforms top methods such as DINO and PAWS on IN1K, and t-ReX* that matches the highly optimized RSB-A1 model on IN1K while performing better on transfer tasks.

Improving the Generalization of Supervised Models

We consider the problem of training a deep neural network on a given classification task, e.g., ImageNet-1K (IN1K), so that it excels at both the training task as well as at other (future) transfer tasks. These two seemingly contradictory properties impose a trade-off between improving the model's generalization and maintaining its performance on the original task. Models trained with self-supervised learning tend to generalize better than their supervised counterparts for transfer learning; yet, they still lag behind supervised models on IN1K. In this paper, we propose a supervised learning setup that leverages the best of both worlds. We extensively analyze supervised training using multi-scale crops for data augmentation and an expendable projector head, and reveal that the design of the projector allows us to control the trade-off between performance on the training task and transferability. We further replace the last layer of class weights with class prototypes computed on the fly using a memory bank and derive two models: t-ReX that achieves a new state of the art for transfer learning and outperforms top methods such as DINO and PAWS on IN1K, and t-ReX* that matches the highly optimized RSB-A1 model on IN1K while performing better on transfer tasks. Code and pretrained models: https://europe.naverlabs.com/t-rex

Diane Larlus

Papers

CATEGORY LEVEL OBJECT SEGMENTATION - Learning to Segment Objects with Latent Aspect Models

On the use of regions for semantic image segmentation

Category Level Object Segmentation by Combining Bag-of-words Models and Markov Random Fields

Improving the Generalization of Supervised Models

No Reason for No Supervision: Improved Generalization in Supervised Models