The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called “ImageNet”, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.

/pdf/imagenet-a-large-scale-hierarchical-image-database-1e8f663s1h.pdf

ImageNet: A large-scale hierarchical image database

The PASCAL Visual Object Classes Challenge

While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models. Lagging behind the growth in model capacity, the available datasets are quickly becoming outdated in terms of size and density. To circumvent this bottleneck, we propose to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop. Starting from a large set of candidate images for each category, we iteratively sample a subset, ask people to label them, classify the others with a trained model, split the set into positives, negatives, and unlabeled based on the classification confidence, and then iterate with the unlabeled set. To assess the effectiveness of this cascading procedure and enable further progress in visual recognition research, we construct a new image dataset, LSUN. It contains around one million labeled images for each of 10 scene categories and 20 object categories. We experiment with training popular convolutional networks and find that they achieve substantial performance gains when trained on this dataset.

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research.

AutoML: A survey of the state-of-the-art

The problem of road or lane perception is a crucial enabler for advanced driver assistance systems. As such, it has been an active field of research for the past two decades with considerable progress made in the past few years. The problem was confronted under various scenarios, with different task definitions, leading to usage of diverse sensing modalities and approaches. In this paper we survey the approaches and the algorithmic techniques devised for the various modalities over the last 5 years. We present a generic break down of the problem into its functional building blocks and elaborate the wide range of proposed methods within this scheme. For each functional block, we describe the possible implementations suggested and analyze their underlying assumptions. While impressive advancements were demonstrated at limited scenarios, inspection into the needs of next generation systems reveals significant gaps. We identify these gaps and suggest research directions that may bridge them.

Recent progress in road and lane detection: a survey

As computer vision research considers more object categories and greater variation within object categories, it is clear that larger and more exhaustive datasets are necessary. However, the process of collecting such datasets is laborious and monotonous. We consider the setting in which many images have been automatically collected for a visual category (typically by automatic internet search), and we must separate relevant images from noise. We present a discriminative learning process which employs active, online learning to quickly classify many images with minimal user input. The principle advantage of this work over previous endeavors is its scalability. We demonstrate precision which is often superior to the state-of-the-art, with scalability which exceeds previous work.

/pdf/towards-scalable-dataset-construction-an-active-learning-3vxxhqt85u.pdf

Towards Scalable Dataset Construction: An Active Learning Approach

DARPA Urban Challenge Princeton University Technical Paper

Stereo Vision for Obstacle Detection in Autonomous Navigation

This paper describes Princeton University’s approach to the 2005 DARPA Grand Challenge, an off-road race for fully autonomous ground vehicles. The system, Prospect Eleven, takes a simple approach to address the problems posed by the Grand Challenge including obstacle detection, path planning, and extended operation in harsh environments. Obstacles are detected using stereo vision, and tracked in the time domain to improve accuracy in localization and reduce false positives. The navigation system processes a geometric representation of the world to identify passable regions in the terrain ahead, and the vehicle is controlled to drive through these regions. Performance of the system is evaluated both during the Grand Challenge and in subsequent desert testing. The vehicle completed 9.3 miles of the course on race day, and extensive portions of the 2004 and 2005 Grand Challenge courses in later tests.

Prospect Eleven: Princeton University's entry in the 2005 DARPA Grand Challenge

This paper describes Princeton University's approach to the 2005 DARPA Grand Challenge, an off-road race for fully autonomous ground vehicles. The system, Prospect Eleven, takes a simple approach to address the problems posed by the Grand Challenge, including obstacle detection, path planning, and extended operation in harsh environments. Obstacles are detected using stereo vision, and tracked in the time domain to improve accuracy in localization and reduce false positives. The navigation system processes a geometric representation of the world to identify passable regions in the terrain ahead, and the vehicle is controlled to drive through these regions. Performance of the system is evaluated both during the Grand Challenge and in subsequent desert testing. The vehicle completed 9.3 miles of the course on race day, and extensive portions of the 2004 and 2005 Grand Challenge courses in later tests. © 2006 Wiley Periodicals, Inc.

Brendan M. Collins

Papers

Towards Scalable Dataset Construction: An Active Learning Approach

DARPA Urban Challenge Princeton University Technical Paper

Stereo Vision for Obstacle Detection in Autonomous Navigation

Prospect Eleven: Princeton University's entry in the 2005 DARPA Grand Challenge

Prospect Eleven: Princeton University's entry in the 2005 DARPA Grand Challenge: Field Reports