The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the 5 years of the challenge, and propose future directions and improvements.

/pdf/imagenet-large-scale-visual-recognition-challenge-caky5nmmix.pdf

ImageNet Large Scale Visual Recognition Challenge

Prelude. Cycle 1. Introduction. Cycle 2. Structure defines function. Cycle 3. Diversity of cortical functions is provided by inhibition. Cycle 4. Windows on the brain. Cycle 5. A system of rhythms: from simple to complex dynamics. Cycle 6. Synchronization by oscillation. Cycle 7. The brain's default state: self-organized oscillations in rest and sleep. Cycle 8. Perturbation of the default patterns by experience. Cycle 9. The gamma buzz: gluing by oscillations in the waking brain. Cycle 10. Perceptions and actions are brain state-dependent. Cycle 11. Oscillations in the "other cortex:" navigation in real and memory space. Cycle 12. Coupling of systems by oscillations. Cycle 13. The tough problem. References.

Rhythms of the brain

Schultz, Wolfram. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80: 1–27, 1998. The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest t...

/pdf/predictive-reward-signal-of-dopamine-neurons-3f94myirza.pdf

Predictive Reward Signal of Dopamine Neurons

We propose a novel approach to learn and recognize natural scene categories. Unlike previous work, it does not require experts to annotate the training set. We represent the image of a scene by a collection of local regions, denoted as codewords obtained by unsupervised learning. Each region is represented as part of a "theme". In previous work, such themes were learnt from hand-annotations of experts, while our method learns the theme distributions as well as the codewords distribution over the themes without supervision. We report satisfactory categorization performances on a large set of 13 categories of complex scenes.

/pdf/a-bayesian-hierarchical-model-for-learning-natural-scene-zzqtm1es9k.pdf

A Bayesian hierarchical model for learning natural scene categories

Visual processing in cortex is classically modeled as a hierarchy of increasingly sophisticated representations, naturally extending the model of simple to complex cells of Hubel and Wiesel. Surprisingly, little quantitative modeling has been done to explore the biological feasibility of this class of models to explain aspects of higher-level visual processing such as object recognition. We describe a new hierarchical model consistent with physiological data from inferotemporal cortex that accounts for this complex visual task and makes testable predictions. The model is based on a MAX-like operation applied to inputs to certain cortical neurons that may have a general role in cortical function.

/pdf/hierarchical-models-of-object-recognition-in-cortex-33mmp5yhnv.pdf

Hierarchical models of object recognition in cortex

Recent experimental work has shown that the primate visual system can analyze complex natural scenes in only 100-150 ms. Such data, when combined with anatomical and physiological knowledge, seriously constrains current models of visual processing. In particular, it suggests that a lot of processing can be achieved using a single feed-forward pass through the visual system, and that each processing layer probably has no more than around 10 ms before the next stage has to respond. In this time, few neurons will have generated more than one spike, ruling out most conventional rate coding models. We have been exploring the possibility of using the fact that strongly activated neurons tend to fire early and that information can be encoded in the order in which a population of cells fire. These ideas have been tested using SpikeNet, a computer program that simulates the activity of very large networks of asynchronously firing neurons. The results have been extremely promising, and we have been able to develop artificial visual systems capable of processing complex natural scenes in real time using standard computer hardware (see http://www.spikenet-technology.com).

Ultra-Rapid Scene Categorization with a Wave of Spikes

Navigation, especially in unknown areas, remains a major problem for the visually impaired (VI). Over the past 50 years, a number of electronic travel aids (ETAs) have been developed with the aim of improving the mobility of the VI. Despite the efforts, these systems are rarely used. Although the explanation is likely to be incomplete, it is possible to identify three important factors: (1) positioning accuracy provided by these devices is not sufficient to guide a VI pedestrian, (2) these systems are based on Geographical Information Systems not adapted to pedestrian mobility, and (3) the guidance methods should be adapted to the task of pedestrian navigation. The NAVIG project aims to answer all these limitations through a participatory design framework with the VI and orientation and mobility instructors. The NAVIG device aims to complement conventional mobility aids (i.e. white cane and guide dog), while also adding unique features to localize specific objects in the environment, restore some visuomotor abilities, and assist navigation.

Navigation and space perception assistance for the visually impaired: The NAVIG project

Responses of neurons in area 7 of the parietal cortex to objects of different significance

We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary Spike-Timing-Dependent-Plasticity learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10 and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods.

Unsupervised Feature Learning With Winner-Takes-All Based STDP.

We have investigated the possibility that rapid processing in the visual system could be achieved by using the order of firing in different neurones as a code, rather than more conventional firing rate schemes. Using SPIKENET, a neural net simulator based on integrate-and-fire neurones and in which neurones in the input layer function as analog-to-delay converters, we have modeled the initial stages of visual processing. Initial results are extremely promising. Even with activity in retinal output cells limited to one spike per neuron per image (effectively ruling out any form of rate coding), sophisticated processing based on asynchronous activation was nonetheless possible.

/pdf/rapid-visual-processing-using-spike-asynchrony-3r94qudl98.pdf

Simon J. Thorpe

Papers

Ultra-Rapid Scene Categorization with a Wave of Spikes

Navigation and space perception assistance for the visually impaired: The NAVIG project

Responses of neurons in area 7 of the parietal cortex to objects of different significance

Unsupervised Feature Learning With Winner-Takes-All Based STDP.

Rapid Visual Processing using Spike Asynchrony