The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the 5 years of the challenge, and propose future directions and improvements.

/pdf/imagenet-large-scale-visual-recognition-challenge-caky5nmmix.pdf

ImageNet Large Scale Visual Recognition Challenge

Prelude. Cycle 1. Introduction. Cycle 2. Structure defines function. Cycle 3. Diversity of cortical functions is provided by inhibition. Cycle 4. Windows on the brain. Cycle 5. A system of rhythms: from simple to complex dynamics. Cycle 6. Synchronization by oscillation. Cycle 7. The brain's default state: self-organized oscillations in rest and sleep. Cycle 8. Perturbation of the default patterns by experience. Cycle 9. The gamma buzz: gluing by oscillations in the waking brain. Cycle 10. Perceptions and actions are brain state-dependent. Cycle 11. Oscillations in the "other cortex:" navigation in real and memory space. Cycle 12. Coupling of systems by oscillations. Cycle 13. The tough problem. References.

Rhythms of the brain

Schultz, Wolfram. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80: 1–27, 1998. The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest t...

/pdf/predictive-reward-signal-of-dopamine-neurons-3f94myirza.pdf

Predictive Reward Signal of Dopamine Neurons

We propose a novel approach to learn and recognize natural scene categories. Unlike previous work, it does not require experts to annotate the training set. We represent the image of a scene by a collection of local regions, denoted as codewords obtained by unsupervised learning. Each region is represented as part of a "theme". In previous work, such themes were learnt from hand-annotations of experts, while our method learns the theme distributions as well as the codewords distribution over the themes without supervision. We report satisfactory categorization performances on a large set of 13 categories of complex scenes.

/pdf/a-bayesian-hierarchical-model-for-learning-natural-scene-zzqtm1es9k.pdf

A Bayesian hierarchical model for learning natural scene categories

Visual processing in cortex is classically modeled as a hierarchy of increasingly sophisticated representations, naturally extending the model of simple to complex cells of Hubel and Wiesel. Surprisingly, little quantitative modeling has been done to explore the biological feasibility of this class of models to explain aspects of higher-level visual processing such as object recognition. We describe a new hierarchical model consistent with physiological data from inferotemporal cortex that accounts for this complex visual task and makes testable predictions. The model is based on a MAX-like operation applied to inputs to certain cortical neurons that may have a general role in cortical function.

/pdf/hierarchical-models-of-object-recognition-in-cortex-33mmp5yhnv.pdf

Hierarchical models of object recognition in cortex

Single units and sensation: still just as relevant today.

Although it has been demonstrated that visual and auditory stimuli can be recalled decades after the initial exposure, previous studies have generally not ruled out the possibility that the material may have been seen or heard during the intervening period. Evidence shows that reactivations of a long-term memory trace play a role in its update and maintenance. In the case of remote or very long-term memories, it is most likely that these reactivations are triggered by the actual re-exposure to the stimulus. In this study we decided to explore whether it is possible to recall stimuli that could not have been re-experienced in the intervening period. We tested the ability of French participants (N = 34, 31 female) to recall 50 TV programs broadcast on average for the last time 44 years ago (from the 60’s and early 70’s). Potential recall was elicited by the presentation of short audiovisual excerpts of these TV programs. The absence of potential re-exposure to the material was strictly controlled by selecting TV programs that have never been rebroadcast and were not available in the public domain. Our results show that 6 TV programs were particularly well identified on average across the 34 participants with a median percentage of 71.7% (SD = 13.6, range: 48.5% - 87.9%). We also obtained 50 single case reports with associated information about the viewing of 23 TV programs including the 6 previous ones. More strikingly, for 2 cases, retrieval of the title was made spontaneously without the need of a four-proposition choice. These results suggest that re-exposures to the stimuli are not necessary to maintain a memory for a lifetime. These new findings raise fundamental questions about the underlying mechanisms used by the brain to store these very old sensory memories.

/pdf/waking-up-buried-memories-of-old-tv-programs-auy2501vwc.pdf

Waking Up Buried Memories of Old TV Programs.

Human observers readily detect targets and repetitions in streams of rapidly presented visual stimuli. It seems intuitive that regularly spaced repeating items should be easier to detect than irregularly spaced ones, since regularity adds predictability and in addition has ecological relevance. Here, we show that this is not necessarily the case, and we point out the intrinsic difficulty in addressing this question. We presented long RSVP streams of never-before-seen natural images containing repetition sequences; an image appearing six times interleaved by one or more non-repeating distractors, and asked participants to detect the repetitions and to afterwards identify the repeated images. We found that the ability to detect and memorize repeated images was preserved even with irregular sequences, and conclude that temporal regularity is not a key factor for detection and memory for repeating images in RSVP streams. These findings have implications for models of repetition processing.

/pdf/regularity-is-not-a-key-factor-for-encoding-repetition-in-1rrqbuxv0v.pdf

Regularity is not a key factor for encoding repetition in rapid image streams

Measuring the receptive field sizes of the mechanisms underlying ultra-rapid saccades to faces

Evidence for rapid neural computation in visual cortex are now available: Event related potentials in human visual system have revealed a frontal negativity specific to no-go trials involving complex shape categorization that develops roughly after 150 ms stimulus onset. Furthermore, in Vl, in the inferotemporal cortex, and in the superior temporal sulcus, cells can discriminate different complex visual stimuli within 5 to 10 ms. Given the anatomical and physiological constraints imposed by our knowledge of the visual system, such data have major implications for theories of visual computation. The suggestion that firing rate is rarely in excess of one spike in a 5-10 ms period in cortex emphasizes this situation. Cortical maps, which reflect a precise topographic neuronal organization where neighboring sites of maximum relative activity on the cortical plane correspond to nearby points in parameter space, might offer a neural substrate for rapid computation.

/pdf/rapid-neural-computation-using-single-spikes-and-time-1m4i7lk81k.pdf

Simon J. Thorpe

Papers

Single units and sensation: still just as relevant today.

Waking Up Buried Memories of Old TV Programs.

Regularity is not a key factor for encoding repetition in rapid image streams

Measuring the receptive field sizes of the mechanisms underlying ultra-rapid saccades to faces

Rapid neural computation using single spikes and time-dependent EPSP modulation