Receptive fields, binocular interaction and functional architecture in the cat's visual cortex
TL;DR: This method is used to examine receptive fields of a more complex type and to make additional observations on binocular interaction and this approach is necessary in order to understand the behaviour of individual cells, but it fails to deal with the problem of the relationship of one cell to its neighbours.
Abstract: What chiefly distinguishes cerebral cortex from other parts of the central nervous system is the great diversity of its cell types and interconnexions. It would be astonishing if such a structure did not profoundly modify the response patterns of fibres coming into it. In the cat's visual cortex, the receptive field arrangements of single cells suggest that there is indeed a degree of complexity far exceeding anything yet seen at lower levels in the visual system. In a previous paper we described receptive fields of single cortical cells, observing responses to spots of light shone on one or both retinas (Hubel & Wiesel, 1959). In the present work this method is used to examine receptive fields of a more complex type (Part I) and to make additional observations on binocular interaction (Part II). This approach is necessary in order to understand the behaviour of individual cells, but it fails to deal with the problem of the relationship of one cell to its neighbours. In the past, the technique of recording evoked slow waves has been used with great success in studies of functional anatomy. It was employed by Talbot & Marshall (1941) and by Thompson, Woolsey & Talbot (1950) for mapping out the visual cortex in the rabbit, cat, and monkey. Daniel & Whitteiidge (1959) have recently extended this work in the primate. Most of our present knowledge of retinotopic projections, binocular overlap, and the second visual area is based on these investigations. Yet the method of evoked potentials is valuable mainly for detecting behaviour common to large populations of neighbouring cells; it cannot differentiate functionally between areas of cortex smaller than about 1 mm2. To overcome this difficulty a method has in recent years been developed for studying cells separately or in small groups during long micro-electrode penetrations through nervous tissue. Responses are correlated with cell location by reconstructing the electrode tracks from histological material. These techniques have been applied to
Citations
More filters
[...]
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.
46,982 citations
01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Abstract: Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
42,067 citations
07 Jun 2015
TL;DR: Inception as mentioned in this paper is a deep convolutional neural network architecture that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
Abstract: We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.
40,257 citations
Book•
[...]
TL;DR: Deep learning as mentioned in this paper is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts, and it is used in many applications such as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames.
Abstract: Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
38,208 citations
TL;DR: This historical survey compactly summarizes relevant work, much of it from the previous millennium, review deep supervised learning, unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
14,635 citations
Cites background from "Receptive fields, binocular interac..."
...Simple cells and complex cells were found in the cat’s visual cortex (e.g., Hubel & Wiesel, 1962; Wiesel & Hubel, 1959)....
[...]
References
More filters
TL;DR: The present investigation, made in acute preparations, includes a study of receptive fields of cells in the cat's striate cortex, which resembled retinal ganglion-cell receptive fields, but the shape and arrangement of excitatory and inhibitory areas differed strikingly from the concentric pattern found in retinalganglion cells.
Abstract: In the central nervous system the visual pathway from retina to striate cortex provides an opportunity to observe and compare single unit responses at several distinct levels. Patterns of light stimuli most effective in influencing units at one level may no longer be the most effective at the next. From differences in responses at successive stages in the pathway one may hope to gain some understanding of the part each stage plays in visual perception. By shining small spots of light on the light-adapted cat retina Kuffler (1953) showed that ganglion cells have concentric receptive fields, with an 'on' centre and an 'off ' periphery, or vice versa. The 'on' and 'off' areas within a receptive field were found to be mutually antagonistic, and a spot restricted to the centre of the field was more effective than one covering the whole receptive field (Barlow, FitzHugh & Kuffler, 1957). In the freely moving lightadapted cat it was found that the great majority of cortical cells studied gave little or no response to light stimuli covering most of the animal's visual field, whereas small spots shone in a restricted retinal region often evoked brisk responses (Hubel, 1959). A moving spot of light often produced stronger responses than a stationary one, and sometimes a moving spot gave more activation for one direction than for the opposite. The present investigation, made in acute preparations, includes a study of receptive fields of cells in the cat's striate cortex. Receptive fields of the cells considered in this paper were divided into separate excitatory and inhibitory ('on' and 'off') areas. In this respect they resembled retinal ganglion-cell receptive fields. However, the shape and arrangement of excitatory and inhibitory areas differed strikingly from the concentric pattern found in retinal ganglion cells. An attempt was made to correlate responses to moving stimuli
4,405 citations
"Receptive fields, binocular interac..." refers background or result in this paper
...2E (see also Hubel & Wiesel, 1959, Fig....
[...]
...The presence in the striate cortex of cells influenced from both eyes has already been observed by several authors (Hubel & Wiesel, 1959; Cornelils & Gruisser, 1959; Burns, Heron & Grafstein, 1960), and is confirmed in Part II of this paper....
[...]
...2, the time elapsing between the two discharges to a moving stimulus was a measure of the rate of movement (see Hubel & Wiesel, 1959, Fig....
[...]
...The optimum response was evoked by simultaneously illuminating the two flanks with two parallel slits (see Hubel & Wiesel, 1959, Fig....
[...]
...Moving stimuli were very effective, probably because of the synergistic effects of leaving an inhibitory area and simultaneously entering an excitatory area (Hubel & Wiesel, 1959)....
[...]
TL;DR: The Limulus preparation shows many features which are similar to other simple sense organs, for instance, stretch receptors, however, instead of photochemical events, stretch-deformation acts as the adequate stimulus on sensory terminals and is translated into a characteristic discharge pattern.
Abstract: THE DISCHARGES carried in the optic nerve fibers contain all the information which the central nervous system receives from the retina. A correct interpretation of discharge patterns therefore constitutes an important step in the analysis of visual events. Further, investigations of nervous activity arising in the eye reveal many aspects of the functional organization of the neural elements within the retina itself. Following studies of discharges in the optic nerve of the eel’s eye by Adrian and Matthews (2,3), Hartline and his colleagues described the discharge pattern in the eye of the Limulus in a series of important and lucid papers (for a summary see 20). In the Limulus the relationship between the stimulus to the primary receptor cell and the nerve discharges proved relatively simple, apparently because the connection between sense cell and nerve fiber was a direct one. Thus, when stimulation is confined to one receptor the discharge in a single Limulus nerve fiber will provide a good indication of excitatory events which take place as a result of photochemical processes. Discharges last for the duration of illumination and their frequency is a measure of stimulus strength. Lately, however, it was shown by Hartline et al. (22) that inhibitory interactions may be revealed when several receptors are excited. On the whole, the Limulus preparation shows many features which are similar to other simple sense organs, for instance, stretch receptors. In the latter, however, instead of photochemical events, stretch-deformation acts as the adequate stimulus on sensory terminals and is translated into a characteristic discharge pattern. The discharge from the cold-blooded vertebrate retina (mainly frogs) proved much more complex. Hartline found three main types when recording from single optic nerve fibers: (i) “on” discharges, similar to those in the Limulus, firing for the duration of the light stimulus, (ii) “off” discharges appearing when a light stimulus was withdrawn, and (iii) ‘con-off” discharges, a combination of the former two, with activity confined mainly to onset and cessation of illumination. The mammalian discharge patterns were studied in a number of species by Granit and his co-workers in the course of their extensive work on the physiology of the visual system (summaries in 13, 15). On the whole, they did not observe any fundamental differences between frog and mammalian discharge types (see later).
2,540 citations
TL;DR: Observations upon the modality and topographical attributes of single neurons of the first somatic sensory area of the cat’s cerebral cortex, the analogue of the cortex of the postcentral gyrus in the primate brain, support an hypothesis of the functional organization of this cortical area.
Abstract: THE PRESENT PAPER describes some observations upon the modality and topographical attributes of single neurons of the first somatic sensory area of the cat’s cerebral cortex, the analogue of the cortex of the postcentral gyrus in the primate brain. These data, together with others upon the response latencies of the cells of different layers of the cortex to peripheral stimuli, support an hypothesis of the functional organization of this cortical area. This is that the neurons which lie in narrow vertical columns, or cylinders, extending from layer II through layer VI make up an elementary unit of organization, for they are activated by stimulation of the same single class of peripheral receptors, from almost identical peripheral receptive fields, at latencies ers. It is early These which are not significantly different for the cells of the various layemphasized that this pattern of organization obtains only for the repetitiv neurons ‘e responses may be rela of ted cortical in quite neurons different to brief peripheral stimuli. organization patterns when analyzed in terms of later discharges. A report of these experiments was made to the American Physiological Society in September, 1955 (10, 17).
2,230 citations
"Receptive fields, binocular interac..." refers background in this paper
...Functional cytoarchitecture There is an interesting parallel between the functional subdivisions of the cortex described in the present paper, and those found in somatosensory cortex by Mountcastle (1957) in the cat, and by Powell & Mountcastle (1959) in the monkey....
[...]
01 Nov 1959
TL;DR: The results show that for the most part within that area of the optic nerve of a frog, it is not the light intensity itself but rather the pattern of local variation of intensity that is the exciting factor.
Abstract: In this paper, we analyze the activity of single fibers in the optic nerve of a frog. Our method is to find what sort of stimulus causes the largest activity in one nerve fiber and then what is the exciting aspect of that stimulus such that variations in everything else cause little change in the response. It has been known for the past 20 years that each fiber is connected not to a few rods and cones in the retina but to very many over a fair area. Our results show that for the most part within that area, it is not the light intensity itself but rather the pattern of local variation of intensity that is the exciting factor. There are four types of fibers, each type concerned with a different sort of pattern. Each type is uniformly distributed over the whole retina of the frog. Thus, there are four distinct parallel distributed channels whereby the frog's eye informs his brain about the visual image in terms of local pattern independent of average illumination. We describe the patterns and show the functional and anatomical separation of the channels. This work has been done on the frog, and our interpretation applies only to the frog.
1,788 citations
784 citations