A theory of edge detection is presented. The analysis proceeds in two parts. (1) Intensity changes, which occur in a natural image over a wide range of scales, are detected separately at different scales. An appropriate filter for this purpose at a given scale is found to be the second derivative of a Gaussian, and it is shown that, provided some simple conditions are satisfied, these primary filters need not be orientation-dependent. Thus, intensity changes at a given scale are best detected by finding the zero values of delta 2G(x,y)*I(x,y) for image I, where G(x,y) is a two-dimensional Gaussian distribution and delta 2 is the Laplacian. The intensity changes thus discovered in each of the channels are then represented by oriented primitives called zero-crossing segments, and evidence is given that this representation is complete. (2) Intensity changes in images arise from surface discontinuities or from reflectance or illumination boundaries, and these all have the property that they are spatially. Because of this, the zero-crossing segments from the different channels are not independent, and rules are deduced for combining them into a description of the image. This description is called the raw primal sketch. The theory explains several basic psychophysical findings, and the operation of forming oriented zero-crossing segments from the output of centre-surround delta 2G filters acting on the image forms the basis for a physiological model of simple cells (see Marr & Ullman 1979).

http://www.hms.harvard.edu/bss/neuro/bornlab/qmbc/beta/day4/marr-hildreth-edge-prsl1980.pdf

Theory of Edge Detection

Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1 ?

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

/pdf/spatiotemporal-energy-models-for-the-perception-of-motion-43mzze32mc.pdf

Spatiotemporal energy models for the perception of motion

Theoretical neuroscience provides a quantitative basis for describing what nervous systems do, determining how they function, and uncovering the general principles by which they operate This text introduces the basic mathematical and computational methods of theoretical neuroscience and presents applications in a variety of areas including vision, sensory-motor integration, development, learning, and memory The book is divided into three parts Part I discusses the relationship between sensory stimuli and neural responses, focusing on the representation of information by the spiking activity of neurons Part II discusses the modeling of neurons and neural circuits on the basis of cellular and synaptic biophysics Part III analyzes the role of plasticity in development and learning An appendix covers the mathematical methods used, and exercises are available on the book's Web site

Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems

The statistical reliability of signals in single neurons in cat and monkey visual cortex.

1. We have examined the responses of simple cells in the cat's atriate cortex to visual patterns that were designed to reveal the extent to which these cells may be considered to sum light-evoked influences linearly across their receptive fields. We used one-dimensional luminance-modulated bars and grating as stimuli; their orientation was always the same as the preferred orientation of the neurone under study. The stimuli were presented on an oscilloscope screen by a digital computer, which also accumulated neuronal responses and controlled a randomized sequence of stimulus presentations. 2. The majority of simple cells respond to sinusoidal gratings that are moving or whose contrast is modulated in time in a manner consistent with the hypothesis that they have linear spatial summation. Their responses to moving gratings of all spatial frequencies are modulated in synchrony with the passage of the gratings' bars across their receptive fields, and they do not produce unmodulated responses even at the highest spatial frequencies. Many of these cells respond to temporally modulated stationary gratings simply by changing their response amplitude sinusoidally as the spatial phase of the grating the grating is varied. Nonetheless, their behavior appears to indicate linear spatial summation, since we show in an Appendix that the absence of a 'null' phase in a visual neurone need not indicate non-linear spatial summation, and further that a linear neurone lacking a 'null' phase should give responses of the form that we have observed in this type of simple cell. 3. A minority of simple cells appears to have significant non-linearities of spatial summation. These neurones respond to moving gratings of high spatial frequency with a partially or totally unmodulated elevation of firing rate. They have no 'null' phases when tested with stationary gratings, and reveal their non-linearity by giving responses to gratings of some spatial phases that are composed partly or wholly of even harmonics of the stimulus frequency ('on-off' responses). 4. We compared simple receptive fields with their sensitivity to sinusoidal gratings of different spatial frequencies. Qualitatively, the most sensitive subregions of simple cells' receptive fields are roughly the same width as the individual bars of the gratings to which they are most sensitive. Quantitatively, their receptive field profiles measured with thin stationary lines, agree well with predicted profiles derived by Fourier synthesis of their spatial frequency tuning curves.

Spatial summation in the receptive fields of simple cells in the cat's striate cortex.

1. The sensitivity to temporally modulated sinusoidal gratings was determined. Two thresholds could be distinguished for the modulated gratings: the contrast at which flicker could be perceived and the contrast at which the spatial structure became distinct.2. The flicker detection thresholds and pattern recognition threshold varied independently as functions of the spatial and temporal frequencies, suggesting that the two thresholds represent the activity of two independent systems of channels.3. The channels detecting flicker prefer low and medium spatial frequencies. They have a pronounced decline in sensitivity at low temporal frequencies of sinusoidal modulation. They respond twice as well to gratings whose phase is alternated repetitively as to gratings turned on and off at the same rate.4. The channels responsible for the discrimination of spatial structure are most responsive at high and medium spatial frequencies. There is no decline in sensitivity at low temporal frequencies. These channels respond equally well to alternating and on/off gratings up to about 8 Hz.5. The temporal properties as revealed with sinusoidal modulation, suggest that the flicker-detecting channels would give transient responses to prolonged presentation of stimuli: the channels responsible for analysing the spatial structure would give sustained responses. The responses of the two types of channel to alternating and on/off gratings confirm this suggestion.

https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/jphysiol.1973.sp010261

Psychophysical evidence for sustained and transient detectors in human vision

1. We have examined the spatial and temporal tuning properties of 238 cortical neurones, recorded using conventional techniques from acutely prepared anaesthetized cats. We determined spatial and temporal frequency tuning curves using sinusoidal grating stimuli presented to each neurone's receptive field by a digital computer on a cathode ray tube. 2. We measured tuning curves either by determining response amplitude as a function of spatial or temporal frequency, or by measuring contrast sensitivity (the inverse of the contrast of the grating that just elicited a detectable response). The two measures give very similar tuning curves in all cases. 3. We recorded from 184 neurones in area 17; of these 156 had receptive fields within 5 degrees of the area centralis. The range of preferred spatial frequency for these neurones was 0.3--3 c/deg, and their spatial frequency tuning band widths varied from 0.7 to 3.2 octaves at half-amplitude. The most common band width was roughly 1.3 octaves. Simple and complex cells in area 17 did not differ in their distributions of preferred spatial frequency, although complex cells were, on average, slightly less selective for spatial frequency than simple cells. 4. We recorded from fifty-four neurones from area 18, and performed several experiments in which we recorded from corresponding portions of both area 17 and area 18 in the same electrode penetration. Neurones in area 18 preferred spatial frequencies that were, on average, one third as high as those preferred by area 17 neurones at the same retinal eccentricity. Thus the range of preferred spatial frequency in area eighteen cells having receptive fields within 5 deg of the area centralis was between less than 0.1 and 0.5 c/deg. The distributions of optimum spatial frequency in the two areas were practically non-overlapping at eccentricities as high as 15 deg, the greatest eccentricity we examined. Neurones in area 18 were about as selective for spatial frequency as were neurones in area 17. 5. We determined temporal frequency tuning characteristics for some neurones from each area, using gratings that moved steadily across the screen. Neurones from area 17 all responded well to low temporal frequencies, and less well to higher frequencies (in excess of, usually, 2 or 4 Hz). In contrast, neurones recorded from area 18 sometimes had similar tuning properties, but more commonly showed a pronounced reduction in response as the temporal frequency was moved either above or below some optimum value (usually 2--8 Hz). 6. We conclude from these results that areas 17 and 18 act in parallel to process different aspects of the visual information relayed from the retina via the lateral geniculate complex. Some or all of the differences between the areas may be attributable to the predominance of Y cell input to area 18 and the predominance of X cell input to area 17...

Spatial and temporal contrast sensitivity of neurones in areas 17 and 18 of the cat's visual cortex.

We can claim that we know what the visual system does once we can predict neural responses to arbitrary stimuli, including those seen in nature. In the early visual system, models based on one or more linear receptive fields hold promise to achieve this goal as long as the models include nonlinear mechanisms that control responsiveness, based on stimulus context and history, and take into account the nonlinearity of spike generation. These linear and nonlinear mechanisms might be the only essential determinants of the response, or alternatively, there may be additional fundamental determinants yet to be identified. Research is progressing with the goals of defining a single "standard model" for each stage of the visual pathway and testing the predictive power of these models on the responses to movies of natural scenes. These predictive models represent, at a given stage of the visual pathway, a compact description of visual computation. They would be an invaluable guide for understanding the underlying biophysical and anatomical mechanisms and relating neural responses to visual perception.

https://www.jneurosci.org/content/jneuro/25/46/10577.full.pdf

David J. Tolhurst

Papers

The statistical reliability of signals in single neurons in cat and monkey visual cortex.

Spatial summation in the receptive fields of simple cells in the cat's striate cortex.

Psychophysical evidence for sustained and transient detectors in human vision

Spatial and temporal contrast sensitivity of neurones in areas 17 and 18 of the cat's visual cortex.

Do we know what the early visual system does