A theory of edge detection is presented. The analysis proceeds in two parts. (1) Intensity changes, which occur in a natural image over a wide range of scales, are detected separately at different scales. An appropriate filter for this purpose at a given scale is found to be the second derivative of a Gaussian, and it is shown that, provided some simple conditions are satisfied, these primary filters need not be orientation-dependent. Thus, intensity changes at a given scale are best detected by finding the zero values of delta 2G(x,y)*I(x,y) for image I, where G(x,y) is a two-dimensional Gaussian distribution and delta 2 is the Laplacian. The intensity changes thus discovered in each of the channels are then represented by oriented primitives called zero-crossing segments, and evidence is given that this representation is complete. (2) Intensity changes in images arise from surface discontinuities or from reflectance or illumination boundaries, and these all have the property that they are spatially. Because of this, the zero-crossing segments from the different channels are not independent, and rules are deduced for combining them into a description of the image. This description is called the raw primal sketch. The theory explains several basic psychophysical findings, and the operation of forming oriented zero-crossing segments from the output of centre-surround delta 2G filters acting on the image forms the basis for a physiological model of simple cells (see Marr & Ullman 1979).

http://www.hms.harvard.edu/bss/neuro/bornlab/qmbc/beta/day4/marr-hildreth-edge-prsl1980.pdf

Theory of Edge Detection

Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1 ?

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

/pdf/spatiotemporal-energy-models-for-the-perception-of-motion-43mzze32mc.pdf

Spatiotemporal energy models for the perception of motion

Theoretical neuroscience provides a quantitative basis for describing what nervous systems do, determining how they function, and uncovering the general principles by which they operate This text introduces the basic mathematical and computational methods of theoretical neuroscience and presents applications in a variety of areas including vision, sensory-motor integration, development, learning, and memory The book is divided into three parts Part I discusses the relationship between sensory stimuli and neural responses, focusing on the representation of information by the spiking activity of neurons Part II discusses the modeling of neurons and neural circuits on the basis of cellular and synaptic biophysics Part III analyzes the role of plasticity in development and learning An appendix covers the mathematical methods used, and exercises are available on the book's Web site

Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems

Coding of the contrasts in natural images by visual cortex (VI) neurons: A Bayesian approach

Simple cells in area 17 of the anaesthetized, paralysed cat were stimulated with stationary sinusoidal gratings whose contrast was temporally modulated in different ways. The response to a temporal waveform which was the sum of two sinusoids (1.25 Hz and 7.75 Hz) was compared with the response elicited by each component when presented alone. The responses to the high temporal frequency in the compound stimulus were relatively enhanced by the addition of the low temporal frequency; those to the low frequency were relatively depressed.

Non-linear temporal summation by simple cells in cat striate cortex demonstrated by failure of superposition.

WHEN the colour of a visual stimulus is repetitively alternated between say red and green, the percept evoked depends upon the temporal frequency of repetition. At very low frequencies (for example, 1–2 Hz), the repetitive change in colour is readily apparent. But at frequencies above about 10 Hz, no changes in colour are perceived; the stimulus in this case would appear a uniform yellow but would still seem to be flickering in brightness. Observations of this kind suggest that the colour of a stimulus may be analysed by a set of channels different from that analysing the brightness or flicker. The flicker or brightness channels respond to higher temporal frequencies of flicker than do the colour channels. This idea was confirmed by King-Smith1 who has described how the sensitivity to coloured flashes of light changes with flash duration. He further described the subjective appearance of the flash, and concluded that a stimulus was not necessarily detected by the same channels as those that detected its colour. When the flash was brief, the stimuli usually looked achromatic, except when the wavelength was below about 490 nm. The intensity of the flash had to be raised above the detection threshold before the colour could be perceived. At longer flash durations, the colour was apparent as soon as the flash was detectable. The channels signalling the colour of a stimulus have a longer ‘integration time’ than those giving an achromatic percept. This kind of experiment examines differences in the high temporal frequency behaviour of the two sets of channels. It is of interest to examine the low temporal frequency behaviour as well because recent neuro-physiological investigations of the monkey's visual system make use of this behaviour in classifying neurones2–5. Some neurones give a sustained response to prolonged presentation of a stationary stimulus; others respond only transiently to the stimulus onset. I show here that human colour channels give sustained responses while the flicker or brightness channels give transient responses.

Colour-coding properties of sustained and transient channels in human vision

Shadows may be "discounted" in human visual perception because they do not provide stable, lighting-invariant, information about the properties of objects in the environment. Using visual search, R. A. Rensink and P. Cavanagh (2004) found that search for an upright discrepant shadow was less efficient than for an inverted one. Here we replicate and extend this work using photographs of real objects (pebbles) and their shadows. The orientation of the target shadows was varied between 30 and 180 degrees. Stimuli were presented upright (light from above, the usual situation in the world) or inverted (light from below, unnatural lighting). RTs for upright images were slower for shadows angled at 30 degrees, exactly as found by Rensink and Cavanagh. However, for all other shadow angles tested, the RTs were faster for upright images. This suggests, for small discrepancies in shadow orientation, a switch of processing from a relatively coarse-scaled shadow system to other general-purpose visual routines. Manipulations of the visual heterogeneity of the pebbles that cast the shadows differentially influenced performance. For inverted images, heterogeneity had the expected influence: reducing search efficiency and increasing overall search time. This effect was greatly reduced when images were presented upright, presumably when the distractors were processed as shadows. We suggest that shadows may be processed in a functionally separate, spatially coarse, mechanism. The pattern of results suggests that human vision does not use a shadow-suppressing system in search tasks.

David J. Tolhurst

Papers

Coding of the contrasts in natural images by visual cortex (VI) neurons: A Bayesian approach

Non-linear temporal summation by simple cells in cat striate cortex demonstrated by failure of superposition.

Colour-coding properties of sustained and transient channels in human vision

Search for gross illumination discrepancies in images of natural objects

Simple cells in the visual cortex of the cat can be narrowly tuned for spatial frequency.