A theory of edge detection is presented. The analysis proceeds in two parts. (1) Intensity changes, which occur in a natural image over a wide range of scales, are detected separately at different scales. An appropriate filter for this purpose at a given scale is found to be the second derivative of a Gaussian, and it is shown that, provided some simple conditions are satisfied, these primary filters need not be orientation-dependent. Thus, intensity changes at a given scale are best detected by finding the zero values of delta 2G(x,y)*I(x,y) for image I, where G(x,y) is a two-dimensional Gaussian distribution and delta 2 is the Laplacian. The intensity changes thus discovered in each of the channels are then represented by oriented primitives called zero-crossing segments, and evidence is given that this representation is complete. (2) Intensity changes in images arise from surface discontinuities or from reflectance or illumination boundaries, and these all have the property that they are spatially. Because of this, the zero-crossing segments from the different channels are not independent, and rules are deduced for combining them into a description of the image. This description is called the raw primal sketch. The theory explains several basic psychophysical findings, and the operation of forming oriented zero-crossing segments from the output of centre-surround delta 2G filters acting on the image forms the basis for a physiological model of simple cells (see Marr & Ullman 1979).

http://www.hms.harvard.edu/bss/neuro/bornlab/qmbc/beta/day4/marr-hildreth-edge-prsl1980.pdf

Theory of Edge Detection

Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1 ?

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

/pdf/spatiotemporal-energy-models-for-the-perception-of-motion-43mzze32mc.pdf

Spatiotemporal energy models for the perception of motion

Theoretical neuroscience provides a quantitative basis for describing what nervous systems do, determining how they function, and uncovering the general principles by which they operate This text introduces the basic mathematical and computational methods of theoretical neuroscience and presents applications in a variety of areas including vision, sensory-motor integration, development, learning, and memory The book is divided into three parts Part I discusses the relationship between sensory stimuli and neural responses, focusing on the representation of information by the spiking activity of neurons Part II discusses the modeling of neurons and neural circuits on the basis of cellular and synaptic biophysics Part III analyzes the role of plasticity in development and learning An appendix covers the mathematical methods used, and exercises are available on the book's Web site

Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems

Background: Homonymous visual field defects (VFD) are common following stroke, and often recover, partially or fully, by unknown mechanisms. In clinical practice,

Visual Contrast Sensitivity Deficits in 'Normal' Visual Field of Patients with Homonymous Visual Field Defects due to Stroke: A Pilot Study

Accuracy of identification of grating contrast by human observers: Bayesian models of V1 contrast processing show correspondence between discrimination and identification performance

Contrast normalization contributes to a biologically-plausible model of receptive-field development in primary visual cortex (V1)

How different are two images when viewed by a human observer? Such knowledge is needed in many situations including when one has to judge the degree to which a graphics representation may be similar to a high-quality photograph of the original scene. There is a class of computational models which attempt to predict such perceived differences. These are derived from theoretical considerations of human vision and are mostly validated from experiments on stimuli such as sinusoidal gratings. We are developing a model of visual difference prediction based on multi-scale analysis of local contrast, to be tested with psychophysical discrimination experiments on natural-scene stimuli. Here, we extend our model to account for differences in the chromatic domain. We describe the model, how it has been derived and how we attempt to validate it psychophysically for monochrome and chromatic images.

/pdf/a-multiresolution-color-model-for-visual-difference-36uyo2gebb.pdf

A multiresolution color model for visual difference prediction

Visual Difference Predictor (VDP) models have played a key role in digital image applications such as the development of image quality metrics. However, little attention has been paid to their applicability to peripheral vision. Central (i.e., foveal) vision is extremely sensitive for the contrast detection of simple stimuli such as sinusoidal gratings, but peripheral vision is less sensitive. Furthermore, crowding is a well-documented phenomenon whereby differences in suprathreshold peripherally viewed target objects (such as individual letters or patches of sinusoidal grating) become more difficult to discriminate when surrounded by other objects (flankers). We examine three factors that might influence the degree of crowding with natural-scene stimuli (cropped from photographs of natural scenes): (1) location in the visual field, (2) distance between target and flankers, and (3) flanker-target similarity. We ask how these factors affect crowding in a suprathreshold discrimination experiment where observers rate the perceived differences between two sequentially presented target patches of natural images. The targets might differ in the shape, size, arrangement, or color of items in the scenes. Changes in uncrowded peripheral targets are perceived to be less than for the same changes viewed foveally. Consistent with previous research on simple stimuli, we find that crowding in the periphery (but not in the fovea) reduces the magnitudes of perceived changes even further, especially when the flankers are closer and more similar to the target. We have tested VDP models based on the response behavior of neurons in visual cortex and the inhibitory interactions between them. The models do not explain the lower ratings for peripherally viewed changes even when the lower peripheral contrast sensitivity was accounted for; nor could they explain the effects of crowding, which others have suggested might arise from errors in the spatial localization of features in the peripheral image. This suggests that conventional VDP models do not port well to peripheral vision.

David J. Tolhurst

Papers

Visual Contrast Sensitivity Deficits in 'Normal' Visual Field of Patients with Homonymous Visual Field Defects due to Stroke: A Pilot Study

Accuracy of identification of grating contrast by human observers: Bayesian models of V1 contrast processing show correspondence between discrimination and identification performance

Contrast normalization contributes to a biologically-plausible model of receptive-field development in primary visual cortex (V1)

A multiresolution color model for visual difference prediction

Perception of differences in natural-image stimuli: Why is peripheral viewing poorer than fovealq