A theory of edge detection is presented. The analysis proceeds in two parts. (1) Intensity changes, which occur in a natural image over a wide range of scales, are detected separately at different scales. An appropriate filter for this purpose at a given scale is found to be the second derivative of a Gaussian, and it is shown that, provided some simple conditions are satisfied, these primary filters need not be orientation-dependent. Thus, intensity changes at a given scale are best detected by finding the zero values of delta 2G(x,y)*I(x,y) for image I, where G(x,y) is a two-dimensional Gaussian distribution and delta 2 is the Laplacian. The intensity changes thus discovered in each of the channels are then represented by oriented primitives called zero-crossing segments, and evidence is given that this representation is complete. (2) Intensity changes in images arise from surface discontinuities or from reflectance or illumination boundaries, and these all have the property that they are spatially. Because of this, the zero-crossing segments from the different channels are not independent, and rules are deduced for combining them into a description of the image. This description is called the raw primal sketch. The theory explains several basic psychophysical findings, and the operation of forming oriented zero-crossing segments from the output of centre-surround delta 2G filters acting on the image forms the basis for a physiological model of simple cells (see Marr & Ullman 1979).

http://www.hms.harvard.edu/bss/neuro/bornlab/qmbc/beta/day4/marr-hildreth-edge-prsl1980.pdf

Theory of Edge Detection

A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

/pdf/spatiotemporal-energy-models-for-the-perception-of-motion-43mzze32mc.pdf

Spatiotemporal energy models for the perception of motion

1. The contrast thresholds of a variety of grating patterns have been measured over a wide range of spatial frequencies.2. Contrast thresholds for the detection of gratings whose luminance profiles are sine, square, rectangular or saw-tooth waves can be simply related using Fourier theory.3. Over a wide range of spatial frequencies the contrast threshold of a grating is determined only by the amplitude of the fundamental Fourier component of its wave form.4. Gratings of complex wave form cannot be distinguished from sine-wave gratings until their contrast has been raised to a level at which the higher harmonic components reach their independent threshold.5. These findings can be explained by the existence within the nervous system of linearly operating independent mechanisms selectively sensitive to limited ranges of spatial frequencies.

Application of fourier analysis to the visibility of gratings

1. It was found that an occipital evoked potential can be elicited in the human by moving a grating pattern without changing the mean light flux entering the eye. Prolonged viewing of a high contrast grating reduces the amplitude of the potential evoked by a low contrast grating.

2. This adaptation to a grating was studied psychophysically by determining the contrast threshold before and after adaptation. There is a temporary fivefold rise in contrast threshold after exposure to a high contrast grating of the same orientation and spatial frequency.

3. By determining the rise of threshold over a range of spatial frequency for a number of adapting frequencies it was found that the threshold elevation is limited to a spectrum of frequencies with a bandwidth of just over an octave at half amplitude, centred on the adapting frequency.

4. The amplitude of the effect and its bandwidth are very similar for adapting spatial frequencies between 3 c/deg. and 14 c/deg. At higher frequencies the bandwidth is slightly narrower. For lower adapting frequencies the peak of the effect stays at 3 c/deg.

5. These and other findings suggest that the human visual system may possess neurones selectively sensitive to spatial frequency and size. The orientational selectivity and the interocular transfer of the adaptation effect implicate the visual cortex as the site of these neurones.

6. This neural system may play an essential preliminary role in the recognition of complex images and generalization for magnification.

/pdf/on-the-existence-of-neurones-in-the-human-visual-system-3pxi9uujkp.pdf

On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images.

Simple cells in the striate cortex have been depicted as half-wave-rectified linear operators. Complex cells have been depicted as energy mechanisms, constructed from the squared sum of the outputs of quadrature pairs of linear operators. However, the linear/energy model falls short of a complete explanation of striate cell responses. In this paper, a modified version of the linear/energy model is presented in which striate cells mutually inhibit one another, effectively normalizing their responses with respect to stimulus contrast. This paper reviews experimental measurements of striate cell responses, and shows that the new model explains a significantly larger body of physiological data.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0952523800009640

Normalization of cell responses in cat striate cortex

1. Spatial summation within cat retinal receptive fields was studied by recording from optic-tract fibres the responses of ganglion cells to grating patterns whose luminance perpendicular to the bars varied sinusoidally about the mean level.

2. Summation over the receptive fields of some cells (X-cells) was found to be approximately linear, while for other cells (Y-cells) summation was very non-linear.

3. The mean discharge frequency of Y-cells (unlike that of X-cells) was greatly increased when grating patterns drifted across their receptive fields.

4. In twenty-one X-cells the relation between the contrast and spatial frequency of drifting sinusoidal gratings which evoked the same small response was measured. In every case it was found that the reciprocal of this relation, the contrast sensitivity function, could be satisfactorily described by the difference of two Gaussian functions.

5. This finding supports the hypothesis that the sensitivities of the antagonistic centre and surround summating regions of ganglion cell receptive fields fall off as Gaussian functions of the distance from the field centre.

6. The way in which the sensitivity of an X-cell for a contrast-edge pattern varied with the distance of the edge from the receptive field centre was determined and found to be consistent with the cell's measured contrast sensitivity function.

7. Reducing the retinal illumination produced changes in the contrast sensitivity function of an X-cell which suggested that the diameters of the summating regions of the receptive field increased while the surround region became relatively ineffective.

https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/jphysiol.1966.sp008107

The contrast sensitivity of retinal ganglion cells of the cat.

Chapter 9 Visual adaptation and retinal gain controls

1. Micro-electrode recordings have been made from single units in the visual cortex of the cat, during stimulation by moving grating patterns generated on a cathode ray tube.

2. The responses of the units have been measured in terms of either the frequency of impulses, or the contrast sensitivity, and expressed as a function of the spatial frequency of the grating pattern.

3. The amplitude of the responses recorded from cortical cell bodies was dependent upon the orientation of the moving grating, and for these units the stimulus was always presented at the preferred orientation. The response amplitude of other units did not depend upon the orientation of the grating stimulus, and these are believed to be the terminations of geniculate fibres.

4. The high spatial frequency end of the response function measured in terms of the contrast sensitivity could be well fitted by an exponential function. Subtraction of the blank count from the impulse frequency data gave a curve fitted by the same exponential function. The low frequency end was less consistent.

5. The spatial frequency at which this exponential function had fallen by one log. unit was taken to specify the position of the unit's response in the spatial frequency spectrum. For all units these values cover a range of four octaves of spatial frequency (from 0·18 to 3·8 cycles per degree).

6. It is suggested that these neurophysiological results support psychophysical evidence for the existence in the visual system of channels, each selectively sensitive to a narrow band of spatial frequencies.

https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/jphysiol.1969.sp008861

The spatial selectivity of the visual cells of the cat.

1. Grating patterns were used to obtain a quantitative description of cells in the visual cortex of the cat whose response amplitude depended critically upon the orientation of the moving grating.

2. In all such cells the impulse frequency was found to decrease linearly with angle on either side of an optimum angle (the preferred angle) until the response fell to zero or to a base frequency. The angular rate of change of response varied between cells and was expressed as the half-width at half amplitude (the angular selectivity).

3. The angular selectivity of thirty-five cells was determined and more than half (nineteen) of these fell within the range 14-26°.

4. Fourteen cells responded optimally only when the grating was moved in one direction. Twenty-one cells responded optimally to two directions of movement 180° apart, but the response in the two directions was not always equal.

5. No significant correlation was found between the response amplitude at the optimum angle and the angular selectivity.

6. The distribution of preferred angles did not show any difference between the oblique orientations and the vertical and horizontal orientations.

7. These results are compared with a previous psychophysical estimate of angular selectivity.

https://physoc.onlinelibrary.wiley.com/doi/epdf/10.1113/jphysiol.1968.sp008604

Christina Enroth-Cugell

Papers

The contrast sensitivity of retinal ganglion cells of the cat.

Chapter 9 Visual adaptation and retinal gain controls

The spatial selectivity of the visual cells of the cat.

The angular selectivity of visual cortical cells to moving gratings.

Receptive field properties of X and Y cells in the cat retina derived from contrast sensitivity measurements