A motion sequence may be represented as a single pattern in x–y–t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

/pdf/spatiotemporal-energy-models-for-the-perception-of-motion-43mzze32mc.pdf

Spatiotemporal energy models for the perception of motion

We propose a model of how humans sense the velocity of moving images. The model exploits constraints provided by human psychophysics, notably that motion-sensing elements appear tuned for two-dimensional spatial frequency, and by the frequency spectrum of a moving image, namely, that its support lies in the plane in which the temporal frequency equals the dot product of the spatial frequency and the image velocity. The first stage of the model is a set of spatial-frequency-tuned, direction-selective linear sensors. The temporal frequency of the response of each sensor is shown to encode the component of the image velocity in the sensor direction. At the second stage, these components are resolved in order to measure the velocity of image motion at each of a number of spatial locations and spatial frequencies. The model has been applied to several illustrative examples, including apparent motion, coherent gratings, and natural image sequences. The model agrees qualitatively with human perception.

Model of human visual-motion sensing

Shannon's rate-distortion function provides a potentially useful lower bound against which to compare the rate-versus-distortion performance of practical encoding-transmission systems. However, this bound is not applicable unless one can arrive at a numerically-valued measure of distortion which is in reasonable correspondence with the subjective evaluation of the observer or interpreter. We have attempted to investigate this choice of distortion measure for monochrome still images. This investigation has considered a class of distortion measures for which it is possible to simulate the optimum (in a rate-distortion sense) encoding. Such simulation was performed at a fixed rate for various measures in the class and the results compared subjectively by observers. For several choices of transmission rate and original images, one distortion measure was fairly consistently rated as yielding the most satisfactory appearing encoded images.

The effects of a visual fidelity criterion of the encoding of images

Our eyes continually move even while we fix our gaze on an object. Although these fixational eye movements have a magnitude that should make them visible to us, we are unaware of them. If fixational eye movements are counteracted, our visual perception fades completely as a result of neural adaptation. So, our visual system has a built-in paradox — we must fix our gaze to inspect the minute details of our world, but if we were to fixate perfectly, the entire world would fade from view. Owing to their role in counteracting adaptation, fixational eye movements have been studied to elucidate how the brain makes our environment visible. Moreover, because we are not aware of these eye movements, they have been studied to understand the underpinnings of visual awareness. Recent studies of fixational eye movements have focused on determining how visible perception is encoded by neurons in various visual areas of the brain.

http://www.neuralcorrelate.com/smc/files/publications/martinez-conde_et_al_nrn04.pdf

The role of fixational eye movements in visual perception

A method of producing red-green and blue-yellow sinusoidal chromatic gratings is used which permits the correction of all chromatic aberrations. A quantitative criterion is adopted to choose the intensity match of the two colours in the stimulus: this is the intensity ratio at which contrast sensitivity for the chromatic grating differs most from the contrast sensitivity for a monochromatic luminance grating. Results show that this intensity match varies with spatial frequency and does not necessarily correspond to a luminance match between the colours. Contrast sensitivities to the chromatic gratings at the criterion intensity match are measured as a function of spatial frequency, using field sizes ranging from 2 to 23 deg. Both blue-yellow and red-green contrast sensitivity functions have similar low-pass characteristics, with no low-frequency attenuation even at low frequencies below 0.1 cycles/deg. These functions indicate that the limiting acuities based on red-green and blue-yellow colour discriminations are similar at 11 or 12 cycles/deg. Comparisons between contrast sensitivity functions for the chromatic and monochromatic gratings are made at the same mean luminances. Results show that, at low spatial frequencies below 0.5 cycles/deg, contrast sensitivity is greater to the chromatic gratings, consisting of two monochromatic gratings added in antiphase, than to either monochromatic grating alone. Above 0.5 cycles/deg, contrast sensitivity is greater to monochromatic than to chromatic gratings.

The contrast sensitivity of human colour vision to red‐green and blue‐yellow chromatic gratings.

The stabilized contrast-sensitivity function measured at a constant retinal velocity is tuned to a particular spatial frequency, which is inversely related to the velocity chosen The Fourier transforms of these constant-velocity passbands have the same form as retinal receptive fields of various sizes At low velocities, in the range of the natural drift motions of the eye, the stabilized contrast-sensitivity function matches the normal, unstablized result At higher velocities (corresponding to motions of objects in the environment), this curve maintains the same shape but shifts toward lower spatial frequencies The constant-velocity passband is displaced across the spatio-temporal frequency domain in a manner that is almost symmetric about the constant-velocity plane at v = 2 deg/s Interpolating these diagonal profiles by a suitable analytic expression, we construct the spatio-temporal threshold surface for stabilized vision, and display its properties in terms of the usual frequency parameters; eg, at low spatial frequencies, the temporal response becomes nearly independent of spatial frequency, while at low temporal frequencies, the spatial response becomes independent of temporal frequency

Motion and vision. II. Stabilized spatio-temporal threshold surface.

Moving the retinal image of a sinusoidal grating at a constant velocity (compensated for eye movements) provides controlled spatial and temporal frequencies at every point in the stimulus field. Using this controlled-velocity technique, we have measured the detection threshold for isoluminance, red/green gratings as a function of their spatial and temporal frequencies. The chromatic contrast-threshold surface obtained in this way is analogous to the achromatic contrast-threshold surface measured previously, but the results are quite different. For very low temporal frequencies (below 0.2 Hz), the chromatic sensitivity decreases steadily with decreasing temporal frequency. Below 0.01 Hz, chromatic patterns disappear completely even at maximum contrast (although achromatic or homochromatic patterns do not). In the region above 0.2 Hz, both achromatic and chromatic thesholds can be explained by the same receptive-field-like model. When the center and the surround components of this model are additively combined, they form the chromatic threshold surface; when the sign of either component is reversed, they form the achromatic one.

Spatiotemporal variation of chromatic and achromatic contrast thresholds

Photopic flicker data are explained in terms of a theoretical model of two retinal processes. The first is a linear diffusion process (presumably in the receptors), with a large dynamic range (∼105). The second is a nonlinear inhibiting network (neural feedback at the synapses of the plexiform layers) that adaptively controls the sensitivity and time constants of the model. The magnitude of its transfer function fits the flicker data quantitatively at all frequencies, over a wide range of adaptation levels. The corresponding small-signal impulse responses are also calculated: their latencies and leading edges (associated with receptor activity) are invariant with adaptation level; the remaining phases of these transient waveforms (associated with the graded potentials of secondary neurons) adapt strongly, in accord with current histology and micro-electrode findings.

Theory of Flicker and Transient Responses, I. Uniform Fields*

Spatio-temporal sine-wave response functions are inferred for the red-, green-, and blue-sensitive receptor mechanisms of the visual process, by selective chromatic-adaptation experiments and Weber-law calculations. The complete spatio-temporal threshold surface for each cone mechanisms is abbreviated to four critical profiles: two contrast-sensitivity curves measured at the temporal frequencies of minimum and maximum sensitivity, and two flicker-sensitivity curves measured at the spatial frequencies of minimum and maximum sensitivity. Throughout most of the spatio-temporal frequency domain, the green sensitivity is greatest, the red is less, and the blue is least of all. The curve shapes can be qualitatively explained in terms of antagonistic interactions in the early visual pathways.

Spatio-temporal frequency characteristics of color-vision mechanisms*

The model derived in the preceding paper is applied to new sine-wave flicker data, obtained with 7° circular, uniform-field and counterphase-grating targets, at four adaptation levels ranging from 1.67 to 1670 td. The data from all eight conditions are well fitted by varying two parameters associated with neural-inhibition processes; the major effect of changing the spatial pattern is on the number of neural units involved. But even when lateral inhibition is minimized, low-frequency feedback still dominates the transient responses of the model. These results can be interpreted in terms of spatial and temporal filtering in the outer and inner plexiform layers of the retina.

D. H. Kelly

Papers

Motion and vision. II. Stabilized spatio-temporal threshold surface.

Spatiotemporal variation of chromatic and achromatic contrast thresholds

Theory of Flicker and Transient Responses, I. Uniform Fields*

Spatio-temporal frequency characteristics of color-vision mechanisms*

Theory of Flicker and Transient Responses,* II. Counterphase Gratings