Two-dimensional spatial linear filters are constrained by general uncertainty relations that limit their attainable information resolution for orientation, spatial frequency, and two-dimensional (2D) spatial position. The theoretical lower limit for the joint entropy, or uncertainty, of these variables is achieved by an optimal 2D filter family whose spatial weighting functions are generated by exponentiated bivariate second-order polynomials with complex coefficients, the elliptic generalization of the one-dimensional elementary functions proposed in Gabor’s famous theory of communication [ J. Inst. Electr. Eng.93, 429 ( 1946)]. The set includes filters with various orientation bandwidths, spatial-frequency bandwidths, and spatial dimensions, favoring the extraction of various kinds of information from an image. Each such filter occupies an irreducible quantal volume (corresponding to an independent datum) in a four-dimensional information hyperspace whose axes are interpretable as 2D visual space, orientation, and spatial frequency, and thus such a filter set could subserve an optimally efficient sampling of these variables. Evidence is presented that the 2D receptive-field profiles of simple cells in mammalian visual cortex are well described by members of this optimal 2D filter family, and thus such visual neurons could be said to optimize the general uncertainty relations for joint 2D-spatial–2D-spectral information resolution. The variety of their receptive-field dimensions and orientation and spatial-frequency bandwidths, and the correlations among these, reveal several underlying constraints, particularly in width/length aspect ratio and principal axis organization, suggesting a polar division of labor in occupying the quantal volumes of information hyperspace. Such an ensemble of 2D neural receptive fields in visual cortex could locally embed coarse polar mappings of the orientation–frequency plane piecewise within the global retinotopic mapping of visual space, thus efficiently representing 2D spatial visual information by localized 2D spectral signatures.

Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters.

Anatomical and physiological observations in monkeys indicate that the primate visual system consists of several separate and independent subdivisions that analyze different aspects of the same retinal image: cells in cortical visual areas 1 and 2 and higher visual areas are segregated into three interdigitating subdivisions that differ in their selectivity for color, stereopsis, movement, and orientation. The pathways selective for form and color seem to be derived mainly from the parvocellular geniculate subdivisions, the depth- and movement-selective components from the magnocellular. At lower levels, in the retina and in the geniculate, cells in these two subdivisions differ in their color selectivity, contrast sensitivity, temporal properties, and spatial resolution. These major differences in the properties of cells at lower levels in each of the subdivisions led to the prediction that different visual functions, such as color, depth, movement, and form perception, should exhibit corresponding differences. Human perceptual experiments are remarkably consistent with these predictions. Moreover, perceptual experiments can be designed to ask which subdivisions of the system are responsible for particular visual abilities, such as figure/ground discrimination or perception of depth from perspective or relative movement--functions that might be difficult to deduce from single-cell response properties.

https://science.sciencemag.org/content/sci/240/4853/740.full.pdf

Segregation of form, color, movement, and depth: anatomy, physiology, and perception

The relative efficiency of any particular image-coding scheme should be defined only in relation to the class of images that the code is likely to encounter. To understand the representation of images by the mammalian visual system, it might therefore be useful to consider the statistics of images from the natural environment (i.e., images with trees, rocks, bushes, etc). In this study, various coding schemes are compared in relation to how they represent the information in such natural images. The coefficients of such codes are represented by arrays of mechanisms that respond to local regions of space, spatial frequency, and orientation (Gabor-like transforms). For many classes of image, such codes will not be an efficient means of representing information. However, the results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy (e.g., correlation between the intensities of neighboring pixels) into first-order redundancy (i.e., the response distribution of the coefficients). Such coding produces a relatively high signal-to-noise ratio and permits information to be transmitted with only a subset of the total number of cells. These results support Barlow's theory that the goal of natural vision is to represent the information in the natural environment with minimal redundancy.

/pdf/relations-between-the-statistics-of-natural-images-and-the-12zwc2ox1f.pdf

Relations between the statistics of natural images and the response properties of cortical cells.

The author reviews recent multichannel models developed in psychophysiology, computer vision, and image processing. In psychophysiology, multichannel models have been particularly successful in explaining some low-level processing in the visual cortex. The expansion of a function into several frequency channels provides a representation which is intermediate between a spatial and a Fourier representation. The author describes the mathematical properties of such decompositions and introduces the wavelet transform. He reviews the classical multiresolution pyramidal transforms developed in computer vision and shows how they relate to the decomposition of an image into a wavelet orthonormal basis. He discusses the properties of the zero crossings of multifrequency channels. Zero-crossing representations are particularly well adapted for pattern recognition in computer vision. >

Multifrequency channel decompositions of images and wavelet models

1. Using the two-dimensional (2D) spatial and spectral response profiles described in the previous two reports, we test Daugman's generalization of Marcelja's hypothesis that simple receptive fields belong to a class of linear spatial filters analogous to those described by Gabor and referred to here as 2D Gabor filters. 2. In the space domain, we found 2D Gabor filters that fit the 2D spatial response profile of each simple cell in the least-squared error sense (with a simplex algorithm), and we show that the residual error is devoid of spatial structure and statistically indistinguishable from random error. 3. Although a rigorous statistical approach was not possible with our spectral data, we also found a Gabor function that fit the 2D spectral response profile of each simple cell and observed that the residual errors are everywhere small and unstructured. 4. As an assay of spatial linearity in two dimensions, on which the applicability of Gabor theory is dependent, we compare the filter parameters estimated from the independent 2D spatial and spectral measurements described above. Estimates of most parameters from the two domains are highly correlated, indicating that assumptions about spatial linearity are valid. 5. Finally, we show that the functional form of the 2D Gabor filter provides a concise mathematical expression, which incorporates the important spatial characteristics of simple receptive fields demonstrated in the previous two reports. Prominent here are 1) Cartesian separable spatial response profiles, 2) spatial receptive fields with staggered subregion placement, 3) Cartesian separable spectral response profiles, 4) spectral response profiles with axes of symmetry not including the origin, and 5) the uniform distribution of spatial phase angles. 6. We conclude that the Gabor function provides a useful and reasonably accurate description of most spatial aspects of simple receptive fields. Thus it seems that an optimal strategy has evolved for sampling images simultaneously in the 2D spatial and spatial frequency domains.

An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex

1. Cells in visual cortex have been alternately considered as bar and edge detectors, or as spatial-frequency filters responding to the two-dimensional Fourier component of patterns. 2. The responses to gratings and to checkerboards allow one to test these alternate models: the Fourier components of a checkerboard pattern do not occur at the same orientation as the edges, nor do the checkerboard spatial frequencies correspond to the check widths. 3. Knowing the orientation tuning of a cell for gratings, one can precisely predict its orientation tuning to checkerboards from the orientation of the fundamental Fourier components of the patterns, not from the orientation of their edges. This was found for both square and rectangular checkerboards, and held for both simple and complex cortical cells. 4. Knowing the spatial tuning of a cell for sine-wave gratings, one can precisely predict its spatial tuning to square and rectangular checkerboards from the spatial frequencies of the fundamental Fourier components of the patterns, not from the widths of their checks. 5. When presented with checkerboards in which not the fundamental but the upper harmonics were within its spatial bandpass, a cell's orientation tuning was found to be predictable from the (quite different) orientation of the higher Fourier harmonic components, but not from the orientation of the edges. 6. Knowing a cell's contrast sensitivity for gratings, one can predict the cell's contrast sensitivity for checkerboards much more accurately from the amplitudes of the two-dimensional Fourier components of the patterns than from the contrasts of the patterns. 7. The orientation tuning, spatial-frequency tuning and responsiveness of cells to a plaid pattern were also found to be predictable from the pattern's two-dimensional Fourier spectrum. 8. Both simple and complex striate cortex cells can thus be characterized as two-dimensional spatial-frequency filters. Since different cells responsive to the same region in the visual field are tuned to different spatial frequencies and orientations, the ensemble of such cells would fairly precisely encode the two-dimensional Fourier spectrum of a patch of visual space.

https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/jphysiol.1979.sp012827

Responses of striate cortex cells to grating and checkerboard patterns.

The spatial tuning of macaque lateral geniculate neurones was compared for luminance-based and color-based lines. Lines of various widths were flashed on and centered on the cell's receptive field, and the size of the increase or decrease in firing was noted. Luminance-based lines consisted of 0.7 log unit increments or decrements. Color-based lines consisted of shifts in wavelength with no change in luminance, e.g., from a red field to a green line on a red field. The cell fired most to intermediate widths of luminance-based lines, but to the widest pure-color lines.

Responses of macaque lateral geniculate cells to luminance and color figures.

Brightness contrast effects shown by single cells in the macaque's lateral geniculate nucleus were studied with black and white lines of various widths, consisting of either: (1) "simultaneous contrast" stimuli in which the line was produced by luminance changes in the flanking areas or (2) "successive contrast" stimuli in which the line itself changed in luminance. Line widths that gave optimal responses and response magnitudes themselves were similar for the two types of stimulus, except for the widest lines used (2 degrees). Thus, simultaneous brightness contrast is a primary determinant of the response of primate LGN cells but only within 2 degrees of the center of the receptive field. Neural processing up to this level cannot therefore explain the long distance effects of simultaneous brightness contrast in human perception.

E. W. Yund

Papers

Responses of striate cortex cells to grating and checkerboard patterns.

Responses of macaque lateral geniculate cells to luminance and color figures.

Brightness contrast effects in monkey lateral geniculate nucleus.