The two basic phenomena that define the problem of visual attention can be illustrated in a simple example. Consider the arrays shown in each panel of Figure 1. In a typical experiment, before the arrays were presented, subjects would be asked to report letters appearing in one color (targets, here black letters), and to disregard letters in the other color (nontargets, here white letters). The array would then be briefly flashed, and the subjects, without any opportunity for eye movements, would give their report. The display mimics our. usual cluttered visual environment: It contains one or more objects that are relevant to current behavior, along with others that are irrelevant. The first basic phenomenon is limited capacity for processing information. At any given time, only a small amount of the information available on the retina can be processed and used in the control of behavior. Subjectively, giving attention to any one target leaves less available for others. In Figure 1, the probability of reporting the target letter N is much lower with two accompa­ nying targets (Figure la) than with none (Figure Ib). The second basic phenomenon is selectivity-the ability to filter out un­ wanted information. Subjectively, one is aware of attended stimuli and largely unaware of unattended ones. Correspondingly, accuracy in identifying an attended stimulus may be independent of the number of nontargets in a display (Figure la vs Ie) (see Bundesen 1990, Duncan 1980).

Neural Mechanisms of Selective Visual Attention

Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

/pdf/computational-modelling-of-visual-attention-3wtbok1ou7.pdf

Computational modelling of visual attention.

We describe a model of visual processing in which feedback connections from a higher- to a lower- order visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.

Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.

Neuronal activity causes local changes in cerebral blood flow, blood volume, and blood oxygenation. Magnetic resonance imaging (MRI) techniques sensitive to changes in cerebral blood flow and blood oxygenation were developed by high-speed echo planar imaging. These techniques were used to obtain completely noninvasive tomographic maps of human brain activity, by using visual and motor stimulus paradigms. Changes in blood oxygenation were detected by using a gradient echo (GE) imaging sequence sensitive to the paramagnetic state of deoxygenated hemoglobin. Blood flow changes were evaluated by a spin-echo inversion recovery (IR), tissue relaxation parameter T1-sensitive pulse sequence. A series of images were acquired continuously with the same imaging pulse sequence (either GE or IR) during task activation. Cine display of subtraction images (activated minus baseline) directly demonstrates activity-induced changes in brain MR signal observed at a temporal resolution of seconds. During 8-Hz patterned-flash photic stimulation, a significant increase in signal intensity (paired t test; P less than 0.001) of 1.8% +/- 0.8% (GE) and 1.8% +/- 0.9% (IR) was observed in the primary visual cortex (V1) of seven normal volunteers. The mean rise-time constant of the signal change was 4.4 +/- 2.2 s for the GE images and 8.9 +/- 2.8 s for the IR images. The stimulation frequency dependence of visual activation agrees with previous positron emission tomography observations, with the largest MR signal response occurring at 8 Hz. Similar signal changes were observed within the human primary motor cortex (M1) during a hand squeezing task and in animal models of increased blood flow by hypercapnia. By using intrinsic blood-tissue contrast, functional MRI opens a spatial-temporal window onto individual brain physiology.

https://www.pnas.org/content/pnas/89/12/5675.full.pdf

Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation.

Beginning with Emlen (1966) and MacArthur and Pianka (1966) and extending through the last ten years, several authors have sought to predict the foraging behavior of animals by means of mathematical models. These models are very similar,in that they all assume that the fitness of a foraging animal is a function of the efficiency of foraging measured in terms of some "currency" (Schoener, 1971) -usually energy- and that natural selection has resulted in animals that forage so as to maximize this fitness. As a result of these similarities, the models have become known as "optimal foraging models"; and the theory that embodies them, "optimal foraging theory." The situations to which optimal foraging theory has been applied, with the exception of a few recent studies, can be divided into the following four categories: (1) choice by an animal of which food types to eat (i.e., optimal diet); (2) choice of which patch type to feed in (i.e., optimal patch choice); (3) optimal allocation of time to different patches; and (4) optimal patterns and speed of movements. In this review we discuss each of these categories separately, dealing with both the theoretical developments and the data that permit tests of the predictions. The review is selective in the sense that we emphasize studies that either develop testable predictions or that attempt to test predictions in a precise quantitative manner. We also discuss what we see to be some of the future developments in the area of optimal foraging theory and how this theory can be related to other areas of biology. Our general conclusion is that the simple models so far formulated are supported are supported reasonably well by available data and that we are optimistic about the value both now and in the future of optimal foraging theory. We argue, however, that these simple models will requre much modification, espicially to deal with situations that either cannot easily be put into one or another of the above four categories or entail currencies more complicated that just energy.

Citation classic - optimal foraging - a selective review of theory and tests

We perceive the visual world as a unitary whole, yet one of the guiding principles of nearly a half century of neurophysiological research since the early recordings by Hartline (1938) has been that the visual system consists of neurons that are driven by stimulation within small discrete portions of the total visual field. These classical receptive fields (CRFs) have been mapped with the excitatory responses evoked by a flashed or moving stimulus, usually a spot or bar of light. Most of the visual neurons, in turn, are organized in a series of maps of the visual field, at least 10 of which exist in the visual cortex in primates as well as additional topographic representations in the lateral geniculate body, pulvinar and optic tectum (Allman 1977, Newsome & Allman 1980, Allman & Kaas 1984). It has been widely assumed that perceptual functions that require the integration of inputs over large portions of the visual field must be either collective properties of arrays of neurons representing the visual field, or features of those neurons at the highest processing levels in the visual system, such as the cells in inferotemporal or posterior parietal cortex that typically possess very large receptive fields and do not appear to be organized in visuotopic maps. These assumptions have been based on the results of the many studies in which receptive fields were mapped with conventional stimuli, presented one at a time, against a featureless background. However, unlike the neurophysiologist's tangent screen, the natural visual scene is rich in features, and there is a growing body of evidence that in many visual neurons stimuli presented outside the CRF strongly and selectively influence neural responses to stimuli presented within the CRF. These results suggest obvious mechanisms for local-global comparisons within visuotopically organized structures. Such broad and specific surround mechanisms could participate in many functions that require the integration of inputs over wide regions of the visual space such as the perceptual constancies, the segregation of figure from ground, and depth perception through motion parallax. In the first section of this paper, we trace the historical development of the evidence of response selectivity for visual stimuli presented beyond the CRF; in the second, examine the anatomical pathways that sub serve these far-reaching surround mechanisms; and in the third, explore the possible relationships between these mechanisms and perception.

Stimulus Specific Responses from Beyond the Classical Receptive Field: Neurophysiological Mechanisms for Local-Global Comparisons in Visual Neurons

The true receptive field of more than 90% of neurons in the middle temporal visual area (MT) extends well beyond the classical receptive field (crf), as mapped with conventional bar or spot stimuli, and includes a surrounding region that is 50 to 100 times the area of the crf. These extensive surrounds are demonstrated by simultaneously stimulating the crf and the surround with moving stimuli. The surrounds commonly have directional and velocity-selective influences that are antagonistic to the response from the crf. The crfs of MT neurons are organized in a topographic representation of the visual field. Thus MT neurons are embedded in an orderly visuotopic array, but are capable of integrating local stimulus conditions within a global context. The extensive surrounds of MT neurons may be involved in figure–ground discrimination, preattentive vision, perceptual constancies, and depth perception through motion cues.

Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).

Positron-emission tomography (PET) can localize functions of the human brain by imaging regional cerebral blood flow (CBF) during voluntary behaviour. Functional brain mapping with PET, however, has been hindered by PET's poor spatial resolution (typically greater than 1 cm). We have developed an image-analysis strategy that can map functional zones not resolved by conventional PET images. Brain areas selectively activated by a behavioural task can be isolated by subtracting a paired control-state image from the task-state image, thereby removing areas not recruited by the task. When imaged in isolation the centre of an activated area can be located very precisely. This allows subtle shifts in response locale due to changes in task to be detected readily despite poor spatial resolution. As an initial application of this strategy we mapped the retinal projection topography of human primary visual cortex. Functional zones separated by less than 3 mm (centre-to-centre) were differentiated using PET CBF images with a spatial resolution of 18 mm. This technique is not limited to a particular brain area or type of behaviour but does require that the increase in CBF produced by the task be both intense and focal.

Mapping human visual cortex with positron emission tomography.

The kinetics of choice: An operant systems analysis.

Magnification, the relative size of the neural representation of a portion of the visual field, decreases more rapidly with increasing visual field eccentricity in striate cortex than in the retinal ganglion cell layer of the owl monkey (Aotus trivirgatus); the proportion of the cells in striate cortex devoted to central vision is much larger than the comparable proportion of retinal ganglion cells. Magnification in striate cortex is a power function of magnification in the retinal ganglion cell layer. A formula for convergence (ganglion cells to cortical neurons) follows from this relationship.

Francis M. Miezin

Papers

Stimulus Specific Responses from Beyond the Classical Receptive Field: Neurophysiological Mechanisms for Local-Global Comparisons in Visual Neurons

Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).

Mapping human visual cortex with positron emission tomography.

The kinetics of choice: An operant systems analysis.

Magnification in striate cortex and retinal ganglion cell layer of owl monkey: a quantitative comparison.