Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

/pdf/computational-modelling-of-visual-attention-3wtbok1ou7.pdf

Computational modelling of visual attention.

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

This article concerns the nature of evoked brain responses and the principles underlying their generation. We start with the premise that the sensory brain has evolved to represent or infer the causes of changes in its sensory inputs. The problem of inference is well formulated in statistical terms. The statistical fundaments of inference may therefore afford important constraints on neuronal implementation. By formulating the original ideas of Helmholtz on perception, in terms of modern-day statistical theories, one arrives at a model of perceptual inference and learning that can explain a remarkable range of neurobiological facts. It turns out that the problems of inferring the causes of sensory input (perceptual inference) and learning the relationship between input and cause (perceptual learning) can be resolved using exactly the same principle. Specifically, both inference and learning rest on minimizing the brain’s free energy, as defined in statistical physics. Furthermore, inference and learning can proceed in a biologically plausible fashion. Cortical responses can be seen as the brain’s attempt to minimize the free energy induced by a stimulus and thereby encode the most likely cause of that stimulus. Similarly, learning emerges from changes in synaptic efficacy that minimize the free energy, averaged over all stimuli encountered. The underlying scheme rests on empirical Bayes and hierarchical models of how sensory input is caused. The use of hierarchical models enables the brain to construct prior expectations in a dynamic and context-sensitive fashion. This scheme provides a principled way to understand many aspects of cortical organization and responses. The aim of this article is to encompass many apparently unrelated anatomical, physiological and psychophysical attributes of the brain within a single theoretical perspective. In terms of cortical architectures, the theoretical treatment predicts that sensory cortex should be arranged hierarchically, that connections should be reciprocal and that forward and backward connections should show a functional asymmetry (forward connections are driving, whereas backward connections are both driving and modulatory). In terms of synaptic physiology, it predicts associative plasticity and, for dynamic models, spike-timing-dependent plasticity. In terms of electrophysiology, it accounts for classical and extra classical receptive field effects and long-latency or endogenous components of evoked cortical responses. It predicts the attenuation of responses encoding prediction error with perceptual learning and explains many phenomena such as repetition suppression, mismatch negativity (MMN) and the P300 in electroencephalography. In psychophysical terms, it accounts for the behavioural correlates of these physiological phenomena, for example, priming and global precedence. The final focus of this article is on perceptual learning as measured with the MMN and the implications for empirical studies of coupling among cortical areas using evoked sensory responses.

/pdf/a-theory-of-cortical-responses-2aiw9zqf1h.pdf

A theory of cortical responses

The mammalian visual system is endowed with a nearly infinite capacity for the recognition of patterns and objects. To have acquired this capability the visual system must have solved what is a fundamentally combinatorial prob­ lem. Any given image consists of a collection of features, consisting of local contrast borders of luminance and wavelength, distributed across the visual field. For one to detect and recognize an object within a scene, the features comprising the object must be identified and segregated from those comprising other objects. This problem is inherently difficult to solve because of the combinatorial nature of visual images. To appreciate this point, consider a simple local feature such as a small vertically oriented line segment placed within a fixed location of the visual field. When combined with other line segments, this feature can form a nearly infinite number of geometrical objects. Any one of these objects may coexist with an equally large number of other

Visual feature integration and the temporal correlation hypothesis

Simultaneous recordings were made from large ensembles of hippocampal "place cells" in three rats during spatial behavioral tasks and in slow-wave sleep preceding and following these behaviors. Cells that fired together when the animal occupied particular locations in the environment exhibited an increased tendency to fire together during subsequent sleep, in comparison to sleep episodes preceding the behavioral tasks. Cells that were inactive during behavior, or that were active but had non-overlapping spatial firing, did not show this increase. This effect, which declined gradually during each post-behavior sleep session, may result from synaptic modification during waking experience. Information acquired during active behavior is thus re-expressed in hippocampal circuits during sleep, as postulated by some theories of memory consolidation.

https://science.sciencemag.org/content/sci/265/5172/676.full.pdf

Reactivation of hippocampal ensemble memories during sleep.

View from the top: hierarchies and reverse hierarchies in the visual system.

https://www.cs.princeton.edu/courses/archive/spr08/cos598B/Readings/AhissarHochstein_TCS2004.pdf

The Reverse Hierarchy Theory of Visual Perceptual Learning

Practising simple visual tasks leads to a dramatic improvement in performing them. This learning is specific to the stimuli used for training. We show here that the degree of specificity depends on the difficulty of the training conditions. We find that the pattern of specificities maps onto the pattern of receptive field selectivities along the visual pathway. With easy conditions, learning generalizes across orientation and retinal position, matching the spatial generalization of higher visual areas. As task difficulty increases, learning becomes more specific with respect to both orientation and position, matching the fine spatial retinotopy exhibited by lower areas. Consequently, we enjoy the benefits of learning generalization when possible, and of fine grain but specific training when necessary. The dynamics of learning show a corresponding feature. Improvement begins with easy cases (when the subject is allowed long processing times) and only subsequently proceeds to harder cases. This learning cascade implies that easy conditions guide the learning of hard ones. Taken together, the specificity and dynamics suggest that learning proceeds as a countercurrent along the cortical hierarchy. Improvement begins at higher generalizing levels, which, in turn, direct harder-condition learning to the subdomain of their lower-level inputs. As predicted by this reverse hierarchy model, learning can be effective using only difficult trials, but on condition that learning onset has previously been enabled. A single prolonged presentation suffices to initiate learning. We call this single-encounter enabling effect 'eureka'.

Task difficulty and the specificity of perceptual learning.

Task difficulty and the specificity of perceptual learning

The performance of adult humans in simple visual tasks improves dramatically with practice. This improvement is highly specific to basic attributes of the trained stimulus, suggesting that the underlying changes occur at low-level processing stages in the brain, where different orientations and spatial frequencies are handled by separate channels. We asked whether these practice effects are determined solely by activity in stimulus-driven mechanisms or whether high-level attentional mechanisms, which are linked to the perceptual task, might control the learning process. We found that practicing one task did not improve performance in an alternative task, even though both tasks used exactly the same visual stimuli but depended on different stimulus attributes (either orientation of local elements or global shape). Moreover, even when the experiment was designed so that the same responses were associated with the same stimuli (although subjects were instructed to attend to the attribute underlying one task), learning did not transfer from one task to the other. These results suggest that specific high-level attentional mechanisms, controlling changes at early visual processing levels, are essential in perceptual learning.

Merav Ahissar

Papers

View from the top: hierarchies and reverse hierarchies in the visual system.

The Reverse Hierarchy Theory of Visual Perceptual Learning

Task difficulty and the specificity of perceptual learning.

Task difficulty and the specificity of perceptual learning

Attentional control of early perceptual learning