Image quality assessment (IQA) aims to use computational models to measure the image quality consistently with subjective evaluations. The well-known structural similarity index brings IQA from pixel- to structure-based stage. In this paper, a novel feature similarity (FSIM) index for full reference IQA is proposed based on the fact that human visual system (HVS) understands an image mainly according to its low-level features. Specifically, the phase congruency (PC), which is a dimensionless measure of the significance of a local structure, is used as the primary feature in FSIM. Considering that PC is contrast invariant while the contrast information does affect HVS' perception of image quality, the image gradient magnitude (GM) is employed as the secondary feature in FSIM. PC and GM play complementary roles in characterizing the image local quality. After obtaining the local quality map, we use PC again as a weighting function to derive a single quality score. Extensive experiments performed on six benchmark IQA databases demonstrate that FSIM can achieve much higher consistency with the subjective evaluations than state-of-the-art IQA metrics.

/pdf/fsim-a-feature-similarity-index-for-image-quality-assessment-42wiyfodry.pdf

FSIM: A Feature Similarity Index for Image Quality Assessment

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The outside world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the governing laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Several lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual \"filling in,\" visual stability despite eye movements, change blindness, sensory substitution, and color perception.

A sensorimotor account of vision and visual consciousness-Authors' Response-Acting out our sensory experience

Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The outside world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the governing laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Several lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual “filling in,” visual stability despite eye movements, change blindness, sensory substitution, and color perception.

/pdf/a-sensorimotor-account-of-vision-and-visual-consciousness-1o8kf5szfs.pdf

A sensorimotor account of vision and visual consciousness

An object recognition system based on the dynamic link architecture, an extension to classical artificial neural networks (ANNs), is presented. The dynamic link architecture exploits correlations in the fine-scale temporal structure of cellular signals to group neurons dynamically into higher-order entities. These entities represent a rich structure and can code for high-level objects. To demonstrate the capabilities of the dynamic link architecture, a program was implemented that can recognize human faces and other objects from video images. Memorized objects are represented by sparse graphs, whose vertices are labeled by a multiresolution description in terms of a local power spectrum, and whose edges are labeled by geometrical distance vectors. Object recognition can be formulated as elastic graph matching, which is performed here by stochastic optimization of a matching cost function. The implementation on a transputer network achieved recognition of human faces and office objects from gray-level camera images. The performance of the program is evaluated by a statistical analysis of recognition results from a portrait gallery comprising images of 87 persons. >

Distortion invariant object recognition in the dynamic link architecture

VISUAL scientists have long sought to explain why the world remains stable during saccades, the ballistic eye-movements that continually displace the retinal image at fast but resolvable1 velocities. An early suggestion was that vision may be actively suppressed during saccades2, but experimental support has been variable3–5. Here we present evidence that saccadic suppression does occur, but that it is selective for patterns modulated in luminance at low spatial frequencies. Patterns of higher spatial frequency, and equiluminant patterns (modulated only in colour) at all spatial frequencies were not suppressed during saccades, but actually enhanced. The selectivity of the suppression suggests that it is confined to the colour-blind magnocellular stream (which provides the dominant input to motion centres and areas involved with attention), where it could dull the otherwise disturbing sense of fast low-spatial-frequency image motion. Masking studies suggest that the suppression precedes the site of contrast masking and may therefore occur early in visual processing, possibly as early as the lateral geniculate nucleus.

Selective suppression of the magnocellular visual pathway during saccadic eye movements

Changes in visual perception at the time of saccades

Saccadic eye movements, in which the eye moves rapidly between two resting positions, shift the position of our retinal images. If our perception of the world is to remain stable, the visual directions associated with retinal sites, and others they report to, must be updated to compensate for changes in the point of gaze. It has long been suspected that this compensation is achieved by a uniform shift of coordinates driven by an extra-retinal position signal, although some consider this to be unnecessary. Considerable effort has been devoted to a search for such a signal and to measuring its time course and accuracy. Here, by using multiple as well as single targets under normal viewing conditions, we show that changes in apparent visual direction anticipate saccades and are not of the same size, or even in the same direction, for all parts of the visual field. We also show that there is a compression of visual space sufficient to reduce the spacing and even the apparent number of pattern elements. The results are in part consistent with electrophysiological findings of anticipatory shifts in the receptive fields of neurons in parietal cortex and superior colliculi.

Compression of visual space before saccades.

There is now considerable evidence that space is compressed when stimuli are flashed shortly before or after the onset of a saccadic eye movement. Here we report that short intervals of time between two successive perisaccadic visual (but not auditory) stimuli are also underestimated, indicating a compression of perceived time. We were even more surprised that in a critical interval before saccades, perceived temporal order is consistently reversed. The very similar time courses of spatial and temporal compression suggest that both are mediated by a common neural mechanism, probably related to the predictive shifts that occur in receptive fields of many visual areas at the time of saccades.

/pdf/saccadic-eye-movements-cause-compression-of-time-as-well-as-1tt5iatikf.pdf

Saccadic eye movements cause compression of time as well as space

The prevailing explanation of Mach bands, the paradoxical bands of light and dark seen where luminance gradients meet plateaux, is that they are due to lateral inhibition in the visual system1–4. This explanation equates Mach bands with distortions in a processed luminance distribution due to selective attenuation of low frequency components. But square waveforms exhibit no Mach bands2,5,6, although they should also be distorted after processing. Measurements of the contrast required to see Mach bands in trapezoidal waveforms and manipulations of their spectra lead us to conclude that phase relationships between Fourier components are important to the structure we perceive. A model based on the odd and even symmetry of visual receptive fields explains our results.

M. Concetta Morrone

Papers

Selective suppression of the magnocellular visual pathway during saccadic eye movements

Changes in visual perception at the time of saccades

Compression of visual space before saccades.

Saccadic eye movements cause compression of time as well as space

Mach bands are phase dependent