Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.

/pdf/image-quality-assessment-from-error-visibility-to-structural-1rlwcqe34t.pdf

Image quality assessment: from error visibility to structural similarity

A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.

/pdf/a-model-of-saliency-based-visual-attention-for-rapid-scene-wv0n3yqxix.pdf

A model of saliency-based visual attention for rapid scene analysis

https://surfer.nmr.mgh.harvard.edu/ftp/articles/dale99-recon1.pdf

Cortical surface-based analysis. I. Segmentation and surface reconstruction

Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors Modern computational techniques (ie, public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validity of the results It is difficult, nay impossible, for researchers to check the accuracy of software by reading the source code; ground truth test datasets are needed Computational reproducibility means providing software so that for the same input anyone obtains the same result, right or wrong Computational validity means obtaining the right result for the ground-truth test data We describe a framework for validating and sharing software implementations, and we illustrate its usage with an example application: population receptive field (pRF) methods for functional MRI data The framework is composed of three main components implemented with containerization methods to guarantee computational reproducibility In our example pRF application, those components are: (1) synthesis of fMRI time series from ground-truth pRF parameters, (2) implementation of four public pRF analysis tools and standardization of inputs and outputs, and (3) report creation to compare the results with the ground truth parameters The framework was useful in identifying realistic conditions that lead to imperfect parameter recovery in all four pRF implementations, that would remain undetected using classic validation methods We provide means to mitigate these problems in future experiments A computational validation framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely upon a few agreed upon packages We hope that the framework will be helpful to validate other critical neuroimaging algorithms, as having a validation framework helps (1) developers to build new software, (2) research scientists to verify the software’s accuracy, and (3) reviewers to evaluate the methods used in publications and grants

https://www.biorxiv.org/content/biorxiv/early/2020/01/08/2020.01.07.897991.full.pdf

A validation framework for neuroimaging software: The case of population receptive fields.

published by Elsevier, and the attached copy is provided by Elsevier for the author's benefit and for the benefit of the author's institution, for non-commercial research and educational use including without limitation use in instruction at your institution, sending it to specific colleagues who you know, and providing a copy to your institution’s administrator. All other uses, reproduction and distribution, including without limitation commercial reprints, selling or licensing copies or access, or posting on open internet sites, your personal or institution’s website or repository, are prohibited. For exceptions, permission may be sought for such use through Elsevier's permissions site at: http://www.elsevier.com/locate/permissionusematerial Wandell B A, Dumoulin S O and Brewer A A (2009) Visual Cortex in Humans. In: Squire LR (ed.) Encyclopedia of Neuroscience, volume 10, pp. 251-257. Oxford: Academic Press.

Visual Cortex in Humans

As the number of image sensor pixels in camera phones increases, users expect camera phone image quality to be comparable to digital still cameras. The mobile imaging industry is aware, however, that simply packing more pixels into the very limited camera module size need not improve image quality. When the size of a pixel array is fixed, increasing the number of image sensor pixels decreases pixel size and thus photon count. Attempts to compensate for the reduction in light sensitivity by increasing exposure durations increase the amount of handheld camera motion blur which effectively reduces spatial resolution. Perversely, what started as an attempt to increase spatial resolution by increasing the number of pixels, may result in a reduction of effective spatial resolution. In this paper, we evaluate how the performance of mobile imaging systems changes with shrinking pixel size, and we propose to replace the widely misused “physical pixel count” with a new metric that we refer to as the “effective pixel count” (EPC). We use this new metric to analyze design tradeoffs for four different pixel sizes (2.8µm, 2.2µm, 1.75µm and 1.4µm) and two different imaging arrays (1/3.2 and 1/8 inch). We show that optical diffraction and camera motion make 1.4 µm pixels perceptually less effective than larger pixels and that this problem is exacerbated by the introduction of zoom optics. Image stabilization optics can increase the effective pixel count and is, therefore, an important feature to include in a mobile imaging system.

/pdf/mobile-imaging-the-big-challenge-of-the-small-pixel-8od7a93kvf.pdf

Mobile Imaging: the big challenge of the small pixel

A learning technique is provided that learns how to process images by exploiting the spatial and spectral correlations inherent in image data to process and enhance images. Using a training set of input and desired output images, regression coefficients are learned that are optimal for a predefined estimation function that estimates the values at a pixel of the desired output image using a collection of similarly located pixels in the input image. Application of the learned regression coefficients is fast, robust to noise, adapts to the particulars of a dataset, and generalizes to a large variety of applications. The invention enables the use of image sensors with novel color filter array designs that offer expanded capabilities beyond existing sensors and take advantage of typical high pixel counts.

Learning of image processing pipeline for digital imaging devices

Color balancing algorithms for digital image processing require an accurate estimate of the physical properties of the ambient scene illuminant, particularly its spectral power distribution. An active imaging method and apparatus estimate these properties by emitting modulated light with a known spectral power distribution into a region of a scene. Backscattered light is detected and demodulated to separate output representing active emitter light from output representing ambient illuminant light. Using the emitter-related detector output and the known emitter spectral power distribution, the surface spectral reflectance function of the illuminated scene region can be computed. Subsequently, the spectral power distribution of the ambient scene illuminant can be computed from the surface reflectance function and the illuminant-related output of the detector. The estimated spectral power distribution can be used in standard color balancing algorithms for digital or film images.

Brian A. Wandell

Papers

A validation framework for neuroimaging software: The case of population receptive fields.

Visual Cortex in Humans

Mobile Imaging: the big challenge of the small pixel

Learning of image processing pipeline for digital imaging devices

System and method for estimating physical properties of objects and illuminants in a scene using modulated light emission