Image quality assessment (IQA) has been a topic of intense research over the last several decades. With each year comes an increasing number of new IQA algorithms, extensions of existing IQA algorithms, and applications of IQA to other disciplines. In this article, I first provide an up-to-date review of research in IQA, and then I highlight several open challenges in this field. The first half of this article provides discuss key properties of visual perception, image quality databases, existing full-reference, no-reference, and reduced-reference IQA algorithms. Yet, despite the remarkable progress that has been made in IQA, many fundamental challenges remain largely unsolved. The second half of this article highlights some of these challenges. I specifically discuss challenges related to lack of complete perceptual models for: natural images, compound and suprathreshold distortions, and multiple distortions, and the interactive effects of these distortions on the images. I also discuss challenges related to IQA of images containing nontraditional, and I discuss challenges related to the computational efficiency. The goal of this article is not only to help practitioners and researchers
keep abreast of the recent advances in IQA, but to also raise awareness of the key limitations of current IQA knowledge.

/pdf/seven-challenges-in-image-quality-assessment-past-present-4y53obboox.pdf

Seven Challenges in Image Quality Assessment: Past, Present, and Future Research

The prediction of an image quality metric (IQM) should be consistent with subjective human evaluation. As the human visual system (HVS) is critical to visual perception, modeling of the HVS is regarded as the most suitable way to achieve perceptual quality predictions. Sparse coding that is equivalent to independent component analysis (ICA) can provide a very good description of the receptive fields of simple cells in the primary visual cortex, which is the most important part of the HVS. With this inspiration, a quality metric called sparse feature fidelity (SFF) is proposed for full-reference image quality assessment (IQA) on the basis of transformation of images into sparse representations in the primary visual cortex. The proposed method is based on the sparse features that are acquired by a feature detector, which is trained on samples of natural images by an ICA algorithm. In addition, two strategies are designed to simulate the properties of the visual perception: 1) visual attention and 2) visual threshold. The computation of SFF has two stages: training and fidelity computation, in addition, the fidelity computation consists of two components: feature similarity and luminance correlation. The feature similarity measures the structure differences between the two images, whereas the luminance correlation evaluates brightness distortions. SFF also reflects the chromatic properties of the HVS, and it is very effective for color IQA. The experimental results on five image databases show that SFF has a better performance in matching subjective ratings compared with the leading IQMs.

Sparse Feature Fidelity for Perceptual Image Quality Assessment

Perceptual image quality assessment by independent feature detector

In this letter, we estimate perceived image quality using sparse representations obtained from generic image databases through an unsupervised learning approach. A color space transformation, a mean subtraction, and a whitening operation are used to enhance descriptiveness of images by reducing spatial redundancy; a linear decoder is used to obtain sparse representations; and a thresholding stage is used to formulate suppression mechanisms in a visual system. A linear decoder is trained with 7 GB worth of data, which corresponds to 100 000 8 × 8 image patches randomly obtained from nearly 1000 images in the ImageNet 2013 database. A patch-wise training approach is preferred to maintain local information. The proposed quality estimator UNIQUE is tested on the LIVE, the Multiply Distorted LIVE, and the TID 2013 databases and compared with 13 quality estimators. Experimental results show that UNIQUE is generally a top performing quality estimator in terms of accuracy, consistency, linearity, and monotonic behavior.

/pdf/unique-unsupervised-image-quality-estimation-9js3ivrgxm.pdf

UNIQUE: Unsupervised Image Quality Estimation

We develop a Generalized Gaussian scale mixture (GGSM) model of the wavelet coefficients of natural and distorted images. The GGSM model, which is more general than and which subsumes the Gaussian scale mixture (GSM) model, is shown to be a better representation of the statistics of the wavelet coefficients of both natural as well as distorted images. We demonstrate the utility of the model by applying it to various image processing applications, including blind distortion identification and no reference image quality assessment (NR-IQA). Similar to the GSM model, the GGSM model is useful for motivating the use of local divisive energy normalization, especially when the wavelet coefficients are computed on distorted pictures. We show that the GGSM model can lead to improved performance in distortion-related applications, while providing a more principled approach to the statistical processing of distorted image signals. The software release of a GGSM-based NR-IQA approach called DIIVINE-GGSM is available online at http://live.ece.utexas.edu/research/quality/diivine-ggsm.zip for further experimentation.

Generalized Gaussian scale mixtures: A model for wavelet coefficients of natural images

Image quality assessment (IQA) is of fundamental importance to numerous image processing applications. Generally, image quality metrics (IQMs) regard image quality as fidelity or similarity with a reference image in some perceptual space. Such a full-reference IQA method is a kind of comparison that involves measuring the similarity or difference between two signals in a perceptually meaningful way. Modeling of the human visual system (HVS) has been regarded as the most suitable way to achieve perceptual quality predictions. In fact, natural image statistics can be an effective approach to simulate the HVS, since statistical models of natural images reveal some important response properties of the HVS. A useful statistical model of natural images is sparse coding, which is equivalent to independent component analysis (ICA). It provides a very good description of the receptive fields of simple cells in the primary visual cortex. Therefore, such a statistical model can be used to simulate the visual processing at the level of the visual cortex when designing IQMs. In this paper, we propose a fidelity criterion for IQA that relates image quality with the correlation between a reference and a distorted image in the form of sparse code. The proposed visual signal fidelity metric, which is called sparse correlation coefficient (SCC), is motivated by the need to capture the correlation between two sets of outputs from a sparse model of simple cell receptive fields. The SCC represents the correlation between two visual signals of images in a cortical visual space. The experimental results after both polynomial and logistic regression demonstrate that SCC is superior to recent state-of-the-art IQMs both in single-distortion and cross-distortion tests.

Hua-wen Chang

Papers

Sparse correlation coefficient for objective image quality assessment