We propose a natural scene statistic-based distortion-generic blind/no-reference (NR) image quality assessment (IQA) model that operates in the spatial domain. The new model, dubbed blind/referenceless image spatial quality evaluator (BRISQUE) does not compute distortion-specific features, such as ringing, blur, or blocking, but instead uses scene statistics of locally normalized luminance coefficients to quantify possible losses of “naturalness” in the image due to the presence of distortions, thereby leading to a holistic measure of quality. The underlying features used derive from the empirical distribution of locally normalized luminances and products of locally normalized luminances under a spatial natural scene statistic model. No transformation to another coordinate frame (DCT, wavelet, etc.) is required, distinguishing it from prior NR IQA approaches. Despite its simplicity, we are able to show that BRISQUE is statistically better than the full-reference peak signal-to-noise ratio and the structural similarity index, and is highly competitive with respect to all present-day distortion-generic NR IQA algorithms. BRISQUE has very low computational complexity, making it well suited for real time applications. BRISQUE features may be used for distortion-identification as well. To illustrate a new practical application of BRISQUE, we describe how a nonblind image denoising algorithm can be augmented with BRISQUE in order to perform blind image denoising. Results show that BRISQUE augmentation leads to performance improvements over state-of-the-art methods. A software release of BRISQUE is available online: http://live.ece.utexas.edu/research/quality/BRISQUE_release.zip for public use and evaluation.

/pdf/no-reference-image-quality-assessment-in-the-spatial-domain-2mnlvf25ig.pdf

No-Reference Image Quality Assessment in the Spatial Domain

It is an important task to faithfully evaluate the perceptual quality of output images in many applications, such as image compression, image restoration, and multimedia streaming. A good image quality assessment (IQA) model should not only deliver high quality prediction accuracy, but also be computationally efficient. The efficiency of IQA metrics is becoming particularly important due to the increasing proliferation of high-volume visual data in high-speed networks. We present a new effective and efficient IQA model, called gradient magnitude similarity deviation (GMSD). The image gradients are sensitive to image distortions, while different local structures in a distorted image suffer different degrees of degradations. This motivates us to explore the use of global variation of gradient based local quality map for overall image quality prediction. We find that the pixel-wise gradient magnitude similarity (GMS) between the reference and distorted images combined with a novel pooling strategy-the standard deviation of the GMS map-can predict accurately perceptual image quality. The resulting GMSD algorithm is much faster than most state-of-the-art IQA methods, and delivers highly competitive prediction accuracy. MATLAB source code of GMSD can be downloaded at http://www4.comp.polyu.edu.hk/~cslzhang/IQA/GMSD/GMSD.htm.

/pdf/gradient-magnitude-similarity-deviation-a-highly-efficient-3jmna5s35n.pdf

Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index

Perceptual visual quality metrics: A survey

Perceptual image quality assessment (IQA) aims to use computational models to measure the image quality in consistent with subjective evaluations. Visual saliency (VS) has been widely studied by psychologists, neurobiologists, and computer scientists during the last decade to investigate, which areas of an image will attract the most attention of the human visual system. Intuitively, VS is closely related to IQA in that suprathreshold distortions can largely affect VS maps of images. With this consideration, we propose a simple but very effective full reference IQA method using VS. In our proposed IQA model, the role of VS is twofold. First, VS is used as a feature when computing the local quality map of the distorted image. Second, when pooling the quality score, VS is employed as a weighting function to reflect the importance of a local region. The proposed IQA index is called visual saliency-based index (VSI). Several prominent computational VS models have been investigated in the context of IQA and the best one is chosen for VSI. Extensive experiments performed on four large-scale benchmark databases demonstrate that the proposed IQA index VSI works better in terms of the prediction accuracy than all state-of-the-art IQA indices we can find while maintaining a moderate computational complexity. The MATLAB source code of VSI and the evaluation results are publicly available online at http://sse.tongji.edu.cn/linzhang/IQA/VSI/VSI.htm.

VSI: a visual saliency-induced index for perceptual image quality assessment.

It is an important task to faithfully evaluate the perceptual quality of output images in many applications such as image compression, image restoration and multimedia streaming. A good image quality assessment (IQA) model should not only deliver high quality prediction accuracy but also be computationally efficient. The efficiency of IQA metrics is becoming particularly important due to the increasing proliferation of high-volume visual data in high-speed networks. We present a new effective and efficient IQA model, called gradient magnitude similarity deviation (GMSD). The image gradients are sensitive to image distortions, while different local structures in a distorted image suffer different degrees of degradations. This motivates us to explore the use of global variation of gradient based local quality map for overall image quality prediction. We find that the pixel-wise gradient magnitude similarity (GMS) between the reference and distorted images combined with a novel pooling strategy the standard deviation of the GMS map can predict accurately perceptual image quality. The resulting GMSD algorithm is much faster than most state-of-the-art IQA methods, and delivers highly competitive prediction accuracy.

Objective quality assessment methods use a computational (mathematical) model to provide estimates of subjective video quality. While such objective models may not mimic subjective opinions accurately in a general scenario, they can be reasonably effective in specific conditions/applications. Hence, they can be an important tool toward automating the testing and standardization of high dynamic range (HDR) video processing algorithms, especially when subjective tests may not be feasible. Therefore, this chapter deals with objective quality assessment of HDR content and elaborates on the issues and challenges that arise. We also discuss and present details of the existing efforts on the topic. Particularly, the focus is on full-reference HDR metrics which take as input two HDR signals (one of them is always assumed to be the reference). Hence, in the context of this chapter, the term “quality” can also be interpreted as “fidelity,” and both can be used interchangeably. Another use case is that of comparing HDR and low dynamic range signals, and this is needed, for instance, when HDR content is tone-mapped to be rendered on a low dynamic range display.

HDR Image and Video Quality Prediction

Spatial audio rendering employs Head Related Transfer Functions (HRTFs) for a realistic reproduction of the sound field. This requires upsampling of the HRTF. Given its popularity for the task of upsampling, a deep learning (DL) based upsampler can appear to be an attractive solution for the said problem. We, however, argue that it is more meaningful to rely on explicit system modeling, and not depend exclusively on DL based data fitting for the said problem.

Upsampling of Personalized HRTF for Spatial Audio Rendering: Why Deep Learning is Problematic?

The landscape in the context of several signal processing applications and even education [1] appears to be significantly affected by the emergence of machine learning (ML) and particularly deep learning (DL). The main reason for this is the ability of DL to model complex and unknown relationships between signals and the tasks of interest. In particular, supervised DL algorithms have been fairly successful at recognizing perceptually or semantically useful signal information in different applications (e.g., identifying objects or regions of interest from image/video signals or recognizing spoken words from speech signals, such as in speech recognition). In all of these tasks, the training process uses labeled data to learn a mapping function (typically implicitly) from signals to the desired information (class label or target label). The trained DL model is then expected to correctly recognize/classify relevant information in a given test signal. Therefore, a DL-based framework is, in general, very appealing since the features and characteristics of the required mapping are learned almost exclusively from the data without resorting to explicit model/system development.

Manish Narwaria

Papers

HDR Image and Video Quality Prediction

Upsampling of Personalized HRTF for Spatial Audio Rendering: Why Deep Learning is Problematic?

Explainable Machine Learning: The importance of a system-centric perspective [Lecture Notes]