Making a “Completely Blind” Image Quality Analyzer
Summary (3 min read)
Introduction
- Consumers are drowning in digital visual content and finding ways to review and control of the quality of digital photographs is becoming quite challenging.
- Their model requires knowledge of the expected image distortions.
- The authors contribution in this direction is the development of a NSS-based modeling framework for OU-DU NR IQA design, resulting in a first of a kind NSS-driven blind OU-DU IQA model which does not require exposure to distorted images a priori, nor any training on human opinion scores.
II. NO REFERENCE OPINION-UNAWARE
- The authors new NR OU-DU IQA model is based on constructing a collection of ‘quality aware’ features and fitting them to 2 (a) (b) Fig.
- The marked blocks in the images (a) and (b) depict instances of natural image patches selected using a local sharpness measure.
- The quality aware features are derived from a simple but highly regular natural scene statistic (NSS) model.
- The quality of a given test image is then expressed as the distance between a multivariate Gaussian (MVG) fit of the NSS features extracted from the test image, and a MVG model of the quality aware features extracted from the corpus of natural images.
A. Spatial Domain NSS
- IQA model is founded on perceptually relevant spatial domain NSS features extracted from local image patches that effectively capture the essential loworder statistics of natural images.
- The coefficients (1) have been observed to reliably follow a Gaussian distribution when computed from natural images that have suffered little or no apparent distortion [10].
- This ideal model, however, is violated when the images do not derive from a natural source (e.g. computer graphics) or when natural images are subjected to unnatural distortions.
- Therefore, BRISQUE is limited to the types of distortions it has been tuned to.
- By comparison, the NIQE Index is not tied to any specific distortion type, yet, as will be shown, delivers nearly comparable predictive power on the same distortions the BRISQUE index has been trained on, with a similar low complexity.
B. Patch Selection
- Once the image coefficients (1) are computed, the image is partitioned into P ×P patches.
- There is a loss of resolution due to defocus blur in parts of most images due to the limited depth of field (DOF) of any single-lens camera.
- This subset of patches is then used to construct a model of the statistics of natural image patches.
- The threshold T is picked to be a fraction p of the peak patch sharpness over the image.
- Examples of this kind of patch selection are shown in Fig.
C. Characterizing Image Patches
- Given a collection of natural image patches selected as above, their statistics are characterized by ‘quality aware’ NSS features computed from each selected patch [3].
- Prior studies of NSS based image quality have shown that the generalized Gaussian distribution effectively captures the behavior of the coefficients (1) of natural and distorted versions of them [13].
- The signs of the transformed image coefficients (1) have been observed to follow a fairly regular structure.
- The products of neighboring coefficients are well-modeled as following a zero mode asymmetric generalized Gaussian distribution (AGGD) [15]: f(x; γ, βl, βr) =.
- All features are computed at two scales to capture multiscale behavior, by low pass filtering and downsampling by a factor of 2, yielding a set of 36 features.
D. Multivariate Gaussian Model
- Images were selected from copyright free Flickr data and from the Berkeley image segmentation database [17] making sure that no overlap occurs with the test image content.
- The images may be viewed at http://live.ece.utexas.edu/research/quality/pristinedata.zip.
E. NIQE Index
- The new OU-DU IQA index, called NIQE, is applied by computing the 36 identical NSS features from patches of the same size P×P from the image to be quality analyzed, fitting them with the MVG model (9), then comparing its MVG fit to the natural MVG model.
- The sharpness criterion (4) is not applied to these patches because loss of sharpness in distorted images is indicative of distortion and neglecting them would lead to incorrect evaluation of the distortion severity.
A. Correlation with Human Judgments of Visual Quality
- To test the performance of the NIQE index, the authors used the LIVE IQA database [2] of 29 reference images and 779 distorted images spanning five different distortion categories – JPEG and JPEG2000 (JP2K) compression, additive white Gaussian noise (WN), Gaussian blur (blur) and a Rayleigh fast fading channel distortion (FF).
- Since all of the OA IQA approaches that the authors compare NIQE to require a training procedure to calibrate the regressor module, they divided the LIVE database randomly into chosen subsets for training and testing.
- This train-test procedure was 4 repeated 1000 times to ensure that there was no bias due to the spatial content used for training.
- The authors use Spearman’s rank ordered correlation coefficient , and Pearson’s correlation coefficient (LCC) to test the model.
- This is a fairly remarkable demonstration of the relationship between quantified image naturalness and perceptual image quality.
B. Number of Natural Images
- Such an analysis provides an idea of the quality prediction power of the NSS features and how well they generalize with respect to image content.
- To undertake this evaluation, the authors varied the number of natural images K from which patches are selected and used for model fitting.
- Figure 2 shows the performance against the number of images.
- It may be observed that a stable natural model can be obtained using a small set of images.
IV. CONCLUSION
- The authors have created a first of a kind blind IQA model that assesses image quality without knowledge of anticipated distortions or human opinions of them.
- The quality of the distorted image is expressed as a simple distance metric between the model statistics and those of the distorted image.
- The new model outperforms FR IQA models and competes with top performing NR IQA trained on human judgments of known distorted images.
- Such a model has great potential to be applied in unconstained environments.
Did you find this useful? Give us your feedback
Citations
2,298 citations
915 citations
837 citations
Cites background or methods from "Making a “Completely Blind” Image Q..."
...The Natural Image Quality Evaluator (NIQE) [76] makes use of measurable deviations from statistical regularities observed in natural images, without exposure to distorted images....
[...]
...In each region, the winning algorithm is the one that achieves the best perceptual quality [77], evaluated by NIQE [76] and Ma [66]....
[...]
783 citations
Cites background from "Making a “Completely Blind” Image Q..."
...…Foundation of China under Grant 61201394, in part by the Shanghai Pujiang Program under Grant 13PJ1408700, in part by the Research Grants Council, Hong Kong, through the General Research Fund under Grant PolyU 5315/12E, and in part by the U.S. National Science Foundation under Grant IIS-1116656....
[...]
...It has been shown that natural scene statistics (NSS) are excellent indicators of the degree of quality degradation of distorted images [10]–[16]....
[...]
...…as: G2 (ω, θ) = e− ( log ( ω ω0 ))2 2σ2r · e− (θ−θ j)2 2σ2θ (10) where θ j = jπ/J , j = {0, 1, ..., J − 1} is the orientation angle, J is the number of orientations, ω0 is the center frequency, σr controls the filter’s radial bandwidth, and σθ determines the angular bandwidth of the filter....
[...]
676 citations
References
40,609 citations
22,840 citations
18,802 citations
6,505 citations
4,333 citations
Related Papers (5)
Frequently Asked Questions (9)
Q2. Why is the sharpness criterion not applied to these patches?
The sharpness criterion (4) is not applied to these patches because loss of sharpness in distorted images is indicative of distortion and neglecting them would lead to incorrect evaluation of the distortion severity.
Q3. What is the NR OU-DU IQA model?
Their ‘completely blind’ IQA model is founded on perceptually relevant spatial domain NSS features extracted from local image patches that effectively capture the essential loworder statistics of natural images.
Q4. What is the gamma function of the GGD?
The generalized Gaussian distribution (GGD) with zeromean is given by:f(x;α, β) = α2βΓ(1/α) exp(−(|x|β)α)(5)3where Γ(·) is the gamma function:Γ(a) =∫∞0ta−1e−tdt a >
Q5. How many features are computed from the natural image patches?
All features are computed at two scales to capture multiscale behavior, by low pass filtering and downsampling by a factor of 2, yielding a set of 36 features.
Q6. What data was used to determine the quality of the distorted image?
Images were selected from copyright free Flickr data and from the Berkeley image segmentation database [17] making sure that no overlap occurs with the test image content.
Q7. What is the significance of the variance field?
The variance field (3) has been largely ignored in the past in NSS based image analysis, but it is a rich source of structural image information that can be used to quantify local image sharpness.
Q8. How many parameters are extracted from the four orientations?
The mean of the distribution is also useful:η = (βr − βl) Γ( 2γ )Γ( 1γ ) . (8)By extracting estimates along the four orientations, 16 parameters are arrived at yielding 18 overall.
Q9. How many natural images were used to obtain the multivariate Gaussian model?
The authors selected a varied set of 125 natural images with sizes ranging from 480 × 320 to 1280 × 720 to obtain the multivariate Gaussian model.