Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.

/pdf/image-quality-assessment-from-error-visibility-to-structural-1rlwcqe34t.pdf

Image quality assessment: from error visibility to structural similarity

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image super-resolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4x upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method.

/pdf/photo-realistic-single-image-super-resolution-using-a-2y0438rlmd.pdf

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep convolutional neural network (CNN) that takes the low-resolution image as the input and outputs the high-resolution one. We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network. But unlike traditional methods that handle each component separately, our method jointly optimizes all layers. Our deep CNN has a lightweight structure, yet demonstrates state-of-the-art restoration quality, and achieves fast speed for practical on-line usage. We explore different network structures and parameter settings to achieve trade-offs between performance and speed. Moreover, we extend our network to cope with three color channels simultaneously, and show better overall reconstruction quality.

Image Super-Resolution Using Deep Convolutional Networks

We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

The utility of Computed Tomography scans in predicting perinephric fat invasion of renal cell carcinoma

Omnidirectional image quality assessment (OIQA) aims to predict the perceptual quality of omnidirectional images that cover the whole 180$\times$360$^{\circ}$ viewing range of the visual environment. Here we propose a blind/no-reference OIQA method named S$^2$ that bridges the gap between low-level statistics and high-level semantics of omnidirectional images. Specifically, statistic and semantic features are extracted in separate paths from multiple local viewports and the hallucinated global omnidirectional image, respectively. A quality regression along with a weighting process is then followed that maps the extracted quality-aware features to a perceptual quality prediction. Experimental results demonstrate that the proposed S$^2$ method offers highly competitive performance against state-of-the-art methods.

Blind Omnidirectional Image Quality Assessment: Integrating Local Statistics and Global Semantics

With the wider availability of High Dynamic Range (HDR) Wide Colour Gamut (WCG) content, both consumers and content producers have become more concerned about the preservation of creative intent. While the accurate representation of colour plays a vital role in preserving creative intent, there are relatively fewer objective image and video quality assessment methods that are available which consider the colour quality. This paper will study the effect of surrounding luminance on perception of a colour stimulus, specifically, whether the perceptual uniformity is preserved in colour spaces and colour differencing methods as the surrounding luminance changes. The work presented in this paper provides important information and insight required for the future development of a successful colour quality assessment model.

Variation of Perceived Colour Difference Under Different Surround Luminance

In the field of intelligent security, due to the negative effect of complicated influence factors, such as low video quality, different size of the target and occlusion, target detection is hard to be well-applicated in real life. Based on the above problems, this paper proposes a multi-layer feature cascade aggregation pyramid network (MCA-FPN) on the basis of Faster RCNN, which can fully combine the different level of semantic information to generate optimized feature maps and improve the expression ability of different size of features finally. Besides, to remove the negative effect from the imbalance distribution of samples, this paper discusses a new sample balanced weighted loss function SB-Loss to increase convergence speed and make the training process more efficiency. Finally, the method proposed by this paper has been experimented on the Pascal VOC dataset, with a maximum accuracy of 86.0%, which is highly competitive in this research area. 

Target Detection Algorithm Based on Feature Optimization and Sample Equalization

Zinc hydroxyfluoride (ZnOHF) is a newly found resistive semiconductor used as a gas-sensing material with excellent selectivity to NO2 because of its unique energy band structure. In this paper, Al3+ doping and UV radiation were used to further improve the gas-sensing performance of ZnOHF. The optimized 0.5 at.% Al-ZnOHF sample exhibits improved sensitivity to 10 ppm NO2 at a lower temperature (100 °C) under UV assistance, as well as a short response/recovery time (35 s/96 s). The gas-sensing mechanism demonstrates that Al3+ doping increases electron concentration and promotes electron transfer of the nanorods by reducing the bandgap of ZnOHF, and the photogenerated electrons and holes with high activity under UV irradiation provide new reaction routes in the gas adsorption and desorption process, effectively promoting the gas-sensing process. The synergistic effect of Al3+ and UV radiation contribute to the enhanced performance of Al-ZnOHF.

https://www.mdpi.com/1996-1944/16/9/3577/pdf?version=1683437825

Zhou Wang

Papers

The utility of Computed Tomography scans in predicting perinephric fat invasion of renal cell carcinoma

Blind Omnidirectional Image Quality Assessment: Integrating Local Statistics and Global Semantics

Variation of Perceived Colour Difference Under Different Surround Luminance

Target Detection Algorithm Based on Feature Optimization and Sample Equalization

Highly Efficient NO2 Sensors Based on Al-ZnOHF under UV Assistance