Existing blind image quality assessment (BIQA) methods are mostly opinion-aware. They learn regression models from training images with associated human subjective scores to predict the perceptual quality of test images. Such opinion-aware methods, however, require a large amount of training samples with associated human subjective scores and of a variety of distortion types. The BIQA models learned by opinion-aware methods often have weak generalization capability, hereby limiting their usability in practice. By comparison, opinion-unaware methods do not need human subjective scores for training, and thus have greater potential for good generalization capability. Unfortunately, thus far no opinion-unaware BIQA method has shown consistently better quality prediction accuracy than the opinion-aware methods. Here, we aim to develop an opinion-unaware BIQA method that can compete with, and perhaps outperform, the existing opinion-aware methods. By integrating the features of natural image statistics derived from multiple cues, we learn a multivariate Gaussian model of image patches from a collection of pristine natural images. Using the learned multivariate Gaussian model, a Bhattacharyya-like distance is used to measure the quality of each image patch, and then an overall quality score is obtained by average pooling. The proposed BIQA method does not need any distorted sample images nor subjective quality scores for training, yet extensive experiments demonstrate its superior quality-prediction performance to the state-of-the-art opinion-aware BIQA methods. The MATLAB source code of our algorithm is publicly available at www.comp.polyu.edu.hk / $\sim $ cslzhang/IQA/ILNIQE/ILNIQE.htm.

A Feature-Enriched Completely Blind Image Quality Evaluator

Color science is a multidisciplinary field with broad applications in industries such as digital imaging, coatings and textiles, food, lighting, archiving, art, and fashion. Accurate definition and measurement of color appearance is a challenging task that directly affects color reproduction in such applications. Color Appearance Models addresses those challenges and offers insight into the preferred solutions. Extensive research on the human visual system (HVS) and color vision has been performed in the last century, and this book contains a good overview of the most important and relevant literature regarding color appearance models.

https://www.spiedigitallibrary.org/journals/Journal-of-Electronic-Imaging/volume-23/issue-2/029901/Color-Appearance-Models/10.1117/1.JEI.23.2.029901.pdf

Color Appearance Models

Neural Style Transfer has shown very exciting results enabling new forms of image manipulation. Here we extend the existing method to introduce control over spatial location, colour information and across spatial scale. We demonstrate how this enhances the method by allowing high-resolution controlled stylisation and helps to alleviate common failure cases such as applying ground textures to sky regions. Furthermore, by decomposing style into these perceptual factors we enable the combination of style information from multiple sources to generate new, perceptually appealing styles from existing ones. We also describe how these methods can be used to more efficiently produce large size, high-quality stylisation. Finally we show how the introduced control measures can be applied in recent methods for Fast Neural Style Transfer.

/pdf/controlling-perceptual-factors-in-neural-style-transfer-zn37ye7zy5.pdf

Controlling Perceptual Factors in Neural Style Transfer

This article details two approaches to compute barycenters of measures using 1-D Wasserstein distances along radial projections of the input measures. The first method makes use of the Radon transform of the measures, and the second is the solution of a convex optimization problem over the space of measures. We show several properties of these barycenters and explain their relationship. We show numerical approximation schemes based on a discrete Radon transform and on the resolution of a non-convex optimization problem. We explore the respective merits and drawbacks of each approach on applications to two image processing problems: color transfer and texture mixing.

/pdf/sliced-and-radon-wasserstein-barycenters-of-measures-s297t9q7hs.pdf

Sliced and Radon Wasserstein Barycenters of Measures

We live in a dynamic visual world where the appearance of scenes changes dramatically from hour to hour or season to season. In this work we study "transient scene attributes" -- high level properties which affect scene appearance, such as "snow", "autumn", "dusk", "fog". We define 40 transient attributes and use crowdsourcing to annotate thousands of images from 101 webcams. We use this "transient attribute database" to train regressors that can predict the presence of attributes in novel images. We demonstrate a photo organization method based on predicted attributes. Finally we propose a high-level image editing method which allows a user to adjust the attributes of a scene, e.g. change a scene to be "snowy" or "sunset". To support attribute manipulation we introduce a novel appearance transfer technique which is simple and fast yet competitive with the state-of-the-art. We show that we can convincingly modify many transient attributes in outdoor scenes.

Transient attributes for high-level understanding and editing of outdoor scenes

To achieve the complex task of interpreting what we see, our brains rely on statistical regularities and patterns in visual data. Knowledge of these regularities can also be considerably useful in visual computing disciplines, such as computer vision, computer graphics, and image processing. The field of natural image statistics studies the regularities to exploit their potential and better understand human vision. With numerous color figures throughout, Image Statistics in Visual Computing covers all aspects of natural image statistics, from data collection to analysis to applications in computer graphics, computational photography, image processing, and art. The authors keep the material accessible, providing mathematical definitions where appropriate to help readers understand the transforms that highlight statistical regularities present in images. The book also describes patterns that arise once the images are transformed and gives examples of applications that have successfully used statistical regularities. Numerous references enable readers to easily look up more information about a specific concept or application. A supporting website also offers additional information, including descriptions of various image databases suitable for statistics. Collecting state-of-the-art, interdisciplinary knowledge in one source, this book explores the relation of natural image statistics to human vision and shows how natural image statistics can be applied to visual computing. It encourages readers in both academic and industrial settings to develop novel insights and applications in all disciplines that relate to visual computing.

Image Statistics in Visual Computing

The statistics of natural images have attracted the attention of researchers in a variety of fields and have been used as a means to better understand the human visual system and its processes. A number of algorithms in computer graphics, vision and image processing take advantage of such statistical findings to create visually more plausible results. With this report we aim to review the state of the art in image statistics and discuss existing and potential applications within computer graphics and related areas.

A Survey of Image Statistics Relevant to Computer Graphics

Few tone mapping operators (TMOs) take color management into consideration, limiting compression to luminance values only. This could lead to changes in image chroma and hues, which are typically managed with a post-processing step. However, current post-processing techniques for tone reproduction do not explicitly consider the target display gamut. Gamut mapping, on the other hand, deals with mapping images from one color gamut to another, usually smaller, gamut but has traditionally focused on smaller scale, chromatic changes. The authors present a combined gamut- and tone-management framework for color-accurate reproduction of high dynamic range images that can prevent hue and luminance shifts while taking gamut boundaries into consideration. Their approach is conceptually and computationally simple, parameter-free, and compatible with existing TMOs.

A Gamut-Mapping Framework for Color-Accurate Reproduction of HDR Images.

Mobile phones and tablets are rapidly gaining significance as omnipresent image and video capture devices. In this context we present an algorithm that allows such devices to capture high dynamic range HDR video. The design of the algorithm was informed by a perceptual study that assesses the relative importance of motion and dynamic range. We found that ghosting artefacts are more visually disturbing than a reduction in dynamic range, even if a comparable number of pixels is affected by each. We incorporated these findings into a real-time, adaptive metering algorithm that seamlessly adjusts its settings to take exposures that will lead to minimal visual artefacts after recombination into an HDR sequence. It is uniquely suitable for real-time selection of exposure settings. Finally, we present an off-line HDR reconstruction algorithm that is matched to the adaptive nature of our real-time metering approach.

Motion aware exposure bracketing for HDR video

Computer graphics as well as related disciplines often benefit from understanding the human visual system and its input. In this paper, we study statistical regularities of both conventional as well as high dynamic range images, and find that several commonly held wisdoms regarding natural image statistics do not directly apply to high dynamic range data. These results have implications for both the study of human vision, as well as for the design of algorithms in computer graphics and computer vision that rely on such statistical regularities.

Tania Pouli

Papers

Image Statistics in Visual Computing

A Survey of Image Statistics Relevant to Computer Graphics

A Gamut-Mapping Framework for Color-Accurate Reproduction of HDR Images.

Motion aware exposure bracketing for HDR video

Statistical regularities in low and high dynamic range images