Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.

/pdf/image-quality-assessment-from-error-visibility-to-structural-1rlwcqe34t.pdf

Image quality assessment: from error visibility to structural similarity

Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

http://ahvazdownload.persiangig.com/document/article/39.pdf

A wavelet tour of signal processing

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.

/pdf/image-to-image-translation-with-conditional-adversarial-5058t24et2.pdf

Image-to-Image Translation with Conditional Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.

Deposits of clastic carbonate-dominated (calciclastic) sedimentary slope systems in the rock record have been identified mostly as linearly-consistent carbonate apron deposits, even though most ancient clastic carbonate slope deposits fit the submarine fan systems better. Calciclastic submarine fans are consequently rarely described and are poorly understood. Subsequently, very little is known especially in mud-dominated calciclastic submarine fan systems. Presented in this study are a sedimentological core and petrographic characterisation of samples from eleven boreholes from the Lower Carboniferous of Bowland Basin (Northwest England) that reveals a >250 m thick calciturbidite complex deposited in a calciclastic submarine fan setting. Seven facies are recognised from core and thin section characterisation and are grouped into three carbonate turbidite sequences. They include: 1) Calciturbidites, comprising mostly of highto low-density, wavy-laminated bioclast-rich facies; 2) low-density densite mudstones which are characterised by planar laminated and unlaminated muddominated facies; and 3) Calcidebrites which are muddy or hyper-concentrated debrisflow deposits occurring as poorly-sorted, chaotic, mud-supported floatstones. These

Phd by thesis

We propose a no-reference algorithm to assess the comfort associated with viewing stereo images and videos. The proposed measure of 3D quality of experience is shown to correlate well with human perception of quality on a publicly available dataset of 3D images/videos and human subjective scores. The proposed measure extracts statistical features from disparity and disparity gradient maps as well as indicators of spatial activity from images. For videos, the measure utilizes these spatial features along with motion compensated disparity differences to predict quality of experience. To the best of our knowledge the proposed approach is the first attempt in algorithmically assessing the subjective quality of experience on a publicly available dataset.

Algorithmic assessment of 3D quality of experience for images and videos

Natural scene statistics (NSSs) provide powerful, perceptually relevant tools that have been successfully used for image quality analysis of visible light images. Since NSS capture statistical regularities that arise from the physical world, they are relevant to long wave infrared (LWIR) images, which differ from visible light images mainly by the wavelengths captured at the imaging sensors. We show that NSS models of bandpass LWIR images are similar to those of visible light images, but with different parameterizations. Using this difference, we exploit the power of NSS to successfully distinguish between LWIR images and visible light images. In addition, we study distortions unique to LWIR and find directional models useful for detecting the halo effect, simple bandpass models useful for detecting hotspots, and combinations of these models useful for measuring the degree of non-uniformity present in many LWIR images. For local distortion identification and measurement, we also describe a method for generating distortion maps using NSS features. To facilitate our evaluation, we analyze the NSS of LWIR images under pristine and distorted conditions, using four databases, each captured with a different IR camera. Predicting human performance for assessing distortion and quality in LWIR images is critical for task efficacy. We find that NSS features improve human targeting task performance prediction. Furthermore, we conducted a human study on the perceptual quality of noise-and blur-distorted LWIR images and create a new blind image quality predictor for IR images.

/pdf/tasking-on-natural-statistics-of-infrared-images-1zf0coj0g3.pdf

Tasking on Natural Statistics of Infrared Images

Measuring digital picture quality, as perceived by human observers, is increasingly important in many applications in which humans are the ultimate consumers of visual information. Standard dynamic range (SDR) images provide 8 b/color/pixel. High dynamic range (HDR) images, usually created from multiple exposures of the same scene, can provide 16 or 32 b/color/pixel, but need to be tonemapped to SDR for display on standard monitors. Multiexposure fusion (MEF) techniques bypass HDR creation by fusing an exposure stack directly to SDR images to achieve aesthetically pleasing luminance and color distributions. Many HDR and MEF databases have a relatively small number of images and human opinion scores, obtained under stringently controlled conditions, thereby limiting realistic viewing. Moreover, many of these databases are intended to compare tone-mapping algorithms, rather than being specialized for developing and comparing image quality assessment models. To overcome these challenges, we conducted a massively crowdsourced online subjective study. The primary contributions described in this paper are: 1) the new ESPL-LIVE HDR Image Database that we created containing diverse images obtained by tone-mapping operators and MEF algorithms, with and without post-processing; 2) a large-scale subjective study that we conducted using a crowdsourced platform to gather more than 300 000 opinion scores on 1811 images from over 5000 unique observers; and 3) a detailed study of the correlation performance of the state-of-the-art no-reference image quality assessment algorithms against human opinion scores of these images. The database is available at http://signal.ece.utexas.edu/%7Edebarati/HDRDatabase.zip .

Large-Scale Crowdsourced Study for Tone-Mapped HDR Pictures

We propose graph-driven approaches to image segmentation by developing diffusion processes defined on arbitrary graphs We formulate a solution to the image segmentation problem modeled as the result of infectious wavefronts propagating on an image-driven graph, where pixels correspond to nodes of an arbitrary graph By relating the popular susceptible-infected-recovered epidemic propagation model to the Random Walker algorithm, we develop the normalized random walker and a lazy random walker variant The underlying iterative solutions of these methods are derived as the result of infections transmitted on this arbitrary graph The main idea is to incorporate a degree-aware term into the original Random Walker algorithm in order to account for the node centrality of every neighboring node and to weigh the contribution of every neighbor to the underlying diffusion process Our lazy random walk variant models the tendency of patients or nodes to resist changes in their infection status We also show how previous work can be naturally extended to take advantage of this degree-aware term, which enables the design of other novel methods Through an extensive experimental analysis, we demonstrate the reliability of our approach, its small computational burden and the dimensionality reduction capabilities of graph-driven approaches Without applying any regular grid constraint, the proposed graph clustering scheme allows us to consider pixel-level, node-level approaches, and multidimensional input data by naturally integrating the importance of each node to the final clustering or segmentation solution A software release containing implementations of this paper and supplementary material can be found at: http://cvspcsntuagr/research/GraphClustering/ 

Graph-Driven Diffusion and Random Walk Schemes for Image Segmentation

The human visual system (HVS) is highly non-uniform in sampling, coding, processing and understanding The spatial resolution of the HVS is highest around the point of fixation (foveation point) and decreases rapidly with increasing eccentricity Currently, most image quality measurement methods are designed for uniform resolution images These methods do not correlate well with the perceived foveated image quality Wavelet analysis delivers a convenient way to simultaneously examine localized spatial as well as frequency information We developed a new image quality metric called foveated wavelet image quality index (FWQI) in the wavelet transform domain FWQI considers multiple factors of the HVS, including the spatial variance of the contrast sensitivity function, the spatial variance of the local visual cut-off frequency, the variance of human visual sensitivity in different wavelet subbands, and the influence of the viewing distance on the display resolution and the HVS features FWQI can be employed for foveated region of interest (ROI) image coding and quality enhancement We show its effectiveness by using it as a guide for optimal bit assignment of an embedded foveated image coding system The coding system demonstrates very good coding performance and scalability in terms of foveated objective as well as subjective quality measurement© (2001) COPYRIGHT SPIE--The International Society for Optical Engineering Downloading of the abstract is permitted for personal use only

Alan C. Bovik

Papers

Algorithmic assessment of 3D quality of experience for images and videos

Tasking on Natural Statistics of Infrared Images

Large-Scale Crowdsourced Study for Tone-Mapped HDR Pictures

Graph-Driven Diffusion and Random Walk Schemes for Image Segmentation

Foveated wavelet image quality index