We develop a framework for assessing the quality of stereoscopic images that have been afflicted by possibly asymmetric distortions. An intermediate image is generated which when viewed stereoscopically is designed to have a perceived quality close to that of the cyclopean image. We hypothesize that performing stereoscopic QA on the intermediate image yields higher correlations with human subjective judgments. The experimental results confirm the hypothesis and show that the proposed framework significantly outperforms conventional 2D QA metrics when predicting the quality of stereoscopically viewed images that may have been asymmetrically distorted.

/pdf/full-reference-quality-assessment-of-stereopairs-accounting-1d9v8eb9j8.pdf

Full-reference quality assessment of stereopairs accounting for rivalry

We develop a no-reference binocular image quality assessment model that operates on static stereoscopic images. The model deploys 2D and 3D features extracted from stereopairs to assess the perceptual quality they present when viewed stereoscopically. Both symmetric- and asymmetric-distorted stereopairs are handled by accounting for binocular rivalry using a classic linear rivalry model. The NSS features are used to train a support vector machine model to predict the quality of a tested stereopair. The model is tested on the LIVE 3D Image Quality Database, which includes both symmetric- and asymmetric-distorted stereoscopic 3D images. The experimental results show that our proposed model significantly outperforms the conventional 2D full-reference QA algorithms applied to stereopairs, as well as the 3D full-reference IQA algorithms on asymmetrically distorted stereopairs.

/pdf/no-reference-quality-assessment-of-natural-stereopairs-4utg62iahb.pdf

No-Reference Quality Assessment of Natural Stereopairs

Finding ways to monitor and control the perceptual quality of digital visual media has become a pressing concern as the volume being transported and viewed continues to increase exponentially. This paper discusses the principles and methods of modern algorithms for automatically predicting the quality of visual signals. By casting the problem as analogous to assessing the efficacy of a visual communication system, it is possible to divide the quality assessment problem into understandable modeling subproblems. Along the way, we will visit models of natural images and videos, of visual perception, and a broad spectrum of applications.

/pdf/automatic-prediction-of-perceptual-image-and-video-quality-1276xl9y2c.pdf

Automatic Prediction of Perceptual Image and Video Quality

The objective approaches of 3D image quality assessment play a key role for the development of compression standards and various 3D multimedia applications. The quality assessment of 3D images faces more new challenges, such as asymmetric stereo compression, depth perception, and virtual view synthesis, than its 2D counterparts. In addition, the widely used 2D image quality metrics (e.g., PSNR and SSIM) cannot be directly applied to deal with these newly introduced challenges. This statement can be verified by the low correlation between the computed objective measures and the subjectively measured mean opinion scores (MOSs), when 3D images are the tested targets. In order to meet these newly introduced challenges, in this paper, besides traditional 2D image metrics, the binocular integration behaviors-the binocular combination and the binocular frequency integration, are utilized as the bases for measuring the quality of stereoscopic 3D images. The effectiveness of the proposed metrics is verified by conducting subjective evaluations on publicly available stereoscopic image databases. Experimental results show that significant consistency could be reached between the measured MOS and the proposed metrics, in which the correlation coefficient between them can go up to 0.88. Furthermore, we found that the proposed metrics can also address the quality assessment of the synthesized color-plus-depth 3D images well. Therefore, it is our belief that the binocular integration behaviors are important factors in the development of objective quality assessment for 3D images.

Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors

Perceptual quality assessment is a challenging issue in 3D signal processing research. It is important to study 3D signal directly instead of studying simple extension of the 2D metrics directly to the 3D case as in some previous studies. In this paper, we propose a new perceptual full-reference quality assessment metric of stereoscopic images by considering the binocular visual characteristics. The major technical contribution of this paper is that the binocular perception and combination properties are considered in quality assessment. To be more specific, we first perform left-right consistency checks and compare matching error between the corresponding pixels in binocular disparity calculation, and classify the stereoscopic images into non-corresponding, binocular fusion, and binocular suppression regions. Also, local phase and local amplitude maps are extracted from the original and distorted stereoscopic images as features in quality assessment. Then, each region is evaluated independently by considering its binocular perception property, and all evaluation results are integrated into an overall score. Besides, a binocular just noticeable difference model is used to reflect the visual sensitivity for the binocular fusion and suppression regions. Experimental results show that compared with the relevant existing metrics, the proposed metric can achieve higher consistency with subjective assessment of stereoscopic images.

Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

In this paper, we propose a quality metric for color stereo images. The concept of our metric is inspired by the behavior of simple and complex cells located in the primary visual cortex. These cells are responsible for merging left and right retinal images. To replicate the task performed by these cells, we adopted an approach based on spatial-frequency transform with the processing of selective orientations. From that, a model that calculates the binocular energy contained in the left and right retinal images has been proposed. The amplitude variation of the binocular energy defines the quality criterion of the reconstructed depth within the Human Visual System (HVS). Finally, from the experimental results, the used criterion seems to be correlated to human judgment obtained by psychophysical tests.

Towards a perceptual quality metric for color stereo images

3D applications are very popular nowadays. They allow to bring new sensations (e.g., cinema, gaming, etc) and new ways for analyzing data (e.g., video-surveillance, pattern recognition, etc). The large availability of 3D data ia better understanding of human 3D perception in order to improve the quality of 3D visual data, increase the visual comfort and avoid visual fatigue and visual illness. In this paper, we explore the binocular perception through its various indices. The focus is put on the binocular energy and its evolution with regard to image impairments. Two types of cells are explored, that is, simple and complex cells responsible of the sensory fusion in the visual cortex. A model is proposed for these cells in order to estimate the binocular energy from color stereo-pairs. The integration of stereoscopic constraints such as unicity, coherence and occlusion allows to refine the proposed model described previously by taking into account the occluded and the non-occluded information. A deep experimentation demonstrates the efficiency of the described modeling. The estimated binocular energy presents a correlation with the impairment level caused by compression or noise.

Binocular Energy Estimation Based on Properties of the Human Visual System

Stereoscopic imaging technologies are seen as the next generation of visual presentation, improving the quality of experience of the viewer. It uses two different sequences acquired from two regular cameras or from a regular camera with an additional specific depth camera. This means that the size of data is at least doubled. Thus, the coding process becomes very crucial. In this framework, we propose a stereoscopic coder based on visual properties. The matching of two images is computed by a binocular energy model based on the simple and complex cells functions allowing the fusion of both retinal images in the visual cortex. Mathematical functions were used to reproduce the behavior of these cells particularly complex wavelet transform (CWT) and bandelet transform. Our coder output is a disparity map, a residual image and the reference image. The innovative part of this work lies in a matching technique based on the binocular energy. The results are presented through comparative curves with one of the most known coder in literature.

Optimizing the disparity map by the integration of HVS binocular properties for efficient coding of stereoscopic images

This paper presents a new metric for the quality evaluation of stereoscopic images. The proposed metric is inspired from the simple and complex cells response, in which the matching of the left and right retinal images is performed, in the visual cortex. In order to estimate the binocular energy, we used the bandelet transform which is computed on complex wavelet transform. By using the complex wavelet transform, we were able to model the behavior of the simple and complex cells. The proposed model allows the estimation of the degradation of the binocular energy according to the impairments of the right and the left images. Subjective tests have been performed to validate the results obtained by our metric. The results shows a high correlation between the predicted score and the human judgement.

Rafik Bensalma

Papers

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Towards a perceptual quality metric for color stereo images

Binocular Energy Estimation Based on Properties of the Human Visual System

Optimizing the disparity map by the integration of HVS binocular properties for efficient coding of stereoscopic images

A stereoscopic quality metric based on binocular perception