A structural similarity (SSIM)-based game theory (GT) approach is proposed for rate-distortion (R-D) optimized CTU-level bit allocation in high efficiency video coding (HEVC). First, a SSIM-based bargaining game is formulated and the Nash bargaining solution (NBS) is proposed, in which a SSIM-based initial minimum utility is defined. Second, we propose a two-stage remaining bit refinement-based bit allocation scheme. The optimization scheme of the SSIM-based bargaining game sufficiently considers the different R-D characteristics of coding tree units (CTUs), in which the feasible utility set is proved to be convex based on the proposed SSIM-based utility and R-SSIM model. Compared with the other state-of-the-art CTU-level bit allocation methods, the R-D performance improvements on Bjontegaard delta bit-rate (BD-BR), Bjontegaard delta peak-signal-to-noise-ratio (BD-PSNR), and BD-SSIM metrics of the proposed method can averagely achieve significant gains, respectively. The achieved R-D performance gains have been very close to the coding performance limits from the FixedQP method. Moreover, the proposed SSIM-GT method also maintains good performances on quality smoothness, bit rate accuracy, and encoding complexity.

SSIM-Based Game Theory Approach for Rate-Distortion Optimized Intra Frame CTU-Level Bit Allocation

The field of assessing three-dimensional (3-D) visual experience is challenging. In this paper, we propose a new blind image quality assessment for stereoscopic images by using binocular guided quality lookup and visual codebook. To be more specific, in the training stage, we construct phase-tuned quality lookup (PTQL) and phase-tuned visual codebook (PTVC) from the binocular energy responses based on stimuli from different spatial frequencies, orientations, and phase shifts. In the test stage, blind quality pooling can be easily achieved by searching the PTQL and PTVC, and the quality score is obtained by averaging the largest values of all patch’s quality. Experimental results on three 3-D image quality assessment databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves high consistency alignment with subjective assessment and low-complexity pooling.

Blind Image Quality Assessment for Stereoscopic Images Using Binocular Guided Quality Lookup and Visual Codebook

We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE).

No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality.

Efficient Enhancement of Stereo Endoscopic Images Based on Joint Wavelet Decomposition and Binocular Combination

Blind quality estimator for 3D images based on binocular combination and extreme learning machine

This paper presents a novel full-reference Stereo Image Quality Assessment (SIQA) measure based on well understood characteristics of the human visual system (HVS), namely contrast sensitivity and frequency and directional selectivity. Additionally, the proposed metric takes into account the stereo interplay between the two views, where one view may affect our perception of the overall quality of the stereo image pair. Therefore, a Binocular Just Noticeable Difference (BJND) model is used to compute the distortion visibility threshold, and the binocular suppression theory is considered in the proposed metric. The scored 3D LIVE IQA database is used to evaluate the correlation of the proposed metric with the DMOS subjective score provided by the database. The obtained experimental results show that the proposed metric correlates much better with the DMOS score than the state-of-the-art metrics do.

/pdf/stereo-image-quality-assessment-using-a-binocular-just-3ojld5qnb9.pdf

Stereo image quality assessment using a binocular just noticeable difference model

The widespread use of 3D acquisition and display technologies has increased the interest of stereo image dataset in various application fields. As a result, it becomes necessary to have an efficient 3D quality assessment method to measure the human perception of stereoscopic images. While most of the state-of-the-art methods belong to the class of full-reference methods which require the original stereo images to be able to assess the quality, we propose in this paper a no-reference quality metric which does not require any information of the original stereo images. The proposed method operates in the wavelet transform domain and adopts a statistical framework to predict the quality of stereo images. More precisely, a joint wavelet decomposition is first performed on the stereo images to exploit simultaneously the intra and inter-views redundancies. A wavelet transform is also applied to their associated estimated disparity maps. Then, relevant features are extracted from the resulting wavelet subbands by resorting to appropriate statistical models. Simulations, carried out on the standard Live 3D image quality database, show that our proposed design model achieves significant improvement compared to the state-of-the-art 3D quality assessment methods. We present a novel no-reference method for stereo image quality assessment.The developed method operates in the wavelet transform domain.A joint multiscale decomposition is applied to the left and right views.Statistical features of both views as well as the disparity map are extracted.

No-reference stereo image quality assessment based on joint wavelet decomposition and statistical models

The recent developments in 3D display technology have opened new horizons and have raised a number of challenges related to the processing and coding of 3D media. Today, stereoscopic image technology is becoming widely used in many fields. The physical limitations of image acquisition systems, however, make stereoscopic technology far from being the most widely accepted solution. Furthermore, the depth/disparity extreme ranges may subject the viewers' eyes to additional strain, causing more discomfort. To address this issue, we propose in this paper to improve stereoscopic image quality by a novel contrast enhancement method that combines local edges and depth information. The contrast is increased locally, at specific depth levels for left and right views. The increase of contrast is controlled based on the depth information, and aims at promoting the nearest objects in the 3D scene. The results obtained from a psychophysical experiment are encouraging and show that the proposed method produces stereo images that are less stressful on the eyes, thus providing more pleasant viewing experience.

/pdf/combining-depth-information-and-local-edge-detection-for-5be6blyn1m.pdf

Combining depth information and local edge detection for stereo image enhancement

In this paper, we present efficient bit allocation methods for stereo image coding purpose. Since the common idea behind most of the existing stereo compression schemes consists of encoding a reference and residual images as well as a disparity map, we mainly focus on the bit allocation issue between the reference and residual images. Generally, this problem is solved in an empirical manner by looking for the optimal rates leading to the minimum distortion value. Thanks to recent approximations of the entropy and distortion functions, we propose accurate and fast bit allocation schemes appropriate for the open-loop- and closed-loop-based stereo coding structures. Experimental results show the benefits which can be drawn from the proposed bit allocation methods.

Efficient Inter-View Bit Allocation Methods for Stereo Image Coding

A novel stereo image compression method based on disparity-compensated residual image structure analysis is presented in this paper. The residual image resulting from the subtraction of the target image by the predicted one is exploited in this scheme. Unlike natural images, for which 2D-orthogonal transforms such as 2D-DCT provides good energy compaction in the frequency domain, stereo residual images exhibit directional and sparse components making them difficult to compress with classical methods. We propose to exploit the directional properties of the residual image, to compress it more efficiently, by using 1D Directional Discrete Cosine Transform (1D-DDCT). This 1D transform is applied to non-overlapping blocks of the residual image, along four directions: i.e. horizontal, vertical, positive diagonal and negative diagonal. The experimental results show that the proposed method, on some test images, outperforms the considered state-of-the-art methods.

Walid Hachicha

Papers

Stereo image quality assessment using a binocular just noticeable difference model

No-reference stereo image quality assessment based on joint wavelet decomposition and statistical models

Combining depth information and local edge detection for stereo image enhancement

Efficient Inter-View Bit Allocation Methods for Stereo Image Coding

1D directional DCT-based stereo residual compression