In this paper, we propose a pseudo-sequence-based two-dimensional (2-D) hierarchical coding structure for light-field image compression. In the proposed scheme, we first decompose the light-field image into multiple views and organize them into a 2-D coding structure according to the spatial coordinates of the corresponding microlens. Then, we mainly develop three algorithms to optimize the 2-D coding structure. First, we propose a 2-D hierarchical coding structure with a limited number of reference frames to exploit the intercorrelations among various views. To be more specific, we divide all the views into four quadrants, and all the views are encoded one quadrant after another to reduce the reference buffer size as much as possible. Inside each quadrant, all the views are encoded hierarchically in both horizontal and vertical directions to fully exploit the correlations among different views. Second, we propose to use the distance between the current view and its reference views instead of the picture order count difference as the criterion for selecting better reference frames for each inter view. The distance-based criterion is also applied to the motion vector scaling process to obtain more accurate motion vector predictors. Third, an optimal bit allocation algorithm taking the influence of the various views on the following encoding views into account is proposed to further exploit the intercorrelations among various views and improve coding efficiency. The entire scheme is implemented in the reference software of high efficiency video coding. The experimental results demonstrate that the proposed novel pseudo-sequence-based 2-D hierarchical structure can achieve maximum 28.4% bit-rate savings compared with the previous pseudo-sequence-based light-field image compression method.

Pseudo-Sequence-Based 2-D Hierarchical Coding Structure for Light-Field Image Compression

Low-delay hierarchical coding structure (LD-HCS), as one of the most important components in the latest High Efficiency Video Coding (HEVC) standard, greatly improves coding performance. It groups consecutive P/B frames into different layers and encodes them with different quantization parameters (QPs) and reference mechanisms in such a way that temporal dependency among frames can be exploited. However, due to varying characteristics of video contents, temporal dependency among coding units differs significantly from each other in the same or different layers, while a fixed LD-HCS scheme cannot take full advantage of the dependency, leading to a substantial loss in coding performance. This paper addresses the temporally dependent rate distortion optimization (RDO) problem by attempting to exploit varying temporal dependency of different units. First, the temporal relationship of different frames under the LD-HCS is examined, and hierarchical temporal propagation chains are constructed to represent the temporal dependency among coding units in different frames. Then, a hierarchical temporally dependent RDO scheme is developed specifically for the LD-HCS based on a source distortion propagation model. Experimental results show that our proposed scheme can achieve 2.5% and 2.3% BD-rate gain in average compared with the HEVC codec under the same configuration of P and B frames, respectively, with a negligible increase in encoding time. Furthermore, coupled with QP adaption, our proposed method can achieve higher coding gains, e.g., with multi-QP optimization, about 5.4% and 5.0% BD-rate saving in average over the HEVC codec under the same setting of P and B frames, respectively.

Temporally Dependent Rate-Distortion Optimization for Low-Delay Hierarchical Video Coding

Rate–distortion optimization is a crucial instrument to achieve the best performance of information lossy video coding based on rate–distortion theory, and it is generally realized by Lagrange optimization using a Lagrangian multiplier An efficient Lagrangian multiplier selection method considering temporal dependency (LMS-TD for short) is introduced in this paper for the random-access (RA) configuration in H265/HEVC (High Efficiency Video Coding) The temporal relationship of distortion and bit-rate is analyzed for videos coded using an H265/HEVC compliant coder (HM-140) with the RA configuration, based on which an optimized set of Lagrangian multipliers is determined The LMS-TD method achieves an average BD-rate (Bjontegaard bit-rate) reduction of 459% Picture quality evaluation was also performed using an objective perceptual video quality measurement technique, ie, the Structural Similarity Index (SSIM), where the LMS-TD method demonstrated superior performance in video quality

An efficient Lagrangian multiplier selection method based on temporal dependency for rate-distortion optimization in H.265/HEVC

In the state-of-the-art H.265/HEVC video coding standard, the motion vector (MV) resolution is fixed to be 1/4-pel for the entire video sequence, while the inherent video characteristics, e.g., texture complexities and motion activities, have been largely ignored. Obviously such strategy may not suffice the demand of high-accuracy motion compensation. In this paper, we propose a specially designed rate–distortion model in terms of the MV resolution by taking the video characteristics into consideration. In particular, the MV resolution selection is formulated as a rate–distortion optimization problem by analyzing the rate-distortion cost of each MV resolution candidate. To further improve the coding performance, the progressive MV resolution strategy is employed, where the optimal progressive MV resolution is determined by decision trees constructed with the rate–distortion model. In this manner, a novel adaptive progressive motion vector resolution selection scheme can be realized and the MV resolution can be adaptively adjusted based on the properties of local content. Extensive experiments and comparisons show that the proposed algorithm significantly improves the coding performance, and 1.8% BD-rate gain on average has been achieved without introducing any noticeable computational complexity.

Adaptive Progressive Motion Vector Resolution Selection Based on Rate–Distortion Optimization

This paper proposes a rate control algorithm by selecting a proper quantization parameter (QP) based on perceptual luminance adaptation in a single-loop encoding fashion. In this paper, the proposed algorithm uses the visual characteristics of humans to adaptively decide the number of bits available for a pixel. Then, a base QP is selected based on the proposed R–λ model. The proposed rate control determines the range of QP based on the base QP and calculates the maximum and minimum bpps within that range. The optimal bpp is obtained from the bpp range by considering the visual characteristics of the human being, and the QP value is determined by the optical bpp through the proposed R–λ model. In the proposed rate control algorithm, the QP value is selected based on the R–λ model by considering the perceptual luminance adaptation model at the CTU level. The number of target bits is decided to decide the QP, subject to visual sensitivity based on JND thresholds. With the use of the proposed rate control algorithm, bits for non-noticeable regions can be saved, and the remaining bits can be consumed for perceptually noticeable regions to enhance the overall subjective quality with the similar amount of total bits. The proposed method shows lower average variances of bits and PSNR fluctuations. Also, the proposed method achieves an approximately 0.19 higher MOS value on average under DSCQS test, compared with the conventional R–λ model-based rate control algorithm.

A perceptual rate control algorithm based on luminance adaptation for HEVC encoders

It is known that, in the current hybrid video coding structure, spatial and temporal prediction techniques are extensively used which introduce strong dependency among coding units. Such dependency poses a great challenge to perform a global rate-distortion optimization (RDO) when encoding a video sequence. RDO is usually performed in a way that coding efficiency of each coding unit is optimized independently without considering dependeny among coding units, leading to a suboptimal coding result for the whole sequence. In this paper, we investigate the inter-frame dependent RDO, where the impact of coding performance of the current coding unit on that of the following frames is considered. Accordingly, an inter-frame dependent rate-distortion optimization scheme is proposed and implemented on the newest video coding standard High Efficiency Video Coding (HEVC) platform. Experimental results show that the proposed scheme can achieve about 3.19% BD-rate saving in average over the state-of-the-art HEVC codec (HM15.0) in the low-delay B coding structure, with no extra encoding time. It obtains a significantly higher coding gain than the multiple QP (±3) optimization technique which would greatly increase the encoding time by a factor of about 6. Coupled with the multiple QP optimization, the proposed scheme can further achieve a higher BD-rate saving of 5.57% and 4.07% in average than the HEVC codec and the multiple QP optimization enabled HEVC codec, respectively.

Frdric Dufaux

Papers

Inter-frame dependent rate-distortion optimization using lagrangian multiplier adaption