There is disclosed a method, apparatus and computer program product in which at least one view component of a first type and at least one view component of a second type are obtained. The order of the texture view component and the depth view component in an access unit is determined and at least one indication of the order is encoded. The coding of the view components is adapted on the basis of the order. There is also disclosed a method, apparatus and computer program product in which at least one encoded view component of a first type and at least one encoded view component of a second type are received. Also at least one encoded indication of the order of the view components is received. The at least one encoded indication is decoded and the decoding of the view component sis adapted on the basis of the order.

Method and apparatus for video coding

Iso/iec jtc 1/sc 29画像符号化

Multiview video coding (MVC) has attracted great attention from industries and research institutes. MVC is used to encode stereoscopic video streams for 3D playout systems such as 3D television, digital cinema, and IP network applications. MVC is an extended version of H.264/AVC that improves the performance of multiview videos. Yet, when compared with single-view video coding, MVC consumes much more time when encoding large amounts of data. Speed-up algorithms, therefore, are essential for realizing related applications. This paper presents a fast mode decision algorithm to avoid the high computational complexity of MVC. The proposed approach aims to reduce candidate modes and make mode decision process more efficient. The minimum and maximum values of rate-distortion cost (RD cost) in the previously encoded view are used to compute a threshold for each mode in the current view. Compared with joint multiview video coding, the experimental results demonstrate that the proposed algorithm provides an average of 79% in time savings with negligible bit rate increase and peak signal-to-noise ratio decrease.

Fast Mode Decision Algorithm Through Inter-View Rate-Distortion Prediction for Multiview Video Coding System

3D High Efficiency Video Coding (3D-HEVC) is an extension of the HEVC standard for coding of multiview videos and depth maps. It inherits the same quadtree coding structure as HEVC for both components, which allows recursively splitting into four equal-sized coding units (CU). One of 11 different prediction modes is chosen to code a CU in inter-frames. Similar to the joint model of H.264/AVC, the mode decision process in HM (reference software of HEVC) is performed using all the possible depth levels and prediction modes to find the one with the least rate distortion cost using a Lagrange multiplier. Furthermore, both motion estimation and disparity estimation need to be performed in the encoding process of 3D-HEVC. Those tools achieve high coding efficiency, but lead to a significant computational complexity. In this article, we propose a fast mode decision algorithm for 3D-HEVC. Since multiview videos and their associated depth maps represent the same scene, at the same time instant, their prediction modes are closely linked. Furthermore, the prediction information of a CU at the depth level X is strongly related to that of its parent CU at the depth level X-1 in the quadtree coding structure of HEVC since two corresponding CUs from two neighboring depth levels share similar video characteristics. The proposed algorithm jointly exploits the inter-view coding mode correlation, the inter-component (texture-depth) correlation and the inter-level correlation in the quadtree structure of 3D-HEVC. Experimental results show that our algorithm saves 66p encoder runtime on average with only a 0.2p BD-Rate increase on coded views and 1.3p BD-Rate increase on synthesized views.

A 3D-HEVC Fast Mode Decision Algorithm for Real-Time Applications

Graphical abstractDisplay Omitted HighlightsA fuzzy rule-based system operates as a selector of color constancy algorithms.The system selects among the White-Patch, Gray-World and Gray-Edge algorithms.The method attains a high rate of correct selection according to the actual scene.Two problems are addressed simultaneously: color constancy and color enhancement.The framework can be used in engineering applications, like video surveillance. This work introduces a fuzzy rule-based system operating as a selector of color constancy algorithms for the enhancement of dark images. In accordance with the actual content of an image, the system selects among three color constancy algorithms, the White-Patch, the Gray-World and the Gray-Edge. These algorithms have been considered because of their accurate remotion of the illuminant, besides showing an outstanding color enhancement on images. The design of the rule-based system is not a trivial task because several features are involved in the selection. Our proposal consists in a fuzzy system, modeling the decision process through simple rules. This approach can handle large amounts of information and is tolerant to ambiguity, while addressing the problem of dark image enhancement. The methodology consists in two main stages. Firstly, a training protocol determines the fuzzy rules, according to features computed from a subset of training images taken from the SFU Laboratory dataset. We choose carefully twelve image features for the formulation of the rules: seven color features, three texture descriptors, and two lighting-content descriptors. In the rules, the fuzzy sets are modeled using Gaussian membership functions. Secondly, experiments are carried out using Mamdani and Larsen fuzzy inferences. For a test image, a color constancy algorithm is selected according to the inference process and the rules previously defined. The results show that our method attains a high rate of correct selection of the most well-suited algorithm for the particular scene.

Automatic selection of color constancy algorithms for dark image enhancement by fuzzy rule-based reasoning

Multiview video plus depth (MVD) is a new 3D video format that would support 3D applications developed by MPEG. Such a format is a combination of texture video and associated depth maps. Consequently, for the efficient transmission of 3D video signals, the compression of texture video and also the depth maps is required. Since high computational complexity of jointly coding between the texture video and depth map is still an open question, this paper introduces a low complexity MVD coding algorithm that adaptively utilizes the texture and depth map correlation. Based on the correlation, we propose four efficient techniques, including depth information based fast mode size decision, adaptive disparity estimation in texture coding, motion vector sharing based on the texture image similarity correlation and the SKIP mode decision in depth coding. Experimental results show that the proposed algorithm can significantly reduce the computational complexity of MVD coding while improve the coding performance and achieve better rendering quality1.

Low complexity multiview video plus depth coding

A depth map represents three-dimensional (3D) scene information and is used to synthesize virtual views in 3D video. Since the quality of synthesized virtual views highly depends on the quality of depth map, efficient depth compression is crucial to realize the 3D video system. However compressing depth map using existing video coding techniques yields unacceptable distortions while rendering virtual views. To solve this problem, we propose an efficient depth map compression method for the view rendering, a novel distortion metric base on view rendering distortions instead of distortion of depth map itself. First, we derive relationships between distortions in coded depth map and rendered view. Then, a region based video characteristics distortion model is proposed for precisely estimation distortion in view synthesis. Finally, experimental results have shown that 1.8 dB coding gain in terms of PSNR and subjective quality improvement of synthesized views are achieved by the proposed method.

Efficient rendering distortion estimation for depth map compression

No-reference quality evaluation is more difficult because it does not refer to any image. However, in some cases, like virtual view images, can not find the reference image, no-reference method had to be used. With the free viewpoint video technology development and application, measuring the performance of the multiview video system is essential and virtual view image quality evaluation is one of the key technologies in the system. The characteristic of virtual view image quality evaluation is that most of the virtual view images haven't reference images and can't be assessed using general measures such as PSNR of full-reference. Therefore, this paper proposed a no-reference image quality evaluation method based on image power spectrum. Experimental results show that the proposed method is consistent with PSNR.

A no-reference image quality evaluation based on power spectrum

With more and more applications of Stereo/multiview video technology, Stereo/multiview video quality assessment is quite needed. In this paper, we have proposed a novel multiview video quality assessment method based on disparity and SSIM. We evaluate video quality from two aspects, stereo perception and image quality. Stereo perception is evaluated by the disparities, and image quality is got by the similarity of every view. The subjective experiments verified the rationality of the presented novel objective method.

A multiview video quality assessment method based on disparity and SSIM

Multiview video plus depth (MVD) is a new 3D video format that would support 3D video applications The primary goals are the high-quality reconstruction of an arbitrary number of views for 3D television (3DTV) and free viewpoint television (FTV) Its quality depends highly on the quality of depth image, a depth compression base B-spline interpolation is proposed for MVD to improve the 3D video coding (3DVC) The depth map is down sampling prior to multiview video coding (MVC) encoding and up sampling after decoding The results investigated by objective as well as subjective evaluations over several multi-view test sequences imply that the proposed methods reduce the bit-rate for depth coding and achieve better rendering quality

Yan Zhang

Papers

Low complexity multiview video plus depth coding

Efficient rendering distortion estimation for depth map compression

A no-reference image quality evaluation based on power spectrum

A multiview video quality assessment method based on disparity and SSIM

Reduced resolution depth compression for multiview video plus depth coding