There is disclosed a method, an apparatus, a server, a client and a non-transitory computer readable medium comprising a computer program stored therein for video coding and decoding. Depth pictures from a plurality of viewing angles are projected into a single viewing angle, making it possible to have pixel-wise joint filtering to be applied to all projected depth values. This approach enables to suppress the noise in the depth map data and provides improved performance for a view synthesis.

Apparatus, a method and a computer program for video coding and decoding

Mobile and IoT scenarios increasingly involve interactive and computation intensive contextual recognition. Existing optimizations typically resort to computation offloading or simplified on-device processing. Instead, we observe that the same application is often invoked on multiple devices in close proximity. Moreover, the application instances often processsimilar contextual data that map to thesame outcome. In this paper, we proposecross-device approximate computation reuse, which minimizes redundant computation by harnessing the "equivalence'' between different input values and reusing previously computed outputs with high confidence. We devise adaptive locality sensitive hashing (A-LSH) and homogenized k nearest neighbors (H-kNN). The former achieves scalable and constant lookup, while the latter provides high-quality reuse and tunable accuracy guarantee. We further incorporate approximate reuse as a service, called \name, in the computation offloading runtime. Extensive evaluation shows that, when given 95% accuracy target, \name\ consistently harnesses over 90% of reuse opportunities, which translates to reduced computation latency and energy consumption by a factor of 3 to 10.

FoggyCache: Cross-Device Approximate Computation Reuse

This paper proposes a novel feature-based photo album compression scheme for cloud storage. The main contribution of this paper is the use of local features rather than pixel values for analyzing and exploring the correlation between images. Unlike previously established image set compression schemes, we adopt content-based feature matching, which is invariant to scale, rotation, and robust to illumination changes, in both correlation estimation and redundancy reduction. Based on feature-based correlation analysis, we organize correlated images as a pseudo sequence by minimizing the predictive cost. Since pseudo sequences have much more complicated correlations than natural videos, we further propose a three-step prediction to reduce inter-image redundancy based on local features. We first propose a feature-based multi-model approach to reduce local geometric deformations between images, resulting in multiple deformed predictions. We then decrease the illumination difference between each deformed prediction and the target image with a photometric transformation. Finally, we reduce redundancy between images using a block-based motion compensation, similar to video compression. Experimental results show that our proposed feature-based coding scheme can be 10 times more efficient than individual JPEG compression.

Photo Album Compression for Cloud Storage Using Local Features

We propose a guided image contrast enhancement framework based on cloud images, in which the context- sensitive and context-free contrast is jointly improved via solving a multi-criteria optimization problem. In particular, the context-sensitive contrast is improved by performing advanced unsharp masking on the input and edge-preserving filtered images, while the context-free contrast enhancement is achieved by the sigmoid transfer mapping. To automatically determine the contrast enhancement level, the parameters in the optimization process are estimated by taking advantages of the retrieved images with similar content. For the purpose of automatically avoiding the involvement of low-quality retrieved images as the guidance, a recently developed no-reference image quality metric is adopted to rank the retrieved images from the cloud. The image complexity from the free-energy-based brain theory and the surface quality statistics in salient regions are collaboratively optimized to infer the parameters. Experimental results confirm that the proposed technique can efficiently create visually-pleasing enhanced images which are better than those produced by the classical techniques in both subjective and objective comparisons.

Guided Image Contrast Enhancement Based on Retrieved Images in Cloud

Different from all the previous reversible data hiding schemes, a completely novel one for the color image is proposed, which reversibly embeds messages into the color host image without modifying its corresponding gray version. The property of grayscale invariance is valuable, because many applications and image processing algorithms for color images are based on the corresponding gray versions, such as black and white printing, producing reading materials for color blind people, single-channel image processing, and so on. Thus, in terms of these applications and image processing algorithms, the presented scheme will make the generated color marked image be free for its further uses. In this paper, the unchanged gray version is utilized efficiently in both the embedding processes and the extracting processes. Messages are embedded into the red and blue channels of color image, and then the green channel is adjusted adaptively to remove the offsets from the gray version caused by modifying its red and blue channels. To return the adjusted green channel, error correcting bits guaranteeing the reversibility are regarded as one part of payloads to be recursively embedded. Therefore, the reversibility and the property of grayscale invariance are both achieved.

Reversible Data Hiding in Color Image With Grayscale Invariance

Spatially scalable video coding (SSVC in short) provides an efficient way to deliver one video at different resolutions. Based on the development of emerging High Efficiency Video Coding (HEVC), we propose a SSVC scheme to provide both single-loop and multi-loop solutions by enabling different inter-layer prediction mechanisms. Specifically, there are three basic inter-layer prediction modes, inter-layer intra, motion and residual prediction, which are first investigated for enhancement layer coding. These three modes provide a basic single-loop SSVC solution for low complexity applications. Beside the correlations explored in single-loop solution, we employ an extra patch learning-based prediction mode named P-mode to further improve coding performance. In P-mode, the temporal and spatial correlations of the reference frames are explored simultaneously by the visual patch-based learning and mapping at pixel level. These correlations help enhance the accuracy of the inter-layer frame prediction from the base-layer reconstruction within a multi-loop structure. The experimental results show the effectiveness of the proposed SSVC scheme compared with simulcast case.

Spatially Scalable Video Coding for HEVC

The biggest challenge in image set compression is how to efficiently remove the set redundancy among images as well as the redundancy inside a single image. Different from all the previous schemes, in this paper we are the first to propose a generic image set compression scheme which removes the set redundancy based on local features in addition to luminance values. The SIFT (Scale Invariant Feature Transform) descriptor which characterizes an image region invariant to scale and rotation is utilized in our scheme to measure and further enhance the correlation among images. Given an image set, we build a minimal cost prediction structure according to the SIFT-based prediction measure between images. We also utilize a SIFT-based global transformation to enhance the correlation between two images by aligning them to each other in terms of both geometry and intensity. The set redundancy and image redundancy are both further reduced by block-based motion estimation and rate-distortion optimal mechanism proposed in HEVC. Experimental results show that our new feature based scheme always produces the best result regardless the image set's properties.

Feature-based image set compression

Spatially scalable video coding (SSVC) provides an efficient way to transmit one video at different resolutions Based on the emerging High Efficiency Video Coding (HEVC), we propose an SSVC scheme to support both single-loop (SL) and multiloop (ML) solutions by enabling different interlayer prediction mechanisms Specifically, we employ two interlayer prediction modes: quadtree-based prediction mode (Q-mode) and learning-based prediction mode (L-mode) The Q-mode is investigated to exploit the interlayer redundancy based on the quadtree coding structure of HEVC Due to the high correlation between layers, Q-mode utilizes the coded information from the base layer quadtree, including coding unit split, prediction unit partition, motion information, and partial texture information of transform unit, to predict the enhancement layer quadtree By enabling Q-mode, we provide a basic SL solution for low complexity applications Besides the correlation explored in Q-mode, we employ an extra L-mode to further improve the coding performance In L-mode, the temporal-spatial correlation is exploited simultaneously by visual patch-based learning and mapping at pixel level This helps us achieve more accurate prediction signals based on the coarse base layer reconstruction within an ML structure Experimental results show the effectiveness of our SSVC scheme compared with the simulcast case and other HEVC-based SSVC schemes

Spatially Scalable Video Coding For HEVC

The key task in image set compression is how to efficiently remove set redundancy among images and within a single image. In this paper, we propose the first multi-model prediction (MoP) method for image set compression to significantly reduce inter image redundancy. Unlike the previous prediction methods, our MoP enhances the correlation between images using feature-based geometric multi-model fitting. Based on estimated geometric models, multiple deformed prediction images are generated to reduce geometric distortions in different image regions. The block-based adaptive motion compensation is then adopted to further eliminate local variances. Experimental results demonstrate the advantage of our approach, especially for images with complicated scenes and geometric relationships.

Zhongbo Shi

Papers

Photo Album Compression for Cloud Storage Using Local Features

Spatially Scalable Video Coding for HEVC

Feature-based image set compression

Spatially Scalable Video Coding For HEVC

Multi-model prediction for image set compression