There is disclosed a method, an apparatus, a server, a client and a non-transitory computer readable medium comprising a computer program stored therein for motion compensated video coding and decoding. Texture block motion information is used to derive disparity/depth motion information. Alternatively, disparity/depth motion information is used to derive texture block motion information.

An apparatus, a method and a computer program for video coding and decoding

Three-dimensional television (3DTV) is believed to be the future of television broadcasting that would replace current 2D HDTV technology. Future 3DTV would bring a more life-like and visually immersive home entertainment experience, in which users will have the freedom to navigate through the scene to choose a different viewpoint. A desired view can be synthesized at the receiver side using depth image-based rendering (DIBR). While this approach has many advantages, one of the key challenges in DIBR is how to fill the holes caused by disocclusion regions and wrong depth values. In this paper, we propose two new approaches for disocclusion removal in DIBR. Both approaches namely hierarchical hole-filling (HHF) and depth adaptive hierarchical hole-filling eliminate the need for any smoothing or filtering of the depth map. Both techniques use a pyramid-like approach to estimate the hole pixels from lower resolution estimates of the 3D wrapped image. The lower resolution estimates involve a pseudo zero canceling plus Gaussian filtering of the wrapped image. The depth adaptive HHF incorporates the depth information to produce a higher resolution rendering around previously occluded areas. Experimental results show that HHF and depth adaptive HHF yield virtual images and stereoscopic videos that are free of geometric distortions and a better rendering quality both subjectively and objectively than traditional hole-filling approaches.

Hierarchical Hole-Filling For Depth-Based View Synthesis in FTV and 3D Video

Systems and methods for the synthesis of light field images from virtual viewpoints in accordance with embodiments of the invention are disclosed. In one embodiment of the invention, a system includes a processor and a memory configured to store captured light field image data and an image manipulation application, wherein the captured light field image data includes image data, pixel position data, and a depth map, and wherein the image manipulation application configures the processor to obtain captured light field image data, determine a virtual viewpoint for the captured light field image data, where the virtual viewpoint includes a virtual location and virtual depth information, compute a virtual depth map based on the captured light field image data and the virtual viewpoint, and generate an image from the perspective of the virtual viewpoint based on the captured light field image data and the virtual depth map.

Generating Images from Light Fields Utilizing Virtual Viewpoints

Systems and methods for extended color processing on Pelican array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating a high resolution image includes obtaining input images, where a first set of images includes information in a first band of visible wavelengths and a second set of images includes information in a second band of visible wavelengths and non-visible wavelengths, determining an initial estimate by combining the first set of images into a first fused image, combining the second set of images into a second fused image, spatially registering the fused images, denoising the fused images using bilateral filters, normalizing the second fused image in the photometric reference space of the first fused image, combining the fused images, determining a high resolution image that when mapped through a forward imaging transformation matches the input images within at least one predetermined criterion.

Extended Color Processing on Pelican Array Cameras

Systems and methods in accordance with embodiments of the invention enable feature based high resolution motion estimation from low resolution images captured using an array camera. One embodiment includes performing feature detection with respect to a sequence of low resolution images to identify initial locations for a plurality of detected features in the sequence of low resolution images, where the at least one sequence of low resolution images is part of a set of sequences of low resolution images captured from different perspectives. The method also includes synthesizing high resolution image portions, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images. The method further including performing feature detection within the high resolution image portions to identify high precision locations for the detected features, and estimating camera motion using the high precision locations for said plurality of detected features.

Feature Based High Resolution Motion Estimation from Low Resolution Images Captured Using an Array Source

An intermediate view synthesis apparatus for synthesizing an intermediate view image from a first image corresponding to a first view and a second image corresponding to a second view different from the first view, the first and second images including depth information, wherein the second image being is divided-up into a non-boundary portion, and a foreground/background boundary region, wherein the intermediate view synthesis apparatus is configured to project and merge the first image and the second image into the intermediate view to obtain an intermediate view image, with treating the foreground/background boundary region subordinated relative to the non-boundary portion. A multi-view data signal extraction apparatus for extracting a multiview data signal from a multi-view representation including a first image corresponding to a first view and a second image corresponding to a second view being different from the first view is also described, the first and second images including depth information.

Karsten Mueller

Papers

Intermediate view synthesis and multi-view data signal extraction