Masayuki Tanimoto

Bio: Masayuki Tanimoto is an academic researcher from Nagoya University. The author has contributed to research in topics: Free viewpoint television & Interpolation. The author has an hindex of 22, co-authored 72 publications receiving 2806 citations. Previous affiliations of Masayuki Tanimoto include Hong Kong University of Science and Technology.

Multiview Imaging and 3DTV

Abstract: The eight articles in this special section are devoted to multi-view imaging and three dimensional television displays.

View generation with 3D warping using depth information for FTV

Abstract: In this paper, we propose a new method of depth-image-based rendering (DIBR) for free-viewpoint TV (FTV). In the conventional method, we estimated the depth of an object on the virtual image plane, which is called view-dependent depth estimation, and the virtual view images are rendered using the view-dependent depth map. In this method, virtual viewpoint images are rendered with 3D warping instead of estimating the view-dependent depth, since depth estimation is usually costly and it is desirable to eliminate it from the rendering process. However, 3D warping causes some problems that do not occur in the method with view-dependent depth estimation; for example, the appearance of holes on the rendered image, and the occurrence of depth discontinuity on the surface of the object at virtual image plane. Depth discontinuity causes artifacts on the rendered image. In this paper, these problems are solved by projecting depth map to the virtual image plane and performing post-filtering on the projected depth map. In the experiments, high-quality arbitrary viewpoint images were obtained by rendering images from relatively small number of cameras.

Free-Viewpoint TV

Abstract: Free-viewpoint television (FTV) is an innovative visual media that enables us to view a three-dimensional (3-D) scene by freely changing our viewpoints. We proposed the concept of FTV and constructed the world???s first real-time system including the complete chain of operation from image capture to display. We also carried out the FTV on a single personal computer (PC) and a mobile player. FTV is based on the ray-space method that represents one ray in real space with one point in the ray-space. We have developed several types of ray capture systems and interfaces such as a 360° capture/ray-reproducing display. FTV is regarded as the ultimate 3DTV, since it can generate infinite number of views. Thus, FTV is the key to immersive communication. Regarding FTV as the most challenging 3-D media, the Motion Picture Experts Group (MPEG) has been conducting its international standardization activities. This article reviews FTV and its related technologies.

Overview of free viewpoint television

Abstract: We have been developing ray-based 3D information systems that consist of ray acquisition, ray processing, and ray display. Free viewpoint television (FTV) based on the ray-space method is a typical example. FTV will bring an epoch-making change in the history of television because it enables us to view a distant 3D world freely by changing our viewpoints as if we were there. We constructed a real-time FTV including the complete chain from capturing to display. A new algorithm was developed to generate free viewpoint images. In addition, a new user interface is presented for FTV to make full use of 3D information. FTV is not a pixel-based system but a ray-based system. We are creating ray-based image engineering through the development of FTV.

View Generation with 3D Warping Using Depth Information for FTV

Abstract: Free viewpoint images can be generated from multi-view images using Ray-Space method. Ray-Space data requires ray interpolation so as to satisfy the plenoptic function. Ray interpolation is realized by estimating view-dependent depth. Depth estimation is usually costly process, thus it is desirable that this process is skipped from rendering process to achieve real-time rendering. This paper proposes a method to render a novel view image using multi-view images and depth maps which are computed in advance. Virtual viewpoint image is generated by 3D warping, which causes some problems that have not occurred in the method with view dependent depth estimation. We handled these problems by projecting depth map to virtual image plane first and perform post-filtering on the projected depth map. We succeeded in obtaining high quality arbitrary viewpoint images from relatively small number of cameras.

Connected Vehicles: Solutions and Challenges

Abstract: Providing various wireless connectivities for vehicles enables the communication between vehicles and their internal and external environments. Such a connected vehicle solution is expected to be the next frontier for automotive revolution and the key to the evolution to next generation intelligent transportation systems (ITSs). Moreover, connected vehicles are also the building blocks of emerging Internet of Vehicles (IoV). Extensive research activities and numerous industrial initiatives have paved the way for the coming era of connected vehicles. In this paper, we focus on wireless technologies and potential challenges to provide vehicle-to-x connectivity. In particular, we discuss the challenges and review the state-of-the-art wireless solutions for vehicle-to-sensor, vehicle-to-vehicle, vehicle-to-Internet, and vehicle-to-road infrastructure connectivities. We also identify future research issues for building connected vehicles.

LED Based Indoor Visible Light Communications: State of the Art

Abstract: Visible Light Communication (VLC) is an emerging field in Optical Wireless Communication (OWC) which utilizes the superior modulation bandwidth of Light Emitting Diodes (LEDs) to transmit data. In modern day communication systems, the most popular frequency band is Radio Frequency (RF) mainly due to little interference and good coverage. However, the rapidly dwindling RF spectrum along with increasing wireless network traffic has substantiated the need for greater bandwidth and spectral relief. By combining illumination and communication, VLC provides ubiquitous communication while addressing the shortfalls and limitations of RF communication. This paper provides a comprehensive survey on VLC with an emphasis on challenges faced in indoor applications over the period 1979–2014. VLC is compared with infrared (IR) and RF systems and the necessity for using this beneficial technology in communication systems is justified. The advantages of LEDs compared to traditional lighting technologies are discussed and comparison is done between different types of LEDs currently available. Modulation schemes and dimming techniques for indoor VLC are discussed in detail. Methods needed to improve VLC system performance such as filtering, equalization, compensation, and beamforming are also presented. The recent progress made by various research groups in this field is discussed along with the possible applications of this technology. Finally, the limitations of VLC as well as the probable future directions are presented.

Efficient Prediction Structures for Multiview Video Coding

Abstract: An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented. The compression method is based on the multiple reference picture technique in the H.264/AVC video coding standard. The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion-compensated prediction. The effectiveness of this approach is demonstrated by an experimental analysis of temporal versus inter-view prediction in terms of the Lagrange cost function. The results show that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient. Hierarchical B pictures are used as basic structure for temporal prediction. Their advantages are combined with inter-view prediction for different temporal hierarchy levels, starting from simulcast coding with no inter-view prediction up to full level inter-view prediction. When using inter-view prediction at key picture temporal levels, average gains of 1.4-dB peak signal-to-noise ratio (PSNR) are reported, while additionally using inter-view prediction at nonkey picture temporal levels, average gains of 1.6-dB PSNR are reported. For some cases, gains of more than 3 dB, corresponding to bit-rate savings of up to 50%, are obtained.

Multimedia Cloud Computing

Abstract: This article introduces the principal concepts of multimedia cloud computing and presents a novel framework. We address multimedia cloud computing from multimedia-aware cloud (media cloud) and cloud-aware multimedia (cloud media) perspectives. First, we present a multimedia-aware cloud, which addresses how a cloud can perform distributed multimedia processing and storage and provide quality of service (QoS) provisioning for multimedia services. To achieve a high QoS for multimedia services, we propose a media-edge cloud (MEC) architecture, in which storage, central processing unit (CPU), and graphics processing unit (GPU) clusters are presented at the edge to provide distributed parallel processing and QoS adaptation for various types of devices.

3-D Video Representation Using Depth Maps

Abstract: Current 3-D video (3DV) technology is based on stereo systems. These systems use stereo video coding for pictures delivered by two input cameras. Typically, such stereo systems only reproduce these two camera views at the receiver and stereoscopic displays for multiple viewers require wearing special 3-D glasses. On the other hand, emerging autostereoscopic multiview displays emit a large numbers of views to enable 3-D viewing for multiple users without requiring 3-D glasses. For representing a large number of views, a multiview extension of stereo video coding is used, typically requiring a bit rate that is proportional to the number of views. However, since the quality improvement of multiview displays will be governed by an increase of emitted views, a format is needed that allows the generation of arbitrary numbers of views with the transmission bit rate being constant. Such a format is the combination of video signals and associated depth maps. The depth maps provide disparities associated with every sample of the video signal that can be used to render arbitrary numbers of additional views via view synthesis. This paper describes efficient coding methods for video and depth data. For the generation of views, synthesis methods are presented, which mitigate errors from depth estimation and coding.