scispace - formally typeset
Search or ask a question
Author

Christoph Fehn

Bio: Christoph Fehn is an academic researcher from Fraunhofer Society. The author has contributed to research in topics: Stereo display & Stereoscopy. The author has an hindex of 11, co-authored 14 publications receiving 1232 citations.

Papers
More filters
Journal ArticleDOI
TL;DR: This paper discusses an advanced approach for a 3DTV service, which is based on the concept of video-plus-depth data representations, and provides a modular and flexible system architecture supporting a wide range of multi-view structures.
Abstract: Due to enormous progress in the areas of auto-stereoscopic 3D displays, digital video broadcast and computer vision algorithms, 3D television (3DTV) has reached a high technical maturity and many people now believe in its readiness for marketing. Experimental prototypes of entire 3DTV processing chains have been demonstrated successfully during the last few years, and the motion picture experts group (MPEG) of ISO/IEC has launched related ad hoc groups and standardization efforts envisaging the emerging market segment of 3DTV. In this context the paper discusses an advanced approach for a 3DTV service, which is based on the concept of video-plus-depth data representations. It particularly considers aspects of interoperability and multi-view adaptation for the case that different multi-baseline geometries are used for multi-view capturing and 3D display. Furthermore it presents algorithmic solutions for the creation of depth maps and depth image-based rendering related to this framework of multi-view adaptation. In contrast to other proposals, which are more focused on specialized configurations, the underlying approach provides a modular and flexible system architecture supporting a wide range of multi-view structures.

434 citations

Proceedings ArticleDOI
09 Jul 2006
TL;DR: The conclusion is that the necessary technology including standard media formats for 3D and free viewpoint is available or will be available in the near future, and that there is a clear demand from industry and user side for such applications.
Abstract: An overview of 3D and free viewpoint video is given in this paper with special focus on related standardization activities in MPEG. Free viewpoint video allows the user to freely navigate within real world visual scenes, as known from virtual worlds in computer graphics. Examples are shown, highlighting standards conform realization using MPEG-4. Then the principles of 3D video are introduced providing the user with a 3D depth impression of the observed scene. Example systems are described again focusing on their realization based on MPEG-4. Finally multi-view video coding is described as a key component for 3D and free viewpoint video systems. The conclusion is that the necessary technology including standard media formats for 3D and free viewpoint is available or will be available in the near future, and that there is a clear demand from industry and user side for such applications. 3D TV at home and free viewpoint video on DVD will be available soon, and will create huge new markets

332 citations

01 Jan 2002
TL;DR: In this article, the authors present a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework, where all parts of the 3D processing chain are optimized to one another.
Abstract: In this paper we will present the concept of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity where industries, research centers and universities have joined forces to design a backwardscompatible, flexible and modular broadcast 3D-TV system, where all parts of the 3D processing chain are optimised to one another. This includes content creation, coding and transmission, display and research in human 3D perception, which be will used to guide the development process. The goals of the project comprise the development of a novel broadcast 3D camera, algorithms to convert existing 2D-video material into 3D, a 2Dcompatible coding and transmission scheme for 3D-video using MPEG2/4/7 technologies and the design of two new autostereoscopic displays.

155 citations

Proceedings ArticleDOI
07 May 2007
TL;DR: An efficient approach is presented to combine the benefits of both disparity-compensated prediction (DCP) and asymmetric coding for the compression of stereoscopic video sequences to achieve very high compression gains without reducing the overall visual quality of the resulting three-dimensional (3D) percept.
Abstract: In this paper, we present an efficient approach for the compression of stereoscopic video sequences. By extending an H.264/AVC-based multi-view video coding (MVC) codec as described in a previous paper to mixed-resolution imagery, i.e., to stereoscopic videos where the right-eye view had been spatially down-sampled in order to reduce the total amount of data to be encoded, we are able to combine the benefits of both disparity-compensated prediction (DCP) and asymmetric coding. By doing so, we expect to achieve very high compression gains without reducing the overall visual quality of the resulting three-dimensional (3D) percept.

87 citations

Proceedings ArticleDOI
25 Oct 2004
TL;DR: Some essential parts of the envisaged 3D signal processing chain such as the real-time generation of "virtual" stereoscopic views from monoscopic color video and associated per-pixel depth information are covered.
Abstract: This paper describes recent advances in a number of R&D areas that are believed to provide 'key technologies' for the further development of a novel, digital, broadcast 3D-TV system The provided results are part of the outcome of the European IST project ATTEST (Advanced Three-Dimensional Television System Technologies), a two-year research initiative that was finalized in March 2004 The paper covers some essential parts of the envisaged 3D signal processing chain such as the real-time generation of "virtual" stereoscopic views from monoscopic color video and associated per-pixel depth information as well as the efficient compression and the backwards-compatible transmission of this advanced data representation format using state-of-the-art video coding standards such as MPEG-2 (color data) and MPEG-4 Visual, resp Advanced Video Coding (depth data) Furthermore, the paper also describes the development of a new, single-user autostereoscopic 3D-TV display (Free2C) This novel, high-quality 3D device utilizes a lenticular lens raster to separate two individual perspective views, which are presented simultaneously on an underlying LC panel To provide the user with a satisfying 3D reproduction within a sufficiently large viewing area - a major problem for many state-of-the-art autostereoscopic 3D displays - the lenticular is constantly readjusted according to the viewer's actual head position, which is measured by a highly accurate, video-based tracking system This approach allows for a variation of the viewing distance within a range of 400 mm to 1100 mm as well as horizontal head movements within a range of about ±30° The feasibility of the new 3D-TV concept is proved through extensive human factors evaluations of the before-described algorithms and components

83 citations


Cited by
More filters
Proceedings ArticleDOI
TL;DR: Details of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework are presented and a comparison with the classical approach of "stereoscopic" video is compared.
Abstract: This paper presents details of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity, where industries, research centers and universities have joined forces to design a backwards-compatible, flexible and modular broadcast 3D-TV system. At the very heart of the described new concept is the generation and distribution of a novel data representation format, which consists of monoscopic color video and associated perpixel depth information. From these data, one or more “virtual” views of a real-world scene can be synthesized in real-time at the receiver side (i. e. a 3D-TV set-top box) by means of so-called depth-image-based rendering (DIBR) techniques. This publication will provide: (1) a detailed description of the fundamentals of this new approach on 3D-TV; (2) a comparison with the classical approach of “stereoscopic” video; (3) a short introduction to DIBR techniques in general; (4) the development of a specific DIBR algorithm that can be used for the efficient generation of high-quality “virtual” stereoscopic views; (5) a number of implementation details that are specific to the current state of the development; (6) research on the backwards-compatible compression and transmission of 3D imagery using state-of-the-art MPEG (Moving Pictures Expert Group) tools.

1,560 citations

Journal ArticleDOI
TL;DR: A systematic, comprehensive and up-to-date review of perceptual visual quality metrics (PVQMs) to predict picture quality according to human perception.

895 citations

Journal ArticleDOI
31 Jan 2011
TL;DR: An overview of the algorithmic design used for extending H.264/MPEG-4 AVC towards MVC is provided and a summary of the coding performance achieved by MVC for both stereo- and multiview video is provided.
Abstract: Significant improvements in video compression capability have been demonstrated with the introduction of the H.264/MPEG-4 advanced video coding (AVC) standard. Since developing this standard, the Joint Video Team of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) has also standardized an extension of that technology that is referred to as multiview video coding (MVC). MVC provides a compact representation for multiple views of a video scene, such as multiple synchronized video cameras. Stereo-paired video for 3-D viewing is an important special case of MVC. The standard enables inter-view prediction to improve compression capability, as well as supporting ordinary temporal and spatial prediction. It also supports backward compatibility with existing legacy systems by structuring the MVC bitstream to include a compatible “base view.” Each other view is encoded at the same picture resolution as the base view. In recognition of its high-quality encoding capability and support for backward compatibility, the stereo high profile of the MVC extension was selected by the Blu-Ray Disc Association as the coding format for 3-D video with high-definition resolution. This paper provides an overview of the algorithmic design used for extending H.264/MPEG-4 AVC towards MVC. The basic approach of MVC for enabling inter-view prediction and view scalability in the context of H.264/MPEG-4 AVC is reviewed. Related supplemental enhancement information (SEI) metadata is also described. Various “frame compatible” approaches for support of stereo-view video as an alternative to MVC are also discussed. A summary of the coding performance achieved by MVC for both stereo- and multiview video is also provided. Future directions and challenges related to 3-D video are also briefly discussed.

683 citations

Journal ArticleDOI
TL;DR: An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented, showing that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient.
Abstract: An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented. The compression method is based on the multiple reference picture technique in the H.264/AVC video coding standard. The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion-compensated prediction. The effectiveness of this approach is demonstrated by an experimental analysis of temporal versus inter-view prediction in terms of the Lagrange cost function. The results show that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient. Hierarchical B pictures are used as basic structure for temporal prediction. Their advantages are combined with inter-view prediction for different temporal hierarchy levels, starting from simulcast coding with no inter-view prediction up to full level inter-view prediction. When using inter-view prediction at key picture temporal levels, average gains of 1.4-dB peak signal-to-noise ratio (PSNR) are reported, while additionally using inter-view prediction at nonkey picture temporal levels, average gains of 1.6-dB PSNR are reported. For some cases, gains of more than 3 dB, corresponding to bit-rate savings of up to 50%, are obtained.

645 citations

Journal ArticleDOI
01 Aug 2004
TL;DR: This is the first real-time end-to-end 3D TV system with enough views and resolution to provide a truly immersive 3D experience and presents the calibration and image alignment procedures that are necessary to achieve good image quality.
Abstract: Three-dimensional TV is expected to be the next revolution in the history of television. We implemented a 3D TV prototype system with real-time acquisition, transmission, and 3D display of dynamic scenes. We developed a distributed, scalable architecture to manage the high computation and bandwidth demands. Our system consists of an array of cameras, clusters of network-connected PCs, and a multi-projector 3D display. Multiple video streams are individually encoded and sent over a broadband network to the display. The 3D display shows high-resolution (1024 × 768) stereoscopic color images for multiple viewpoints without special glasses. We implemented systems with rear-projection and front-projection lenticular screens. In this paper, we provide a detailed overview of our 3D TV system, including an examination of design choices and tradeoffs. We present the calibration and image alignment procedures that are necessary to achieve good image quality. We present qualitative results and some early user feedback. We believe this is the first real-time end-to-end 3D TV system with enough views and resolution to provide a truly immersive 3D experience.

486 citations