Oliver Schreer

Bio: Oliver Schreer is an academic researcher from Heinrich Hertz Institute. The author has contributed to research in topics: Rendering (computer graphics) & Virtual reality. The author has an hindex of 20, co-authored 108 publications receiving 2157 citations. Previous affiliations of Oliver Schreer include Fraunhofer Society.

Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability

Abstract: Due to enormous progress in the areas of auto-stereoscopic 3D displays, digital video broadcast and computer vision algorithms, 3D television (3DTV) has reached a high technical maturity and many people now believe in its readiness for marketing. Experimental prototypes of entire 3DTV processing chains have been demonstrated successfully during the last few years, and the motion picture experts group (MPEG) of ISO/IEC has launched related ad hoc groups and standardization efforts envisaging the emerging market segment of 3DTV. In this context the paper discusses an advanced approach for a 3DTV service, which is based on the concept of video-plus-depth data representations. It particularly considers aspects of interoperability and multi-view adaptation for the case that different multi-baseline geometries are used for multi-view capturing and 3D display. Furthermore it presents algorithmic solutions for the creation of depth maps and depth image-based rendering related to this framework of multi-view adaptation. In contrast to other proposals, which are more focused on specialized configurations, the underlying approach provides a modular and flexible system architecture supporting a wide range of multi-view structures.

An immersive 3D video-conferencing system using shared virtual team user environments

Abstract: Videoconferencing is going to become attractive for geo-graphically distributed team collaboration, specifically to avoid travelling and to increase flexibility. Against this background this paper presents a next generation system - a 3D videoconference providing immersive tele-presence and natural representation of all participants in a shared virtual meeting space to enhance quality of human-centred communication. This system is based on the principle of a shared virtual table environment, which guarantees correct eye contact and gesture reproduction. The key features of our system are presented and compared to other approaches like tele-cubicles. Furthermore the current system design and details of the real-time hardware and software concept are explained.

3D Videocommunication: Algorithms, concepts and real-time systems in human centred communication

Abstract: List of Contributors. Symbols. Abbreviations. Introduction by O. Schreer, P. Kauff, T. Sikora. SECTION I: APPLICATIONS OF 3D VIDEOCOMMUNICATION. 1. History of Telepresence (W.A. IJsselsteijn). 2. 3D TV broadcasting (C. Fehn). 3. 3D in Content Creation and Post-Production (O. Grau). 4. Free Viewpoint Systems (M. Tanimoto). 5. Immersive Videoconferencing (P. Kauff, O. Schreer). SECTION II: 3D DATA REPRESENTATION AND PROCESSING. 6. Fundamentals of Multiple View Geometry (E. Trucco, S. Ivekovic, A. Fusiello). 7. Stereo Analysis (N. Atzpadin, J. Mulligan). 8. Reconstruction of Volumetric 3D Models (P. Eisert). 9. View Synthesis and Rendering Methods (R. Koch, J.-F. Evers-Senne). 10. 3D Audio Capture and Analysis (M. Schwab, P. Noll). 11. Coding and Standardization (A. Smolic, Th. Sikora). SECTION III: 3D REPRODUCTION. 12. Human Factors of 3D Displays (W.A. IJsselsteijn, P.J.H. Seuntiens, L.M.J. Meesters). 13. 3D Displays (S. Pastoor). 14. Mixed Reality Displays (S. Pastoor, C. Conomis). 15. Spatialised Audio and 3D Audio Rendering (Th. Sporer, S. Brix). SECTION IV: 3D DATA SENSORS. 16. Sensor-Based Depth Capturing (Joao G.M. Goncalves, V. Sequeira). 17. Tracking and User Interface for Mixed Reality (T. Ebrahimi, Y. Abdeljaoued, D. Marimon). Index.

Stereo analysis by hybrid recursive matching for real-time immersive video conferencing

Abstract: Real-time stereo analysis is an important research area in computer vision. In this context, we propose a stereo algorithm for an immersive video-conferencing system by which conferees at different geographical places can meet under similar conditions as in the real world. For this purpose, virtual views of the remote conferees are generated and adapted to the current viewpoint of the local participant. Dense vector fields of high accuracy are required in order to guarantee an adequate quality of the virtual views. Due to the usage of a wide baseline system with strongly convergent camera configurations, the dynamic disparity range is about 150 pixels. Considering computational costs, a full search or even a local search restricted to a small window of a few pixels, as it is implemented in many real-time algorithms, is not suitable for our application because processing on full-resolution video according to CCIR 601 TV standard with 25 frames per second is addressed-the most desirable as a pure software solution running on available processors without any support from dedicated hardware. Therefore, we propose in this paper a new fast algorithm for stereo analysis, which circumvents the window search by using a hybrid recursive matching strategy based on the effective selection of a small number of candidates. However, stereo analysis requires more than a straightforward application of stereo matching. The crucial problem is to produce accurate stereo correspondences in all parts of the image. Especially, errors in occluded regions and homogenous or less structured regions lead to disturbing artifacts in the synthesized virtual views. To cope with this problem, mismatches have to be detected and substituted by a sophisticated interpolation and extrapolation scheme.

Three-dimensional image processing in the future of immersive media

Abstract: This survey paper discusses the three-dimensional image processing challenges posed by present and future immersive telecommunications, especially immersive video conferencing and television. We introduce the concepts of presence, immersion, and co-presence and discuss their relation to virtual collaborative environments in the context of communications. Several examples are used to illustrate the current state of the art. We highlight the crucial need of real-time, highly realistic video with adaptive viewpoint for future immersive communications and identify calibration, multiple-view analysis, tracking, and view synthesis as the fundamental image-processing modules addressing such a need. For each topic, we sketch the basic problem and representative solutions from the image processing literature.

Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV

Abstract: This paper presents details of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity, where industries, research centers and universities have joined forces to design a backwards-compatible, flexible and modular broadcast 3D-TV system. At the very heart of the described new concept is the generation and distribution of a novel data representation format, which consists of monoscopic color video and associated perpixel depth information. From these data, one or more “virtual” views of a real-world scene can be synthesized in real-time at the receiver side (i. e. a 3D-TV set-top box) by means of so-called depth-image-based rendering (DIBR) techniques. This publication will provide: (1) a detailed description of the fundamentals of this new approach on 3D-TV; (2) a comparison with the classical approach of “stereoscopic” video; (3) a short introduction to DIBR techniques in general; (4) the development of a specific DIBR algorithm that can be used for the efficient generation of high-quality “virtual” stereoscopic views; (5) a number of implementation details that are specific to the current state of the development; (6) research on the backwards-compatible compression and transmission of 3D imagery using state-of-the-art MPEG (Moving Pictures Expert Group) tools.

Hand and Mind: What Gestures Reveal about Thought

Abstract: Hand and Mind: What Gestures Reveal about Thought. David McNeill. Chicago and London: University of Chicago Press, 1992. 416 pp.

Fundamentals of Liquid Crystal Devices

Abstract: Foreword. Series Editor's Foreword. Preface. 1. Liquid crystal physics.* Introduction.* Thermodynamics and statistic physics.* Orientational order.* Elastic properties of liquid crystals.* Response of liquid crystals to electro-magnetic fields.* Anchoring effects of nematic liquid crystal at surfaces. 2. Propagation of light in anisotropic optical medium.* Electromagnetic wave.* Polarization.* Propagation of light in uniform anisotropic optical media.* Propagation of light in cholesteric liquid crystals. 3. Optical modeling methods.* Jones matrix method.* Mueller matrix method.* Berreman 4x4 method. 4. Effects of Electric field on Liquid Crystals.* Dielectric interaction.* Flexoelectric Effect.* Ferroelectricity in liquid crystals. 5. Freedericksz transition.* Calculus of variation.* The Fredeericksz transition: statics.* The Freedericksz transition: dynamics. 6. Liquid Crystal Materials.* Introduction.* Refractive indices.* Dielectric constants.* Rotational Viscosity.* Elastic constant.* Figure-of-merits.* Refractive index matching between liquid crystals and polymers. 7. Modeling of liquid crystal director configuration.* Electric energy of liquid crystals.* Modeling electric field.* Simulation of liquid crystal director configuration. 8. Transmissive liquid crystal display.* Introduction.* Twisted nematic cells.* In plane switching (IPS) mode.* Vertical alignment (VA) mode.* Multi-domain Vertical Alignment (MVA) Cells.* Optically compensated bend (OCB) cell. 9. Reflective and Trasreflective display.* Introduction.* Reflective liquid crystal displays.* Transflector.* Classification of Transflective LCDs.* Dual-cell-gap Transflective LCDs.* Single-cell-gap Transflective LCDs.* Performance of transflective LCDs. 10. Liquid crystal display matrices, drive schemes and bistable displays.* Segmented displays.* Passive matrix displays and drive scheme.* Active Matrix Displays.* Bistable ferroelectric liquid crystal displays and drive scheme.* Bistable nematic displays.* Bistable cholesteric reflective display. 11. Liquid crystal/polymer composites. * Introduction.* Phase separation.* Scattering properties of liquid crystal/polymer composites.* Polymer dispersed liquid crystals.* Polymer stabilization liquid crystals.* Displays from liquid crystal/polymer composites. 12. Tunable liquid crystal photonic devices. * Introduction.* Laser beam steering.* Variable Optical Attenuators.* Tunable-Focus Lens.* Polarization-Independent LC Devices. Index.

Efficient Prediction Structures for Multiview Video Coding

Abstract: An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented. The compression method is based on the multiple reference picture technique in the H.264/AVC video coding standard. The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion-compensated prediction. The effectiveness of this approach is demonstrated by an experimental analysis of temporal versus inter-view prediction in terms of the Lagrange cost function. The results show that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient. Hierarchical B pictures are used as basic structure for temporal prediction. Their advantages are combined with inter-view prediction for different temporal hierarchy levels, starting from simulcast coding with no inter-view prediction up to full level inter-view prediction. When using inter-view prediction at key picture temporal levels, average gains of 1.4-dB peak signal-to-noise ratio (PSNR) are reported, while additionally using inter-view prediction at nonkey picture temporal levels, average gains of 1.6-dB PSNR are reported. For some cases, gains of more than 3 dB, corresponding to bit-rate savings of up to 50%, are obtained.

Holoportation: Virtual 3D Teleportation in Real-time

Abstract: We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.