Wearable near-eye displays for virtual and augmented reality (VR/AR) have seen enormous growth in recent years. While researchers are exploiting a plethora of techniques to create life-like three-dimensional (3D) objects, there is a lack of awareness of the role of human perception in guiding the hardware development. An ultimate VR/AR headset must integrate the display, sensors, and processors in a compact enclosure that people can comfortably wear for a long time while allowing a superior immersion experience and user-friendly human–computer interaction. Compared with other 3D displays, the holographic display has unique advantages in providing natural depth cues and correcting eye aberrations. Therefore, it holds great promise to be the enabling technology for next-generation VR/AR devices. In this review, we survey the recent progress in holographic near-eye displays from the human-centric perspective.

Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective.

After more than 20 years of research, ADAS are common in modern vehicles available in the market. Automated Driving systems, still in research phase and limited in their capabilities, are starting early commercial tests in public roads. These systems rely on the information provided by on-board sensors, which allow to describe the state of the vehicle, its environment and other actors. Selection and arrangement of sensors represent a key factor in the design of the system. This survey reviews existing, novel and upcoming sensor technologies, applied to common perception tasks for ADAS and Automated Driving. They are put in context making a historical review of the most relevant demonstrations on Automated Driving, focused on their sensing setup. Finally, the article presents a snapshot of the future challenges for sensing technologies and perception, finishing with an overview of the commercial initiatives and manufacturers alliances that will show the intention of the market in sensors technologies for Automated Vehicles.

/pdf/a-review-of-sensor-technologies-for-perception-in-automated-1ipvoag54i.pdf

A Review of Sensor Technologies for Perception in Automated Driving

Holography is considered to be the ultimate display technology since it can account for all human visual cues such as stereopsis and eye focusing. Aside from hardware constraints for building holographic displays, there are still many research challenges regarding holographic signal processing that need to be tackled. In this overview, we delineate the steps needed to realize an end-to-end chain from digital content acquisition to display, involving the efficient generation, representation, coding and quality assessment of digital holograms. We discuss the current state-of-the-art and what hurdles remain to be taken to pave the way towards realistic visualization of dynamic holographic content.

https://biblio.vub.ac.be/vubirfiles/40443189/Blinder_SPIC_2019.pdf

Signal processing challenges for digital holographic video display systems

We demonstrate a tabletop holographic display system for simultaneously serving continuous parallax 3.2-inch 360-degree three-dimensional holographic image content to multiple observers at a 45-degree oblique viewing circumference. To achieve this, localized viewing windows are to be seamlessly generated on the 360-degree viewing circumference. In the proposed system, four synchronized high-speed digital micro-mirror displays are optically configured to comprise a single 2 by 2 multi-vision panel that enables size enlargement and time-division-multiplexing of holographic image content. Also, a specially designed optical image delivery sub-system that is composed of parabolic mirrors and an aspheric lens is designed as an essential part for achieving an enlarged 3.2-inch holographic image and a large 45-degree oblique viewing angle without visual distortion.

360-degree tabletop electronic holographic display.

This work presents color holographic display, which is based on a single phase only spatial light modulator (SLM). In the display entire area of the SLM is illuminated by an on-axis white light beam generated by a single large LED. The holographic display fully utilizes SLM bandwidth and has capability of full-color, full frame rate imaging of outstanding quality. This is achieved through: (i) optimal use of the source coherence volume, (ii) application of the single white light LED source, (iii) a development of a novel concept of color multiplexing technique with color filter mask in Fourier plane of the SLM, (iv) and a complex coding with improved diffraction efficiency. Within experimental part of the paper we show single color, full-color holographic 2D and 3D images generated for reconstruction depth exceeding 10 cm.

Color holographic display with white light LED source and single phase only SLM

This paper presents a wide angle holographic display system with extended viewing angle in both horizontal and vertical directions. The display is constructed from six spatial light modulators (SLM) arranged on a circle and an additional SLM used for spatiotemporal multiplexing and a viewing angle extension in two perpendicular directions. The additional SLM, that is synchronized with the SLMs on the circle is placed in the image space. This method increases effective space bandwidth product of display system data from 12.4 to 50 megapixels. The software solution based on three Nvidia graphic cards is developed and implemented in order to achieve fast and synchronized displaying. The experiments presented for both synthetic and real 3D data prove the possibility to view binocularly having good quality images reconstructed in full FoV of the display.

Wide angle holographic display system with spatiotemporal multiplexing.

This paper presents the full technology chain supporting wide angle digital holographic television from holographic capture of real world objects/scenes to holographic display with an extended viewing angle. The data are captured with multiple CCD cameras located around an object. The display system is based on multiple tilted spatial light modulators (SLMs) arranged in a circular configuration. The capture-display system is linked by a holographic data processing module, which allows for significant decoupling of the capture and display systems. The presented experimental results, based on the reconstruction of real world, variable in time scenes, illustrates imaging dynamics, viewing angle and quality.

Multiwavefront digital holographic television.

We present HoloFace, an open-source framework for face alignment, head pose estimation and facial attribute retrieval for Microsoft HoloLens. HoloFace implements two state-of-the-art face alignment methods which can be used interchangeably: one running locally and one running on a remote backend. Head pose estimation is accomplished by fitting a deformable 3D model to the landmarks localized using face alignment. The head pose provides both the rotation of the head and a position in the world space. The parameters of the fitted 3D face model provide estimates of facial attributes such as mouth opening or smile. Together the above information can be used to augment the faces of people seen by the HoloLens user, and thus their interaction. Potential usage scenarios include facial recognition, emotion recognition, eye gaze tracking and many others. We demonstrate the capabilities of our framework by augmenting the faces of people seen through the HoloLens with various objects and animations.

HoloFace: Augmenting Human-to-Human Interactions on HoloLens

Structured light camera which is being designed with the joined effort of Institute of Radioelectronics and Institute of Optoelectronics (both being large units of the Warsaw University of Technology within the Faculty of Electronics and Information Technology) combines various hardware and software contemporary technologies. In hardware it is integration of a high speed stripe projector and a stripe camera together with a standard high definition video camera. In software it is supported by sophisticated calibration techniques which enable development of advanced application such as real time 3D viewer of moving objects with the free viewpoint or 3D modeller for still objects.

Structured light camera calibration

Digital image correlation (DIC) is a well-established technique for the measurement and monitoring of displacements and strains in engineering objects. In technical applications, an object's surface is modified by applying random, artificial structure, and monochrome cameras are typically used. The challenge in the measurement of Cultural Heritage objects (CHO) lies in the necessity of using the natural surface texture (no modifications are allowed), which in most of the cases is far from the ideal random pattern required for the successful implementation of DIC. On the other hand, spectrally separated content of natural CHO textures may provide additional spatial information and support DIC analysis. In this work, we propose a colour image pre-processing path that provides the input data best fitted to high accuracy 2D DIC and 3D DIC analysis. For this task we employ cameras with a colour filter array (CFA) with a Bayer pattern. This pattern has twice the number of green photosites as red or blue arranged in a 2x2pixel repeating-unit. From this sparse colour sampling, a full colour image is produced by interpolating the unknown values in each color channel. The interpolation is achieved via any one of the many demosaicing algorithms available to calculate a red, green, and blue value for every pixel position that only data from one channel was captured. These three R, G, B channels can then be combined to create a monochrome image suitable for conventional DIC analysis. This work's main objective is to select the best demosaicing algorithms for the spectral and monochromatic channels to minimize displacement and strain reconstruction errors (STD and P/V). The systematic analysis will be performed for objects (mock-ups) with artificial texture and fully controlled displacements. The expected errors due to interpolation procedures will be identified. Next, the best demosaicing procedure will be implemented into the full processing path of colour DIC used for selected CHO with natural texture (parchments, oil paintings). The analysis results based on monochrome, spectral, and mixed spectral data channels will be compared from the point of view of their applicability to different types of natural texture and achieved measurement accuracy.

Piotr Garbat

Papers

Wide angle holographic display system with spatiotemporal multiplexing.

Multiwavefront digital holographic television.

HoloFace: Augmenting Human-to-Human Interactions on HoloLens

Structured light camera calibration

Colour digital image correlation method for monitoring of cultural heritage objects with natural texture