MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

We present novel designs for virtual and augmented reality near-eye displays based on phase-only holographic projection. Our approach is built on the principles of Fresnel holography and double phase amplitude encoding with additional hardware, phase correction factors, and spatial light modulator encodings to achieve full color, high contrast and low noise holograms with high resolution and true per-pixel focal control. We provide a GPU-accelerated implementation of all holographic computation that integrates with the standard graphics pipeline and enables real-time (≥90 Hz) calculation directly or through eye tracked approximations. A unified focus, aberration correction, and vision correction model, along with a user calibration process, accounts for any optical defects between the light source and retina. We use this optical correction ability not only to fix minor aberrations but to enable truly compact, eyeglasses-like displays with wide fields of view (80°) that would be inaccessible through conventional means. All functionality is evaluated across a series of hardware prototypes; we discuss remaining challenges to incorporate all features into a single device.

Holographic near-eye displays for virtual and augmented reality

Digital optical holograms can achieve nanometre-scale resolution as a result of recent advances in metasurface technologies. This has raised hopes for applications in data encryption, data storage, information processing and displays. However, the hologram bandwidth has remained too low for any practical use. To overcome this limitation, information can be stored in the orbital angular momentum of light, as this degree of freedom has an unbounded set of orthogonal helical modes that could function as information channels. Thus far, orbital angular momentum holography has been achieved using phase-only metasurfaces, which, however, are marred by channel crosstalk. As a result, multiplex information from only four channels has been demonstrated. Here, we demonstrate an orbital angular momentum holography technology that is capable of multiplexing up to 200 independent orbital angular momentum channels. This has been achieved by designing a complex-amplitude metasurface in momentum space capable of complete and independent amplitude and phase manipulation. Information was then extracted by Fourier transform using different orbital angular momentum modes of light, allowing lensless reconstruction and holographic videos to be displayed. Our metasurface can be three-dimensionally printed in a polymer matrix on SiO2 for large-area fabrication.

/pdf/complex-amplitude-metasurface-based-orbital-angular-momentum-1t1lmm8j59.pdf

Complex-amplitude metasurface-based orbital angular momentum holography in momentum space.

Recently, metasurfaces composed of artificially fabricated subwavelength structures have shown remarkable potential for the manipulation of light with unprecedented functionality. Here, we first demonstrate a metasurface application to realize a compact near-eye display system for augmented reality with a wide field of view. A key component is a see-through metalens with an anisotropic response, a high numerical aperture with a large aperture, and broadband characteristics. By virtue of these high-performance features, the metalens can overcome the existing bottleneck imposed by the narrow field of view and bulkiness of current systems, which hinders their usability and further development. Experimental demonstrations with a nanoimprinted large-area see-through metalens are reported, showing full-color imaging with a wide field of view and feasibility of mass production. This work on novel metasurface applications shows great potential for the development of optical display systems for future consumer electronics and computer vision applications.

/pdf/metasurface-eyepiece-for-augmented-reality-xpx53ytnpl.pdf

Metasurface eyepiece for augmented reality

Nanophotonics has garnered intensive attention due to its unique capabilities in molding the flow of light in the subwavelength regime. Metasurfaces (MSs) and photonic integrated circuits (PICs) enable the realization of mass-producible, cost-effective, and highly efficient flat optical components for imaging, sensing, and communications. In order to enable nanophotonics with multi-purpose functionalities, chalcogenide phase-change materials (PCMs) have been introduced as a promising platform for tunable and reconfigurable nanophotonic frameworks. Integration of non-volatile chalcogenide PCMs with unique properties such as drastic optical contrasts, fast switching speeds, and long-term stability grants substantial reconfiguration to the more conventional static nanophotonic platforms. In this review, we discuss state-of-the-art developments as well as emerging trends in tunable MSs and PICs using chalcogenide PCMs. We outline the unique material properties, structural transformation, electro-optic, and thermo-optic effects of well-established classes of chalcogenide PCMs. The emerging deep learning-based approaches for the optimization of reconfigurable MSs and the analysis of light-matter interactions are also discussed. The review is concluded by discussing existing challenges in the realization of adjustable nanophotonics and a perspective on the possible developments in this promising area.

/pdf/tunable-nanophotonics-enabled-by-chalcogenide-phase-change-3q12l5zhtw.pdf

Tunable nanophotonics enabled by chalcogenide phase-change materials

Reconstruction of light profiles with amplitude and phase information, called holography, is an attractive optical technology with various significant applications such as three-dimensional imaging and optical data storage. Subwavelength spatial control of both amplitude and phase of light is an essential requirement for an ideal hologram. However, traditional holographic devices suffer from their restricted capabilities of incomplete modulation in both amplitude and phase of visible light; this results in sacrifice of optical information and undesirable occurrences of critical noises in holographic images. Herein, we have proposed a novel metasurface that is capable of completely controlling both the amplitude and phase profiles of visible light independently with subwavelength spatial resolution. The full, continuous, and broadband control of both amplitude and phase was achieved using X-shaped meta-atoms based on the expanded concept of the Pancharatnam-Berry phase. The first experimental demonstrations of the complete complex-amplitude holograms with subwavelength definition at visible wavelengths were achieved, and excellent performances with a remarkable signal-to-noise ratio as compared to those of traditional phase-only holograms were obtained. Extraordinary control capability with versatile advantages of our metasurface paves a way to an ideal holography, which is expected to be a significant advancement in the field of optical holography and metasurfaces.

/pdf/complete-amplitude-and-phase-control-of-light-using-kok2nzsck1.pdf

Complete amplitude and phase control of light using broadband holographic metasurfaces

A holographic display system for realizing a three-dimensional optical see-through augmented reality (AR) is proposed. A multi-functional holographic optical element (HOE), which simultaneously performs the optical functions of a mirror and a lens, is adopted in the system. In the proposed method, a mirror that is used to guide the light source into a reflection type spatial light modulator (SLM) and a lens that functions as Fourier transforming optics are recorded on a single holographic recording material by utilizing an angular multiplexing technique of volume hologram. The HOE is transparent and performs the optical functions just for Bragg matched condition. Therefore, the real-world scenes that are usually distorted by a Fourier lens or an SLM in the conventional holographic display can be observed without visual disturbance by using the proposed mirror-lens HOE (MLHOE). Furthermore, to achieve an optimized optical recording condition of the MLHOE, the optical characteristics of the holographic material are measured. The proposed holographic AR display system is verified experimentally.

Holographic display for see-through augmented reality using mirror-lens holographic optical element.

Reconstruction of light profiles with amplitude and phase information, called holography, is an attractive optical technique to display three-dimensional images. Due to essential requirements for an ideal hologram, subwavelength control of both amplitude and phase is crucial. Nevertheless, traditional holographic devices have suffered from their limited capabilities of incomplete modulation in both amplitude and phase of visible light. Here, we propose a novel metasurface that is capable of completely controlling both amplitude and phase profiles of visible light independently with subwavelength spatial resolution. The simultaneous, continuous, and broadband control of amplitude and phase is achieved by using X-shaped meta-atoms based on expanded concept of the Pancharatnam-Berry phase. The first experimental demonstrations of complete complex-amplitude holograms with subwavelength definition are achieved and show excellent performances with remarkable signal-to-noise ratio compared to traditional phase-only holograms. Extraordinary control capability with versatile advantages of our metasurface paves a way to an ideal holography, which is expected to be a significant advance in the field of optical holography and metasurfaces.

/pdf/complete-amplitude-and-phase-control-of-light-using-51piy6evoh.pdf

Complete amplitude and phase control of light using broadband holographic metasurface

We propose a see-through additive light field display as a novel type of compressive light field display. We utilize holographic optical elements (HOEs) as transparent additive layers. The HOE layers are almost free from diffraction unlike spatial light modulator layers, which makes this additive light field display more advantageous when modifying the number of layers, thickness, and pixel density compared with conventional compressive displays. Meanwhile, the additive light field display maintains advantages of compressive light field displays. The proposed additive light field display shows bright and full-color volumetric images in high definition. In addition, users can view real-world scenes beyond the displays. Hence, we expect that our method can contribute to the realization of augmented reality. Here, we describe implementation of a prototype additive light field display with two additive layers, evaluate the performance of transparent HOE layers, describe several results of display experiments, discuss the diffraction effect of spatial light modulators, and analyze the ability of the additive light field display to express uncorrelated light fields.

Additive light field displays: realization of augmented reality with holographic optical elements

The ultimate 3D displays should provide both psychological and physiological cues for depth recognition. However, it has been challenging to satisfy the essential features without making sacrifices in the resolution, frame rate, and eye box. Here, we present a tomographic near-eye display that supports a wide depth of field, quasi-continuous accommodation, omni-directional motion parallax, preserved resolution, full frame, and moderate field of view within a sufficient eye box. The tomographic display consists of focus-tunable optics, a display panel, and a fast spatially adjustable backlight. The synchronization of the focus-tunable optics and the backlight enables the display panel to express the depth information. We implement a benchtop prototype near-eye display, which is the most promising application of tomographic displays. We conclude with a detailed analysis and thorough discussion of the display's optimal volumetric reconstruction. of tomographic displays. Creating realistic 3D displays that are comfortable to use is both a complicated and important problem. Here the authors present a tomographic near-eye display that uses high-speed synchronization of a lens focus and backlight to produce realistic depth cues for effective and comfortable 3D viewing.

/pdf/tomographic-near-eye-displays-2rfru77rff.pdf

Jaebum Cho

Papers

Complete amplitude and phase control of light using broadband holographic metasurfaces

Holographic display for see-through augmented reality using mirror-lens holographic optical element.

Complete amplitude and phase control of light using broadband holographic metasurface

Additive light field displays: realization of augmented reality with holographic optical elements

Tomographic near-eye displays.