Abstract Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers. Here, we present a computer-free, all-optical image reconstruction method to see through random diffusers at the speed of light. Using deep learning, a set of transmissive diffractive surfaces are trained to all-optically reconstruct images of arbitrary objects that are completely covered by unknown, random phase diffusers. After the training stage, which is a one-time effort, the resulting diffractive surfaces are fabricated and form a passive optical network that is physically positioned between the unknown object and the image plane to all-optically reconstruct the object pattern through an unknown, new phase diffuser. We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown, random diffusers, never used during training. Unlike digital methods, all-optical diffractive reconstructions do not require power except for the illumination light. This diffractive solution to see through diffusers can be extended to other wavelengths, and might fuel various applications in biomedical imaging, astronomy, atmospheric sciences, oceanography, security, robotics, autonomous vehicles, among many others. 

https://elight.springeropen.com/counter/pdf/10.1186/s43593-022-00012-4

Computational imaging without a computer: seeing through random diffusers at the speed of light

The remarkable potential of metasurface holography promises revolutionary advancements for imaging, chip-integrated augmented/virtual reality (AR/VR) technology, and flat optical displays. The choice of constituent element geometry constrains many potential applications purveyed through polarization-independent optical response. The limited capabilities and degree of freedoms in commonly used meta-atoms restrict the design flexibility to break the conventional trade-off between polarization-insensitivity and bandwidth. Here, we propose a geometric phase-enabled novel design strategy to break this conventional trade-off. The proposed strategy ensures the realization of broad-band polarization-insensitivity through a simplified design procedure. An identical output wavefront manipulation is achieved by adjusting the phase delay freedom of geometric phase engineering under different incident polarization conditions. For proof of concept, a metahologram device is fabricated by an optimized complementary metal-oxide-semiconductor (CMOS)-compatible material of hydrogenated amorphous silicon (a-Si:H). This metahologram device reproduces the required hologram with high image fidelity and efficiency under different polarization scenarios of white light incidence. Due to the simple design strategy, low computational cost, and easy fabrication, the proposed technique can be an excellent candidate for realizing polarization-insensitive metahologram devices.

Broad-Band Polarization-Insensitive Metasurface Holography with a Single-Phase Map.

Replacing electrons with photons is a compelling route towards light-speed, highly parallel, and low-power artificial intelligence computing. Recently, all-optical diffractive neural deep neural networks have been demonstrated. However, the existing architectures often comprise bulky components and, most critically, they cannot mimic the human brain for multitasking. Here, we demonstrate a multi-skilled diffractive neural network based on a metasurface device, which can perform on-chip multi-channel sensing and multitasking at the speed of light in the visible. The metasurface is integrated with a complementary metal oxide semiconductor imaging sensor. Polarization multiplexing scheme of the subwavelength nanostructures are applied to construct a multi-channel classifier framework for simultaneous recognition of digital and fashionable items. The areal density of the artificial neurons can reach up to 6.25x106/mm2 multiplied by the number of channels. Our platform provides an integrated solution with all-optical on-chip sensing and computing for applications in machine vision, autonomous driving, and precision medicine.

Metasurface-Enabled On-Chip Multiplexed Diffractive Neural Networks in the Visible

Futuristic holographic displays will essentially require broadband chiro‐optical effects for medical imaging, virtual reality, smart security, and optical encryption. However, conventional metasurfaces cannot provide such on‐chip realization of broadband chiro‐optical effects. Moreover, the simultaneous conversion of amplitude, polarization, and phase (APP) at optical wavelengths to introduce giant chirality has not been realized yet. In this paper, a planar all‐dielectric metasurface is proposed incorporating extra degrees of freedom to comprehend the conversion of APP with broadband chiro‐optical effects in terms of giant asymmetric transmission with maximum efficiency of ≈77% at the wavelength of 567 nm. The underlying mechanism behind induced chiro‐optical effects is also investigated using higher‐order multipolar dielectric resonances. Moreover, experimental validation is performed using the reproduced polarization‐encrypted meta‐holograms at broadband visible wavelengths. This work expands the scope of meta‐nanophotonics with potential applications in bioimaging and polarization‐encrypted displays for healthcare and smart security applications.

Broadband Chiro‐Optical Effects for Futuristic Meta‐Holographic Displays

Computational meta-optics brings a twist on the accelerating hardware with the benefits of ultrafast speed, ultra-low power consumption, and parallel information processing in versatile applications. Recent advent of metasurfaces have enabled the full manipulation of electromagnetic waves within subwavelength scales, promising the multifunctional, high-throughput, compact and flat optical processors. In this trend, metasurfaces with nonlocality or multi-layer structures are proposed to perform analog optical computations based on Green's function or Fourier transform, intrinsically constrained by limited operations or large footprints/volume. Here, we showcase a Fourier-based metaprocessor to impart customized highly flexible transfer functions for analog computing upon our single-layer Huygens' metasurface. Basic mathematical operations, including differentiation and cross-correlation, are performed by directly modulating complex wavefronts in spatial Fourier domain, facilitating edge detection and pattern recognition of various image processing. Our work substantiates an ultracompact and powerful kernel processor, which could find important applications for optical analog computing and image processing. 

/pdf/single-layer-spatial-analog-meta-processor-for-imaging-12x2r59z.pdf

Single-layer spatial analog meta-processor for imaging processing

We demonstrate optical networks composed of diffractive layers trained using deep learning to encode the spatial information of objects into the power spectrum of the diffracted light, which are used to classify objects with a single-pixel spectroscopic detector. Using a plasmonic nanoantenna-based detector, we experimentally validated this single-pixel machine vision framework at terahertz spectrum to optically classify the images of handwritten digits by detecting the spectral power of the diffracted light at ten distinct wavelengths, each representing one class/digit. We also coupled this diffractive network-based spectral encoding with a shallow electronic neural network, which was trained to rapidly reconstruct the images of handwritten digits based on solely the spectral power detected at these ten distinct wavelengths, demonstrating task-specific image decompression. This single-pixel machine vision framework can also be extended to other spectral-domain measurement systems to enable new 3D imaging and sensing modalities integrated with diffractive network-based spectral encoding of information.

https://advances.sciencemag.org/content/advances/7/13/eabd7690.full.pdf

Spectrally encoded single-pixel machine vision using diffractive networks

Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers. We present a computer-free, all-optical image reconstruction method to see through random diffusers at the speed of light. Using deep learning, a set of diffractive surfaces are designed/trained to all-optically reconstruct images of objects that are covered by random phase diffusers. We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown, random diffusers, never used during training. Unlike digital methods, all-optical diffractive reconstructions do not require power except for the illumination light. This diffractive solution to see through diffusers can be extended to other wavelengths, and might fuel various applications in biomedical imaging, astronomy, atmospheric sciences, oceanography, security, robotics, among others.

Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light

An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.

Biopsy-free in vivo virtual histology of skin using deep learning.

We demonstrate diffractive optical networks that are trained with deep learning to engineer input terahertz pulses into desired temporal waveforms using passive diffractive surfaces that control the spectral phase and amplitude of the output pulse.

Terahertz Pulse Shaping Using Diffractive Optical Networks

We present and experimentally demonstrate a deep learning-driven machine-vision framework that trains diffractive surfaces to encode the spatial information objects into the output power spectrum for all-optical image classification using a single-pixel spectroscopic detector.

Jingxi Li

Papers

Spectrally encoded single-pixel machine vision using diffractive networks

Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light

Biopsy-free in vivo virtual histology of skin using deep learning.

Terahertz Pulse Shaping Using Diffractive Optical Networks

Single-Pixel Machine Vision Using Spectral Encoding Through Diffractive Optical Networks