Course Description This is an advanced course in which we explore the field of Statistical Optics. Topics covered include such subjects as the statistical properties of natural (thermal) and laser light, spatial and temporal coherence, effects of partial coherence on optical imaging instruments, effects on imaging due to randomly inhomogeneous media, and a statistical treatment of the detection of light. Development of this more comprehensive model of the behavior of light draws upon the use of tools traditionally available to the electrical engineer, such as linear system theory and the theory of stochastic processes.

http://ieeexplore.ieee.org/iel5/3/23094/01073023.pdf

Statistical optics

Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography. In this study, we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training. This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts. This neural network-based method is fast to compute and reconstructs phase and amplitude images of the objects using only one hologram, requiring fewer measurements in addition to being computationally faster. We validated this method by reconstructing the phase and amplitude images of various samples, including blood and Pap smears and tissue sections. These results highlight that challenging problems in imaging science can be overcome through machine learning, providing new avenues to design powerful computational imaging systems.

/pdf/phase-recovery-and-holographic-image-reconstruction-using-1ue2w051g5.pdf

Phase recovery and holographic image reconstruction using deep learning in neural networks

Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these developments have always been to the benefit of image interpretation and machine vision, only recently has it become evident that machine learning architectures, and deep neural networks in particular, can be effective for computational image formation, aside from interpretation. The deep learning approach has proven to be especially attractive when the measurement is noisy and the measurement operator ill posed or uncertain. Examples reviewed here are: super-resolution; lensless retrieval of phase and complex amplitude from intensity; photon-limited scenes, including ghost imaging; and imaging through scatter. In this paper, we cast these works in a common framework. We relate the deep-learning-inspired solutions to the original computational imaging formulation and use the relationship to derive design insights, principles, and caveats of more general applicability. We also explore how the machine learning process is aided by the physics of imaging when ill posedness and uncertainties become particularly severe. It is hoped that the present unifying exposition will stimulate further progress in this promising field of research.

On the use of deep learning for computational imaging

Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging. We explore the central prevailing themes of this emerging area and present a taxonomy that can be used to categorize different problems and reconstruction methods. Our taxonomy is organized along two central axes: (1) whether or not a forward model is known and to what extent it is used in training and testing, and (2) whether or not the learning is supervised or unsupervised, i.e., whether or not the training relies on access to matched ground truth image and measurement pairs. We also discuss the tradeoffs associated with these different reconstruction approaches, caveats and common failure modes, plus open problems and avenues for future work.

Deep Learning Techniques for Inverse Problems in Imaging

Digitally sampled in-line holograms may be linearly filtered to reconstruct a representation of the original object distribution, thereby decoding the information contained in the hologram. The decoding process is performed by digital computation rather than optically. Substitution of digital for optical decoding has several advantages, including selective suppression of the twin-image artifact, elimination of the far-field requirement, and automation of the data reduction and analysis process. The proposed filter is a truncated series expansion of the inverse of that operator that maps object opacity function to hologram intensity. The first term of the expansion is shown to be equivalent to conventional (optical) reconstruction, with successive terms increasingly sup-pressing the twin image. The algorithm is computationally efficient, requiring only a single fast Fourier transform pair.

Digital decoding of in-line holograms

In this manuscript, we propose a novel framework of computational ghost imaging, i.e., ghost imaging using deep learning (GIDL). With a set of images reconstructed using traditional GI and the corresponding ground-truth counterparts, a deep neural network was trained so that it can learn the sensing model and increase the quality image reconstruction. Moreover, detailed comparisons between the image reconstructed using deep learning and compressive sensing shows that the proposed GIDL has a much better performance in extremely low sampling rate. Numerical simulations and optical experiments were carried out for the demonstration of the proposed GIDL.

/pdf/deep-learning-based-ghost-imaging-4avwd8zh1l.pdf

Deep-learning-based ghost imaging.

Most of the neural networks proposed so far for computational imaging (CI) in optics employ a supervised training strategy, and thus need a large training set to optimize their weights and biases. Setting aside the requirements of environmental and system stability during many hours of data acquisition, in many practical applications, it is unlikely to be possible to obtain sufficient numbers of ground-truth images for training. Here, we propose to overcome this limitation by incorporating into a conventional deep neural network a complete physical model that represents the process of image formation. The most significant advantage of the resulting physics-enhanced deep neural network (PhysenNet) is that it can be used without training beforehand, thus eliminating the need for tens of thousands of labeled data. We take single-beam phase imaging as an example for demonstration. We experimentally show that one needs only to feed PhysenNet a single diffraction pattern of a phase object, and it can automatically optimize the network and eventually produce the object phase through the interplay between the neural network and the physical model. This opens up a new paradigm of neural network design, in which the concept of incorporating a physical model into a neural network can be generalized to solve many other CI problems.

/pdf/phase-imaging-with-an-untrained-neural-network-225tm5sdo3.pdf

Phase imaging with an untrained neural network.

It is well known that in-line digital holography (DH) makes use of the full pixel count in forming the holographic imaging. But it usually requires phase-shifting or phase retrieval techniques to remove the zero-order and twin-image terms, resulting in the so-called two-step reconstruction process, i.e., phase recovery and focusing. Here, we propose a one-step end-to-end learning-based method for in-line holography reconstruction, namely, the eHoloNet, which can reconstruct the object wavefront directly from a single-shot in-line digital hologram. In addition, the proposed learning-based DH technique has strong robustness to the change of optical path difference between reference beam and object light and does not require the reference beam to be a plane or spherical wave.

eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction.

The problem of imaging through thick scattering media is encountered in many disciplines of science, ranging from mesoscopic physics to astronomy. Photons become diffusive after propagating through a scattering medium with an optical thickness of over 10 times the scattering mean free path. As a result, no image but only noise-like patterns can be directly formed. We propose a hybrid neural network for computational imaging through such thick scattering media, demonstrating the reconstruction of image information from various targets hidden behind a white polystyrene slab of 3 mm in thickness or 13.4 times the scattering mean free path. We also demonstrate that the target image can be retrieved with acceptable quality from a very small fraction of its scattered pattern, suggesting that the speckle pattern produced in this way is highly redundant. This leads to a profound question of how the information of the target being encoded into the speckle is to be addressed in future studies.

/pdf/learning-based-lensless-imaging-through-optically-thick-27u378u54d.pdf

Learning-based lensless imaging through optically thick scattering media

Typical methods of automatic estimation of focusing in digital holography calculate every single reconstructed frame to get a critical function and then ascertain the focal plane by finding the extreme value of that function. Here, we propose a digital holographic autofocusing method that computes the focused distance using the first longitudinal difference of the magnitude of the reconstructed image. We demonstrate the proposed method with both numerical simulations and optical experiments of amplitude-contrast and phase-contrast objects. The results suggest that the proposed method performs better than other existing methods, in terms of applicability and computation efficiency, with potential applications in industrial and biomedical inspections where automatic focus tracking is necessary.

Meng Lyu

Papers

Deep-learning-based ghost imaging.

Phase imaging with an untrained neural network.

eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction.

Learning-based lensless imaging through optically thick scattering media

Fast autofocusing in digital holography using the magnitude differential.