Abstract With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional “physics-based” approach, deep-learning-enabled optical metrology is a kind of “data-driven” approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined. 

/pdf/deep-learning-in-optical-metrology-a-review-2ggtgtrp.pdf

Deep learning in optical metrology: a review

/pdf/deep-learning-in-optical-metrology-a-review-1etdbnzu.pdf

Phase unwrapping is an ill-posed classical problem in many practical applications of significance such as 3D profiling through fringe projection, synthetic aperture radar and magnetic resonance imaging. Conventional phase unwrapping techniques estimate the phase either by integrating through the confined path (referred to as path-following methods) or by minimizing the energy function between the wrapped phase and the approximated true phase (referred to as minimum-norm approaches). However, these conventional methods have some critical challenges like error accumulation and high computational time and often fail under low SNR conditions. To address these problems, this paper proposes a novel deep learning framework for unwrapping the phase and is referred to as “PhaseNet 2.0”. The phase unwrapping problem is formulated as a dense classification problem and a fully convolutional DenseNet based neural network is trained to predict the wrap-count at each pixel from the wrapped phase maps. To train this network, we simulate arbitrary shapes and propose new loss function that integrates the residues by minimizing the difference of gradients and also uses $L_{1}$ loss to overcome class imbalance problem. The proposed method, unlike our previous approach PhaseNet, does not require post-processing, highly robust to noise, accurately unwraps the phase even at the severe noise level of −5 dB, and can unwrap the phase maps even at relatively high dynamic ranges. Simulation results from the proposed framework are compared with different classes of existing phase unwrapping methods for varying SNR values and discontinuity, and these evaluations demonstrate the advantages of the proposed framework. We also demonstrate the generality of the proposed method on 3D reconstruction of synthetic CAD models that have diverse structures and finer geometric variations. Finally, the proposed method is applied to real-data for 3D profiling of objects using fringe projection technique and digital holographic interferometry. The proposed framework achieves significant improvements over existing methods while being highly efficient with interactive frame-rates on modern GPUs.

PhaseNet 2.0: Phase Unwrapping of Noisy Data Based on Deep Learning Approach

This study proposes and evaluates five deep fully convolutional networks (FCNs) for the semantic segmentation of a single tree species: SegNet, U-Net, FC-DenseNet, and two DeepLabv3+ variants. The performance of the FCN designs is evaluated experimentally in terms of classification accuracy and computational load. We also verify the benefits of fully connected conditional random fields (CRFs) as a post-processing step to improve the segmentation maps. The analysis is conducted on a set of images captured by an RGB camera aboard a UAV flying over an urban area. The dataset also contains a mask that indicates the occurrence of an endangered species called Dipteryx alata Vogel, also known as cumbaru, taken as the species to be identified. The experimental analysis shows the effectiveness of each design and reports average overall accuracy ranging from 88.9% to 96.7%, an F1-score between 87.0% and 96.1%, and IoU from 77.1% to 92.5%. We also realize that CRF consistently improves the performance, but at a high computational cost.

https://www.mdpi.com/1424-8220/20/2/563/pdf

Applying Fully Convolutional Architectures for Semantic Segmentation of a Single Tree Species in Urban Environment on High Resolution UAV Optical Imagery

Interferometric synthetic aperture radar (InSAR) is a radar technique widely used in geodesy and remote sensing applications, e.g., topography reconstruction and subsidence estimation. Phase unwrapping (PU) is one of the key procedures of InSAR signal processing. Artificial intelligence (AI) techniques have proven to be potentially powerful in many fields and have been introduced into the PU domain, achieving superior performance. In this article, we provide a comprehensive overview of AI-based PU techniques in InSAR. We survey the AI-based single-baseline (SB) PU methods and then review the AI techniques related to multibaseline (MB) PU. In addition, we show several experimental examples of these methods, from both simulated and real InSAR data sets, which gives readers an overview of AI-based PU processing's potential and limitations. It is our hope that this article will provide researchers with guidelines and inspiration to further enhance the development of AI-based PU.

Artificial Intelligence In Interferometric Synthetic Aperture Radar Phase Unwrapping: A Review

Two-dimensional phase unwrapping algorithms are widely used in optical metrology and measurements. The high noise from interference measurements, however, often leads to the failure of conventional phase unwrapping algorithms. In this paper, we propose a deep convolutional neural network (DCNN) based method to perform rapid and robust two-dimensional phase unwrapping. In our approach, we employ a DCNN architecture, DeepLabV3+, with noise suppression and strong feature representation capabilities. The employed DCNN is first used to perform semantic segmentation to obtain the segmentation result of the wrapped phase map. We then combine the wrapped phase map with the segmentation result to generate the unwrapped phase. We benchmarked our results by comparing them with well-established methods. The reported approach out-performed the conventional path-dependent and path-independent algorithms. We also tested the robustness of the reported approach using interference measurements from optical metrology setups. Our results, again, clearly out-performed the conventional phase unwrap algorithms. The reported approach may find applications in optical metrology and microscopy imaging.

Rapid and robust two-dimensional phase unwrapping via deep learning.

Optical sectioning imaging with high spatial resolution deep inside scattering samples such as mammalian brain is of great interest in biological study. Conventional two-photon microscopy deteriorates in focus when light scattering increases. Here we develop an optical sectioning enhanced two-photon technique which incorporates structured illumination into line-scanning spatial-temporal focusing microscopy (LTSIM), and generate patterned illumination via laser intensity modulation synchronized with scanning. LTSIM brings scattering background elimination and in-focus contrast enhancement, and realizes nearly 2-fold increase in spatial resolution to ∼208 nm laterally and ∼0.94 µm axially. In addition, the intensity modulated line-scanning implementation of LTSIM enables fast and flexible generation of structured illumination, permitting adjustable spatial frequency profiles to optimize image contrast. The highly qualified optical sectioning ability of our system is demonstrated on samples including tissue phantom, C. elegans and mouse brain at depths over hundreds of microns.

Jia Hou

Papers

Rapid and robust two-dimensional phase unwrapping via deep learning.

Contrast and resolution enhanced optical sectioning in scattering tissue using line-scanning two-photon structured illumination microscopy