Abstract With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional “physics-based” approach, deep-learning-enabled optical metrology is a kind of “data-driven” approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined. 

/pdf/deep-learning-in-optical-metrology-a-review-2ggtgtrp.pdf

Deep learning in optical metrology: a review

/pdf/deep-learning-in-optical-metrology-a-review-1etdbnzu.pdf

Calibration of fringe projection profilometry: A comparative review

Micro deep learning profilometry for high-speed 3D surface imaging

By combining a fringe projection setup with a telecentric lens, a fringe pattern could be projected and imaged within a small area, making it possible to measure the three-dimensional (3D) surfaces of micro-components. This paper focuses on the flexible calibration of the fringe projection profilometry (FPP) system using a telecentric lens. An analytical telecentric projector-camera calibration model is introduced, in which the rig structure parameters remain invariant for all views, and the 3D calibration target can be located on the projector image plane with sub-pixel precision. Based on the presented calibration model, a two-step calibration procedure is proposed. First, the initial parameters, e.g., the projector-camera rig, projector intrinsic matrix, and coordinates of the control points of a 3D calibration target, are estimated using the affine camera factorization calibration method. Second, a bundle adjustment algorithm with various simultaneous views is applied to refine the calibrated parameters, especially the rig structure parameters and coordinates of the control points forth 3D target. Because the control points are determined during the calibration, there is no need for an accurate 3D reference target, whose is costly and extremely difficult to fabricate, particularly for tiny objects used to calibrate the telecentric FPP system. Real experiments were performed to validate the performance of the proposed calibration method. The test results showed that the proposed approach is very accurate and reliable.

Calibration method for projector-camera-based telecentric fringe projection profilometry system.

Two major methods for 3D reconstruction in fringe projection profilometry, phase-height mapping and stereovision, have their respective problems: the former has low-flexibility in practical application due to system restrictions and the latter requires time-consuming homogenous points searching. Given these limitations, we propose a phase-3D mapping method developed from back-projection stereovision model to achieve flexible and high-efficient 3D reconstruction for fringe projection profilometry. We showed that all dimensional coordinates (X, Y, and Z), but not just the height coordinate (Z), of a measured point can be mapped from phase through corresponding rational functions directly and independently. To determine the phase-3D mapping coefficients, we designed a flexible two-step calibration strategy. The first step, ray reprojection calibration, is to determine the stereovision system parameters; the second step, sampling-mapping calibration, is to fit the mapping coefficients using the calibrated stereovision system parameters. Experimental results demonstrated that the proposed method was suitable for flexible and high-efficient 3D reconstruction that eliminates practical restrictions and dispenses with the time-consuming homogenous point searching.

Phase-3D mapping method developed from back-projection stereovision model for fringe projection profilometry.

This Letter reports a novel method to establish the metric relationship of depth value between object space and image space for unfocused plenoptic cameras. A three-dimensional (3D) measurement system was introduced to precisely construct benchmarks and matching features to compute the metric depths in the object space and the corresponding depth values in the image space for metric calibration. After metric calibration, precise measurement of the depth dimension was possible. Furthermore, with the aid of metric spatio-angular parameters determined via light field ray calibration, transverse dimensions were computed from the measured depth, realizing light field 3D measurement for unfocused plenoptic cameras. Finally, we experimentally performed accuracy analysis of the proposed method with depth measurement precision of 0.5 mm in a depth range of 300 mm, which illuminated potential applications of unfocused plenoptic cameras in the field of 3D measurement.

Light field 3D measurement using unfocused plenoptic cameras.

Ambiguity caused by a wrapped phase is an intrinsic problem in fringe projection-based 3D shape measurement. Among traditional methods for avoiding phase ambiguity, spatial phase unwrapping is sensitive to sensor noise and depth discontinuity, and temporal phase unwrapping requires additional encoding information that leads to an increase of image sequence acquisition time or a reduction of fringe contrast. Here, to the best of our knowledge, we report a novel method of absolute phase unwrapping based on light field imaging. In a recorded light field under structured illumination, i.e., a structured light field, a wrapped phase-encoded field can be retrieved and resampled in diverse image planes associated with several possible fringe orders in a measurement volume. Then, by leveraging phase consistency constraint in the resampled wrapped phase-encoded field, correct fringe orders can be determined to unwrap the wrapped phase without any additional encoding information. Experimental results demonstrated that the proposed method was suitable for accurate and robust absolute phase unwrapping.

Light-field-based absolute phase unwrapping.

An improved spatiotemporal correlation method for high-accuracy random speckle 3D reconstruction

Micro-Electro-Mechanical System (MEMS) scanning is increasingly popular in 3D surface measurement with the merits of the compact structure and high frame-rate. In this paper, we achieve real-time fringe structured 3D reconstruction by using a uniaxial MEMS-based projector. To overcome the limitations on uniaxial MEMS-based projector of lensless structure and unidirectional fringe projection, a novel isophase plane model is proposed, in which the laser line from MEMS-based projector is regarded as an isophase plane. Our model directly establishes the mapping relationship between phase and spatial 3D coordinates through the intersection point of camera back-projection light ray and isophase plane. Furthermore, a flexible calibration strategy to obtain 3D mapping coefficients is introduced with a specially designed planar target. Experiments demonstrated that our method can achieve high-accuracy and real-time 3D reconstruction.

Tang Qijian

Papers

Phase-3D mapping method developed from back-projection stereovision model for fringe projection profilometry.

Light field 3D measurement using unfocused plenoptic cameras.

Light-field-based absolute phase unwrapping.

An improved spatiotemporal correlation method for high-accuracy random speckle 3D reconstruction

High-efficiency 3D reconstruction with a uniaxial MEMS-based fringe projection profilometry