Abstract With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional “physics-based” approach, deep-learning-enabled optical metrology is a kind of “data-driven” approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined. 

/pdf/deep-learning-in-optical-metrology-a-review-2ggtgtrp.pdf

Deep learning in optical metrology: a review

/pdf/deep-learning-in-optical-metrology-a-review-1etdbnzu.pdf

We present a learning-based Shack-Hartmann wavefront sensor (SHWS) to achieve the high-order aberration detection without image segmentation or centroid positioning. Zernike coefficient amplitudes of aberrations measured from biological samples are referred and expanded to generate the training datasets. With one SHWS pattern inputted, up to 120th Zernike modes could be predicted within 10.9 ms with 95.56% model accuracy by a personal computer. The statistical experimental results show that compared with traditional modal-based SHWS, the root mean squared error in phase residuals of this method is reduced by ∼40.54% and the Strehl ratio of the point spread functions is improved by ∼27.31%. The aberration detection performance of this method is also validated on a mouse brain slice with 300 µm thickness and the median improvement of peak-to-background ratio of this method is ∼30% to 40% compared with traditional SHWS. With the high detection accuracy, simple processes, fast prediction speed and good compatibility, this work offers a potential approach to improve the wavefront sensing ability of SHWS, which could be combined with an existing adaptive optics system and be further applied in biological applications.

Learning-based Shack-Hartmann wavefront sensor for high-order aberration detection.

Physical and chemical mechanisms of tissue optical clearing.

The vector vortex beams (VVB) possessing non-separable states of light, in which polarization and orbital angular momentum (OAM) are coupled, have attracted more and more attentions in science and technology, due to the unique nature of the light field. However, atmospheric transmission distortion is a recurring challenge hampering the practical application, such as communication and imaging. In this work, we built a deep learning based adaptive optics system to compensate the turbulence aberrations of the vector vortex mode in terms of phase distribution and mode purity. A turbulence aberration correction convolutional neural network (TACCNN) model, which can learn the mapping relationship of intensity profile of the distorted vector vortex modes and the turbulence phase generated by first 20 Zernike modes, is well designed. After supervised learning plentiful experimental samples, the TACCNN model compensates turbulence aberration for VVB quickly and accurately. For the first time, experimental results show that through correction, the mode purity of the distorted VVB improves from 19% to 70% under the turbulence strength of D/r0 = 5.28 with correction time 100 ms. Furthermore, both spatial modes and the light intensity distribution can be well compensated in different atmospheric turbulence.

Turbulence aberration correction for vector vortex beams using deep neural networks on experimental data.

Non-invasive, real-time imaging and deep focus into tissue are in high demand in biomedical research. However, the aberration that is introduced by the refractive index inhomogeneity of biological tissue hinders the way forward. A rapid focusing with sensor-less aberration corrections, based on machine learning, is demonstrated in this paper. The proposed method applies the Convolutional Neural Network (CNN), which can rapidly calculate the low-order aberrations from the point spread function images with Zernike modes after training. The results show that approximately 90 percent correction accuracy can be achieved. The average mean square error of each Zernike coefficient in 200 repetitions is 0.06. Furthermore, the aberration induced by 1-mm-thick phantom samples and 300-µm-thick mouse brain slices can be efficiently compensated through loading a compensation phase on an adaptive element placed at the back-pupil plane. The phase reconstruction requires less than 0.2 s. Therefore, this method offers great potential for in vivo real-time imaging in biological science.

Machine learning guided rapid focusing with sensor-less aberration corrections

The conventional Shack–Hartmann wavefront sensor (SHWS) requires wavefront slope measurements of every micro-lens for wavefront reconstruction. In this Letter, we applied deep learning on the SHWS to directly predict the wavefront distributions without wavefront slope measurements. The results show that our method could provide a lower root mean square wavefront error in high detection speed. The performance of the proposed method is also evaluated on challenging wavefronts, while the conventional approaches perform insufficiently. This Letter provides a new approach, to the best of our knowledge, to perform direct wavefront detection in SHWS-based applications.

Deep learning assisted Shack-Hartmann wavefront sensor for direct wavefront detection.

The doughnut-shaped beam has been widely applied in the field of super-resolution microscopic imaging, micro-nanostructure lithography, ultra-high-density storage, and laser trapping. However, how to maintain the doughnut-shaped focus inside the scattering medium becomes a challenge, due to the wavefront aberrations. Here we demonstrate a machine learning based adaptive optics method to recover the doughnut-shaped focus with high speed. In our method, the relationship between the distorted doughnut-shaped intensity point spread function and the coefficients of the first 15 Zernike modes for phase correction is established. Experimental results show that the wavefront aberration with 101,784 optical control elements can be predicted within ~17 ms even using a personal computer, and 97.5% correction accuracy can be achieved in 200 repeated tests. Besides, we successfully apply this method in the scanning microscopy theoretically. With a large number of optical control elements and fast operation speed, our method may pave the way for many important applications in bioimaging, such as deep tissue stimulated emission depletion (STED) microscopy.

Machine learning based adaptive optics for doughnut-shaped beam.

Adaptive optics has been widely used in biological science to recover high-resolution optical image deep into the tissue, where optical distortion detection with high speed and accuracy is strongly...

https://www.worldscientific.com/doi/pdf/10.1142/S1793545820400015

Hu Lejia

Papers

Machine learning guided rapid focusing with sensor-less aberration corrections

Learning-based Shack-Hartmann wavefront sensor for high-order aberration detection.

Deep learning assisted Shack-Hartmann wavefront sensor for direct wavefront detection.

Machine learning based adaptive optics for doughnut-shaped beam.

Simplifying the detection of optical distortions by machine learning