Three-dimensional Shape Reconstruction from Single-shot Speckle Image Using Deep Convolutional Neural Networks

Accurate 3D reconstruction via fringe-to-phase network

Retrieving three-dimensional (3D) shape information from a single two-dimensional (2D) image has recently gained enormous attention in a variety of fields. In spite of recent advancements in algorithms and hardware developments, the easy-to-use characteristics and the accuracy of the 3D shape reconstruction are always of great interest. This paper presents a robust 3D shape reconstruction technique that integrates structured-light 3D imaging scheme with deep convolutional neural network (CNN) learning. The structured-light patterns facilitate the featuring process while the CNN modeling surpasses the complexity of the traditional 3D shape reconstructions. In the supervised learning pipeline, the input is either a single fringe-pattern or a single speckle-pattern image, and the output is its corresponding high-accuracy 3D shape label. Unlike the well-received autoencoder-based CNN model, a global guidance network path with multi-scale feature fusion is introduced into the CNN model to improve the accuracy of the 3D shape reconstruction. Experimental evaluations have been conducted to demonstrate the validity and robustness of the proposed technique, which provides a promising tool for ever-increasing scientific research and engineering applications.

hNet: Single-shot 3D shape reconstruction using structured light and h-shaped global guidance network

Reconstructing 3D geometric representation of objects with deep learning frameworks has recently gained a great deal of interest in numerous fields. The existing deep-learning-based 3D shape reconstruction techniques generally use a single red–green–blue (RGB) image, and the depth reconstruction accuracy is often highly limited due to a variety of reasons. We present a 3D shape reconstruction technique with an accuracy enhancement strategy by integrating the structured-light scheme with deep convolutional neural networks (CNNs). The key idea is to transform multiple (typically two) grayscale images consisting of fringe and/or speckle patterns into a 3D depth map using an end-to-end artificial neural network. Distinct from the existing autoencoder-based networks, the proposed technique reconstructs the 3D shape of target using a refinement approach that fuses multiple feature maps to obtain multiple outputs with an accuracy-enhanced final output. A few experiments have been conducted to verify the robustness and capabilities of the proposed technique. The findings suggest that the proposed network approach can be a promising 3D reconstruction technique for future academic research and industrial applications.

MIMONet: Structured-light 3D shape reconstruction by a multi-input multi-output network.

Exploiting Phase-Based Motion Magnification for the measurement of subtle 3D deformation maps with FP + 2D-DIC

With ever-increasing demand for three-dimensional (3D) imaging and shape measurements in a variety of fields, measurement accuracy has become of vital importance to numerous scientific and engineering applications. This paper presents an experimental investigation into the accuracy comparison of two prevalent 3D imaging and shape measurement methods: fringe projection profilometry (FPP) and 3D digital image correlation (3D-DIC) techniques. A detailed description of their principles reveals their inherent similarities and fundamental differences. A measurement system composed of both techniques is employed in the study, and a test target with speckle checkerboard patterns on its surface is adopted to allow simultaneous FPP and 3D-DIC measurements. The evaluation puts emphasis on how the geometric angles between key hardware components affect the 3D measurement accuracy. Experiments show that the depth and height measurements of both techniques can reach sub-micron accuracy, and the relative accuracy of the 3D shape or position measurements can reach 1/600 000.

https://iopscience.iop.org/article/10.1088/2515-7647/abcbe4/pdf

Jiali Liang

Papers

Accuracy assessment of fringe projection profilometry and digital image correlation techniques for three-dimensional shape measurements