Micro-object pose estimation with sim-to-real transfer learning using small dataset
Dandan Zhang,Antoine Barbot,Florent Seichepine,Frank P.-W. Lo,Wenjia Bai,Guang-Zhong Yang,Benny Lo +6 more
Reads0
Chats0
TLDR
In this paper , a learning-to-match approach is used to map the generated data and the experimental data to a low-dimensional space with the same data distribution for different pose labels, which ensures effective feature embedding.Abstract:
Abstract Three-dimensional (3D) pose estimation of micro/nano-objects is essential for the implementation of automatic manipulation in micro/nano-robotic systems. However, out-of-plane pose estimation of a micro/nano-object is challenging, since the images are typically obtained in 2D using a scanning electron microscope (SEM) or an optical microscope (OM). Traditional deep learning based methods require the collection of a large amount of labeled data for model training to estimate the 3D pose of an object from a monocular image. Here we present a sim-to-real learning-to-match approach for 3D pose estimation of micro/nano-objects. Instead of collecting large training datasets, simulated data is generated to enlarge the limited experimental data obtained in practice, while the domain gap between the generated and experimental data is minimized via image translation based on a generative adversarial network (GAN) model. A learning-to-match approach is used to map the generated data and the experimental data to a low-dimensional space with the same data distribution for different pose labels, which ensures effective feature embedding. Combining the labeled data obtained from experiments and simulations, a new training dataset is constructed for robust pose estimation. The proposed method is validated with images from both SEM and OM, facilitating the development of closed-loop control of micro/nano-objects with complex shapes in micro/nano-robotic systems. read more
Citations
More filters
Journal ArticleDOI
Lessons from a Space Lab - An Image Acquisition Perspective
Leo Pauly,Michele L. Jamrozik,M. Ortiz del Castillo,Olivia Borgue,Inderpreet Singh,Mohatashem Makhdoomi,Olga-Orsalia Christidi-Loumpasefski,Vincent Gaudilliere,Carol Martinez,A. Rathinam,Andreas Heintz,Miguel A. Olivares-Mendez,Djamila Aouada +12 more
TL;DR: The results from the experiment analyses show that the market survey complimented by experimental analyses is required for effective equipment selection in a space lab development project.
Journal ArticleDOI
Advanced medical micro-robotics for early diagnosis and therapeutic interventions
Dandan Zhang,Thomas E. Gorochowski,Lucia Marucci,Hyun-Taek Lee,Bruno Gil,Bing Li,Sabine Hauert,Eric M. Yeatman +7 more
TL;DR: In this paper , the major challenges, current trends and significant achievements for developing versatile and intelligent micro-robotics with a focus on applications in early diagnosis and therapeutic interventions are reviewed.
Journal ArticleDOI
Fabrication and optical manipulation of micro-robots for biomedical applications
Dandan Zhang,Yunxiao Ren,Antoine Barbot,Florent Seichepine,Benny Lo,Zhuo Ma,Guang-Zhong Yang +6 more
TL;DR: Optical manipulation is a technology that enables accurate manipulation of micro-robots in fluidic environment as discussed by the authors , which can be used as micro-tools to perform indirect micro-objects manipulation via optical tweezers.
Robotic vectorial field alignment for spin-based quantum sensors
TL;DR: In this paper , a robotic arm equipped with a magnet can sensitise an NV center quantum magnetometer in challenging conditions unachievable with standard techniques, which opens up the prospect of integrating robotics across many quantum degrees of freedom in constrained settings, allowing for increased prototyping speed, control, and robustness.
References
More filters
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings ArticleDOI
Image-to-Image Translation with Conditional Adversarial Networks
TL;DR: Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Proceedings ArticleDOI
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
TL;DR: CycleGAN as discussed by the authors learns a mapping G : X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss.
Posted Content
Image-to-Image Translation with Conditional Adversarial Networks
TL;DR: Conditional Adversarial Network (CA) as discussed by the authors is a general-purpose solution to image-to-image translation problems, which can be used to synthesize photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Proceedings Article
Model-agnostic meta-learning for fast adaptation of deep networks
TL;DR: An algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning is proposed.