MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
Martin Rünz,Maud Buffier,Lourdes Agapito +2 more
- pp 10-20
Reads0
Chats0
TLDR
MaskFusion as discussed by the authors is a real-time object-aware, semantic and dynamic RGB-D SLAM system that goes beyond traditional systems which output a purely geometric map of a static scene.Abstract:
We present MaskFusion, a real-time, object-aware, semantic and dynamic RGB-D SLAM system that goes beyond traditional systems which output a purely geometric map of a static scene. MaskFusion recognizes, segments and assigns semantic class labels to different objects in the scene, while tracking and reconstructing them even when they move independently from the camera. As an RGB-D camera scans a cluttered scene, image-based instance-level semantic segmentation creates semantic object masks that enable realtime object recognition and the creation of an object-level representation for the world map. Unlike previous recognition-based SLAM systems, MaskFusion does not require known models of the objects it can recognize, and can deal with multiple independent motions. MaskFusion takes full advantage of using instance-level semantic segmentation to enable semantic labels to be fused into an object-aware map, unlike recent semantics enabled SLAM systems that perform voxel-level semantic segmentation. We show augmented-reality applications that demonstrate the unique features of the map output by MaskFusion: instance-aware, semantic and dynamic. Code will be made available.read more
Citations
More filters
Proceedings ArticleDOI
Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping
TL;DR: Kimera as discussed by the authors is an open-source C++ library for real-time metric-semantic visual-inertial SLAM by enabling mesh reconstruction and semantic labeling in 3D.
Proceedings ArticleDOI
Fusion++: Volumetric Object-Level SLAM
TL;DR: In this article, Mask-RCNN instance segmentation is used to initialise compact per-object Truncated Signed Distance Function (TSDF) reconstructions with object size-dependent resolutions and a novel 3D foreground mask.
Proceedings ArticleDOI
SuMa++: Efficient LiDAR-based Semantic SLAM
Xieyuanli Chen,Andres Milioto,Emanuele Palazzolo,Philippe Giguère,Jens Behley,Cyrill Stachniss +5 more
TL;DR: An extension of a recently published surfel-based mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process, which enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints.
Journal ArticleDOI
Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
Margarita Grinvald,Fadri Furrer,Tonci Novkovic,Jen Jen Chung,Cesar Cadena,Roland Siegwart,Juan Nieto +6 more
TL;DR: This work presents an approach to incrementally build volumetric object-centric maps during online scanning with a localized RGB-D camera and demonstrates that the proposed approach for building instance-level semantic maps is competitive with state-of-the-art methods, while additionally able to discover objects of unseen categories.
Proceedings ArticleDOI
PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things
TL;DR: PanopticFusion as discussed by the authors predicts pixel-wise panoptic labels (class labels for stuff regions and instance IDs for thing regions) for incoming RGB frames by fusing 2D semantic and instance segmentation outputs.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Posted Content
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
TL;DR: Faster R-CNN as discussed by the authors proposes a Region Proposal Network (RPN) to generate high-quality region proposals, which are used by Fast R-NN for detection.
Proceedings ArticleDOI
Mask R-CNN
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation, which extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition.
Proceedings Article
Faster R-CNN: towards real-time object detection with region proposal networks
TL;DR: Ren et al. as discussed by the authors proposed a region proposal network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals.
Proceedings Article
Mask R-CNN
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation that outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners.