Open AccessPosted Content
Object Goal Navigation using Goal-Oriented Semantic Exploration
Reads0
Chats0
TLDR
A modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category and outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map- based methods.Citations
More filters
Proceedings ArticleDOI
PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
Santhosh K. Ramakrishnan,Devendra Singh Chaplot,Ziad Al-Halah,Jitendra Malik,Kristen Grauman +4 more
TL;DR: A network that predicts two complementary potential functions conditioned on a semantic map and uses them to decide where to look for an unseen object is proposed, a modular approach that disentangles the skills of 'where to look?' for an object and 'how to navigate to $(x,\ y)$?’.
Journal ArticleDOI
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration
TL;DR: This paper translates the success of zero-shot vision models to the popular embodied AI task of object navigation, and finds that a straightforward CoW, with CLIP-based object localization plus classical exploration, and no additional training, often outperforms learnable approaches in terms of success, efficiency, and robustness to dataset distribution shift.
Proceedings ArticleDOI
Semantic Exploration from Language Abstractions and Pretrained Representations
Allison C. Tam,Neil C. Rabinowitz,Andrew K. Lampinen,Nicholas Roy,Stephanie C.Y. Chan,DJ Strouse,Jane X. Wang,Andrea Banino,Felix Hill +8 more
TL;DR: This work evaluates vision-language representations, pretrained on natural image captioning datasets, and shows that these pretrained representations drive meaningful, task-relevant exploration and improve performance on 3D simulated environments.
Proceedings ArticleDOI
Open-vocabulary Queryable Scene Representations for Real World Planning
Boyuan Chen,Fei Xia,Brian Ichter,Kanishka Rao,K. Gopalakrishnan,Michael S. Ryoo,Austin Stone,Daniel Kappler +7 more
TL;DR: NLMap is developed, an open-vocabulary and queryable scene representation that allows robots to operate without a fixed list of objects nor executable options, enabling real robot operation unachievable by previous methods.
Proceedings ArticleDOI
Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
TL;DR: A large-scale study of imitating human demonstrations on tasks that require a virtual robot to search for objects in new environments - ObjectGoal Navigation and Pick&place - finds the IL-trained agent learns efficient object-search behavior from humans.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Book ChapterDOI
Microsoft COCO: Common Objects in Context
Tsung-Yi Lin,Michael Maire,Serge Belongie,James Hays,Pietro Perona,Deva Ramanan,Piotr Dollár,C. Lawrence Zitnick +7 more
TL;DR: A new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding by gathering images of complex everyday scenes containing common objects in their natural context.
Proceedings ArticleDOI
Feature Pyramid Networks for Object Detection
TL;DR: This paper exploits the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost and achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles.
Proceedings ArticleDOI
Mask R-CNN
TL;DR: This work presents a conceptually simple, flexible, and general framework for object instance segmentation, which extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition.
Automatic differentiation in PyTorch
Adam Paszke,Sam Gross,Soumith Chintala,Gregory Chanan,Edward Z. Yang,Zachary DeVito,Zeming Lin,Alban Desmaison,Luca Antiga,Adam Lerer +9 more
TL;DR: An automatic differentiation module of PyTorch is described — a library designed to enable rapid research on machine learning models that focuses on differentiation of purely imperative programs, with a focus on extensibility and low overhead.