Yao Lu

TL;DR: It is shown how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.

TL;DR: In this article , the authors present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties and verify their conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size and data diversity based on a large-scale data collection on real robots performing real-world tasks.

TL;DR: In this article , a probabilistic filtering strategy is proposed to solve complex, long-horizon embodiment tasks in a robotic setting by leveraging the knowledge of both language models and grounded models of the environment.

TL;DR: In this paper , a pre-trained vision-language model is used to extract object-identifying information from the language command and image, and conditions the robot policy on the current image, the instruction, and the extracted object information.

TL;DR: It is demonstrated that modeling the predictive information improves success rates on the training tasks and leads to better zero-shot transfer to unseen novel tasks, and is evaluated on real robots, achieving substantial and consistent improvement over QT-Opt.