Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new goals, and (2) data inefficiency, i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows better generalization. To address the second issue, we propose the AI2-THOR framework, which provides an environment with high-quality 3D scenes and a physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment.

Target-driven visual navigation in indoor scenes using deep reinforcement learning

Surveys the developments of the last 20 years in the area of vision for mobile robot navigation. Two major components of the paper deal with indoor navigation and outdoor navigation. For each component, we have further subdivided our treatment of the subject on the basis of structured and unstructured environments. For indoor robots in structured environments, we have dealt separately with the cases of geometrical and topological models of space. For unstructured environments, we have discussed the cases of navigation using optical flows, using methods from the appearance-based paradigm, and by recognition of specific objects in the environment.

/pdf/vision-for-mobile-robot-navigation-a-survey-4hb3uw9u9w.pdf

Vision for mobile robot navigation: a survey

This book presents methods to study the controllability and the stabilization of nonlinear control systems in finite and infinite dimensions. The emphasis is put on specific phenomena due to nonlinearities. In particular, many examples are given where nonlinearities turn out to be essential to get controllability or stabilization. Various methods are presented to study the controllability or to construct stabilizing feedback laws. The power of these methods is illustrated by numerous examples coming from such areas as celestial mechanics, fluid mechanics, and quantum mechanics. The book is addressed to graduate students in mathematics or control theory, and to mathematicians or engineers with an interest in nonlinear control systems governed by ordinary or partial differential equations.

/pdf/control-and-nonlinearity-4tu8aduvuy.pdf

Control and Nonlinearity

The subject of the paper is the motion control problem of wheeled mobile robots (WMRs) in environments without obstacles. With reference to the popular unicycle kinematics, it is shown that dynamic feedback linearization is an efficient design tool leading to a solution simultaneously valid for both trajectory tracking and setpoint regulation problems. The implementation of this approach on the laboratory prototype SuperMARIO, a two-wheel differentially driven mobile robot, is described in detail. To assess the quality of the proposed controller, we compare its performance with that of several existing control techniques in a number of experiments. The obtained results provide useful guidelines for WMR control designers.

/pdf/wmr-control-via-dynamic-feedback-linearization-design-25vbjh5hnl.pdf

WMR control via dynamic feedback linearization: design, implementation, and experimental validation

http://www.centropiaggio.unipi.it/sites/default/files/softarm-MMT.pdf

An atlas of physical human-robot interaction

An algorithmic solution method is presented for the problem of autonomous robot motion in completely unknown environments. Our approach is based on the alternate execution of two fundamental processes: map building and navigation. In the former, range measures are collected through the robot exteroceptive sensors and processed in order to build a local representation of the surrounding area. This representation is then integrated in the global map so far reconstructed by filtering out insufficient or conflicting information. In the navigation phase, an A*-based planner generates a local path from the current robot position to the goal. Such a path is safe inside the explored area and provides a direction for further exploration. The robot follows the path up to the boundary of the explored area, terminating its motion if unexpected obstacles are encountered. The most peculiar aspects of our method are the use of fuzzy logic for the efficient building and modification of the environment map, and the iterative application of A*, a complete planning algorithm which takes full advantage of local information. Experimental results for a NOMAD 200 mobile robot show the real-time performance of the proposed method, both in static and moderately dynamic environments.

https://www.diag.uniroma1.it/%7Elabrob/pub/papers/SMC98.pdf

Real-time map building and navigation for autonomous robots in unknown environments

The subject of this chapter is the motion control problem of wheeled mobile robots (WMRs) With reference to the unicycle kinematics, we review and compare several control strategies for trajectory tracking and posture stabilization in an environment free of obstacles Experiments are reported for SuperMARIO, a two-wheel differentially-driven mobile robot From the comparison of the obtained results, guidelines are provided for WMR end-users

/pdf/control-of-wheeled-mobile-robots-an-experimental-overview-5embqjci9i.pdf

Control of Wheeled Mobile Robots: An Experimental Overview

We present a decentralized cooperative exploration strategy for a team of mobile robots equipped with range finders. A roadmap of the explored area, with the associate safe region, is built in the form of a sensor-based random graph (SRG). This is expanded by the robots by using a randomized local planner that automatically realizes a tradeoff between information gain and navigation cost. The nodes of the SRG represent view configurations that have been visited by at least one robot, and are connected by arcs that represent safe paths. These paths have been actually traveled by the robots or added to the SRG to improve its connectivity. Decentralized cooperation and coordination mechanisms are used so as to guarantee exploration efficiency and avoid conflicts. Simulations and experiments are presented to show the performance of the proposed technique.

The Sensor-based Random Graph Method for Cooperative Robot Exploration

An essential component of an autonomous mobile robot is the exteroceptive sensory system. Sensing capabilities should be integrated with a method for extracting a representation of the environment from uncertain sensor data and with an appropriate planning algorithm. In this article, fuzzy logic concepts are used to introduce a tool useful for robot perception as well as for planning collision-free motions. In particular, a map of the environment is defined as the fuzzy set of unsafe points, whose membership function quantifies the possibility for each point to belong to an obstacle. The computation of this set is based on a specific sensor model and makes use of intermediate sets generated from range measures and aggregated by means of fuzzy set operators. This general approach is applied to a robot with ultrasonic rangefinders. The resulting map building algorithm performs well, as confirmed by a comparison with stochastic methods. The planning problem on fuzzy maps can be solved by defining various path cost functions, corresponding to different strategies, and by searching the map for optimal paths. To this end, proper instances of the A* algorithm are devised. Experimental results for a Nomad 200™ robot moving in a real-world environment are presented. © 1997 John Wiley & Sons, Inc.

Marilena Vendittelli

Papers

WMR control via dynamic feedback linearization: design, implementation, and experimental validation

Real-time map building and navigation for autonomous robots in unknown environments

Control of Wheeled Mobile Robots: An Experimental Overview

The Sensor-based Random Graph Method for Cooperative Robot Exploration

Fuzzy Maps: A New Tool for Mobile Robot Perception and Planning