Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new goals, and (2) data inefficiency, i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows better generalization. To address the second issue, we propose the AI2-THOR framework, which provides an environment with high-quality 3D scenes and a physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment.

Target-driven visual navigation in indoor scenes using deep reinforcement learning

Mobile robot vision-based navigation has been the source of countless research contributions, from the domains of both vision and control. Vision is becoming more and more common in applications such as localization, automatic map construction, autonomous navigation, path following, inspection, monitoring or risky situation detection. This survey presents those pieces of work, from the nineties until nowadays, which constitute a wide progress in visual navigation techniques for land, aerial and autonomous underwater vehicles. The paper deals with two major approaches: map-based navigation and mapless navigation. Map-based navigation has been in turn subdivided in metric map-based navigation and topological map-based navigation. Our outline to mapless navigation includes reactive techniques based on qualitative characteristics extraction, appearance-based localization, optical flow, features tracking, plane ground detection/tracking, etc... The recent concept of visual sonar has also been revised.

Visual Navigation for Mobile Robots: A Survey

This paper presents a new real-time localization system for a mobile robot. We show that autonomous navigation is possible in outdoor situation with the use of a single camera and natural landmarks. To do that, we use a three step approach. In a learning step, the robot is manually guided on a path and a video sequence is recorded with a front looking camera. Then a structure from motion algorithm is used to build a 3D map from this learning sequence. Finally in the navigation step, the robot uses this map to compute its localization in real-time and it follows the learning path or a slightly different path if desired. The vision algorithms used for map building and localization are first detailed. Then a large part of the paper is dedicated to the experimental evaluation of the accuracy and robustness of our algorithms based on experimental data collected during two years in various environments.

/pdf/monocular-vision-for-mobile-robot-localization-and-nipy611zhs.pdf

Monocular Vision for Mobile Robot Localization and Autonomous Navigation

Intraplatoon information management strategies for dealing with safe and stable operation are proposed in this paper. New algorithms to mitigate communication delays are presented, and Matlab/Simulink-based simulation results are reported. We argue that using anticipatory information from both the platoon's leader and the followers significantly impacts platoon string stability. The obtained simulation results suggest that the effects of communication delays may be almost completely canceled out. The platoon presents a very stable behavior, even when subjected to strong acceleration patterns. When the communication channel is subjected to a strong load, proper algorithms may be selected, lowering network load and maintaining string stability. Upon emergency occurrences, the platoon's timely response may be ensured by dynamically increasing the weight of the platoons' leaders data over the behavior of their followers. The simulation results suggest that the algorithms are robust under several demanding scenarios. To assess if current intervehicle communication technology can cope with the proposed information-updating schemes, research into its operation was conducted through a network simulator.

/pdf/platooning-with-ivc-enabled-autonomous-vehicles-strategies-3iwa312kzn.pdf

Platooning With IVC-Enabled Autonomous Vehicles: Strategies to Mitigate Communication Delays, Improve Safety and Traffic Flow

Highway platooning of vehicles has been identified as a promising framework in developing intelligent transportation systems. By autonomous or semi-autonomous vehicle control and intervehicle coordination, an appropriately managed platoon can potentially offer enhanced safety, improved highway utility, increased fuel economy, and reduced emissions. This paper is focused on quantitative characterization of the impact of communication information structures and contents on platoon safety. By comparing different information structures that combine front sensors, rear sensors, and wireless communication channels, as well as different information contents such as distances, speeds, and drivers' actions, we reveal a number of intrinsic relationships between vehicle coordination and communications in platoons. Typical communication standards and related communication latency are used as benchmark cases in this paper. The findings of this paper provide useful guidelines in sensor selections, communication resource allocations, and vehicle coordination.

Communication Information Structures and Contents for Enhanced Safety of Highway Vehicle Platoons

In this paper, a complete system for outdoor robot navigation is presented. It uses only monocular vision. The robot is first guided on a path by a human. During this learning step, the robot records a video sequence. From this sequence, a three dimensional map of the trajectory and the environment is built. When this map has been computed, the robot is able to follow the same trajectory by itself. Experimental results carried out with an urban electric vehicle are shown and compared to the ground truth.

/pdf/outdoor-autonomous-navigation-using-monocular-vision-4wy6atct55.pdf

Outdoor autonomous navigation using monocular vision

To solve problems of traffic saturation in cities, new alternative ''urban transportation systems" are based on electric vehicles in free-access. One necessary functionality of such systems is their ability to move in a platoon fashion. Platooning of these automatic guided vehicles, relying on RTK-GPS sensors and inter-vehicles communication, is addressed in this paper. The developed control law is based on a global control strategy; actually, it can take into account all the platoon state, and not only the immediate previous vehicle state. Distance here is understood as difference of curvilinear abscissa along a reference trajectory. Relying on nonlinear control theory, lateral and longitudinal control are fully decoupled, and therefore addressed independently. To ensure passengers comfort, additional monitoring functions supervise our control system. Then, experiment, carried out with urban vehicles, and simulations of long platoon, are presented.

A global control strategy for urban vehicles platooning relying on nonlinear decoupling laws

Accurate automatic guidance of an urban electric vehicle relying on a kinematic GPS sensor

In order to solve problems of traffic saturation in cities, new alternative” Urban Transportation Systems” are based on electric vehicles in free-access. One necessary functionality of such systems is their ability to move in a platoon fashion. Platooning of these automatic guided vehicles, relying on RTK-GPS sensors and inter-vehicles communication, is addressed in this paper. More precisely, vehicles platoon is expected to follow a curved reference path. Relying on nonlinear control theory, lateral and longitudinal control are fully decoupled, and therefore addressed independently. To ensure passengers comfort, additional monitoring functions supervise our control system. Then, simulations followed by experiments carried out with urban vehicles, are presented.

/pdf/nonlinear-control-for-urban-vehicles-platooning-relying-upon-tn5yd9v8ri.pdf

Nonlinear Control for Urban Vehicles Platooning, Relying upon a Unique Kinematic GPS

Automatic guidance of urban electric vehicles relying on a sensory memory is addressed in this presentation. Two exteroceptive sensors are successively investigated in order to build the sensory memory: an RTK GPS sensor and a monocular camera. Then, nonlinear control laws, achieving an exact decoupling between lateral and longitudinal guidance functionalities is presented, so that path following and platoon control can be addressed separately. Inter-vehicles communication, relying on WiFi technology, is used in order to enable a global platoon control: each vehicle is aware on-line of the whole platoon configuration and not only of the preceding vehicle relative localization. Full scale experiments, carried out with Cycab vehicles, are finally reported, and demonstrate high accuracy vehicles control with any of the two exteroceptive sensors considered here

Jonathan Bom

Papers

Outdoor autonomous navigation using monocular vision

A global control strategy for urban vehicles platooning relying on nonlinear decoupling laws

Accurate automatic guidance of an urban electric vehicle relying on a kinematic GPS sensor

Nonlinear Control for Urban Vehicles Platooning, Relying upon a Unique Kinematic GPS

Autonomous Navigation and Platooning using a Sensory Memory