Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

We present a system that estimates the motion of a stereo head or a single moving camera based on video input. The system operates in real-time with low delay and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates what we call visual odometry, i.e. motion estimates from visual input alone. No prior knowledge of the scene nor the motion is necessary. The visual odometry can also be used in conjunction with information from other sources such as GPS, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive and handheld platforms. We focus on results with an autonomous ground vehicle. We give examples of camera trajectories estimated purely from images over previously unseen distances and periods of time.

Visual odometry

A copy of Guangbo jiemu bao [Broadcast Program Report] was being passed from hand to hand among a group of young people eager to be the first to read the article introducing the program "What Is Revolutionary Love?" It said: "… Young friends, you are certainly very concerned about this problem'. So, we would like you to meet the young women workers Meng Xiaoyu and Meng Yamei and the older cadre Miss Feng. They are the three leading characters in the short story ‘The Place of Love.’ Through the description of the love lives of these three, the story induces us to think deeply about two questions that merit further examination.

This is How the Discussion Started

Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new goals, and (2) data inefficiency, i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows better generalization. To address the second issue, we propose the AI2-THOR framework, which provides an environment with high-quality 3D scenes and a physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment.

Target-driven visual navigation in indoor scenes using deep reinforcement learning

Visual odometry (VO) is the process of estimating the egomotion of an agent (e.g., vehicle, human, and robot) using only the input of a single or If multiple cameras attached to it. Application domains include robotics, wearable computing, augmented reality, and automotive. The term VO was coined in 2004 by Nister in his landmark paper. The term was chosen for its similarity to wheel odometry, which incrementally estimates the motion of a vehicle by integrating the number of turns of its wheels over time. Likewise, VO operates by incrementally estimating the pose of the vehicle through examination of the changes that motion induces on the images of its onboard cameras. For VO to work effectively, there should be sufficient illumination in the environment and a static scene with enough texture to allow apparent motion to be extracted. Furthermore, consecutive frames should be captured by ensuring that they have sufficient scene overlap.

Visual Odometry [Tutorial]

Building a spatially consistent model is a key functionality to endow a mobile robot with autonomy. Without an initial map or an absolute localization means, it requires to concurrently solve the localization and mapping problems. For this purpose, vision is a powerful sensor, because it provides data from which stable features can be extracted and matched as the robot moves. But it does not directly provide 3D information, which is a difficulty for estimating the geometry of the environment. This article presents two approaches to the SLAM problem using vision: one with stereovision, and one with monocular images. Both approaches rely on a robust interest point matching algorithm that works in very diverse environments. The stereovision based approach is a classic SLAM implementation, whereas the monocular approach introduces a new way to initialize landmarks. Both approaches are analyzed and compared with extensive experimental results, with a rover and a blimp.

/pdf/vision-based-slam-stereo-and-monocular-approaches-171crt5xdb.pdf

Vision-Based SLAM: Stereo and Monocular Approaches

Target detection and tracking encompasses a variety of decisional problems such as coverage, surveillance, search, patrolling, observing and pursuit-evasion along with others. These problems are studied by several communities, that tackle them using diverse formulations, hypotheses and approaches. This variety and the fact that target related robotics problems are pertinent for a large spectrum of applications has motivated a large amount of contributions, which have mostly been surveyed according to one or another viewpoint. In this article, our objective is to go beyond the frontiers of specific communities and specific problems, and to enlarge the scope of prior surveys. We define classes of missions and problems, and relate the results from various communities according to a unifying taxonomy. We review various work related to each class of problems identified in the taxonomy, highlighting the different approaches, models and results. Finally, we propose a transverse synthesis which analyses the approaches, models and lacks that are recurrent through all the tackled problems, and isolate the current main research directions.

/pdf/multi-robot-target-detection-and-tracking-taxonomy-and-3bl0jlwqk9.pdf

Multi-robot target detection and tracking: taxonomy and survey

This paper deals with the task allocation problem in multi-robot systems. We propose a completely distributed architecture, where robots dynamically allocate their tasks while they are building their plans. We first focus on the problem of simple "goto" tasks allocation: our approach involves an incremental task allocation algorithm based on the Contract-Net protocol. We introduce a parameter called equity coefficient in order to equilibrate the workload between the different robots and to control the triggering of the auction process. Then, we address the problem raised by temporal constraints between tasks by dynamically specifying temporary hierarchies among the tasks. Tests run in simulation quantify the benefits of our improvements.

/pdf/a-distributed-tasks-allocation-scheme-in-multi-uav-context-52ig495h8f.pdf

A distributed tasks allocation scheme in multi-UAV context

Autonomous long-range navigation in partially known planetary-like terrains is still an open challenge for robotics. Navigating hundreds of meters without any human intervention requires the robot to be able to build various representations of its environment, to plan and execute trajectories according to the kind of terrain traversed, to control its motions and to localize itself as it moves. All these activities have to be scheduled, triggered, controlled and interrupted according to the rover context. In this paper, we briefly review some functionalities that have been developed in our laboratory, and implemented on board the Marsokhod model robot, Lama. We then present how the various concurrent instances of the perception, localization and motion generation functionalities are integrated. Experimental results illustrate the functionalities throughout the paper.

/pdf/autonomous-rover-navigation-on-unknown-terrains-2c0asjgsfr.pdf

Autonomous Rover Navigation on Unknown Terrains

This paper presents a method to incorporate 3D line segments in vision based SLAM. A landmark initialization method that relies on the Plucker coordinates to represent a 3D line is introduced: a Gaussian sum approximates the feature initial state and is updated as new observations are gathered by the camera. Once initialized, the landmarks state is estimated along an EKF-based SLAM approach: constraints associated with the Plucker representation are considered during the update step of the Kalman filter. The whole SLAM algorithm is validated in simulation runs and results obtained with real data are presented.

/pdf/monocular-vision-based-slam-using-line-segments-2s3uqy7f0r.pdf

Simon Lacroix

Papers

Vision-Based SLAM: Stereo and Monocular Approaches

Multi-robot target detection and tracking: taxonomy and survey

A distributed tasks allocation scheme in multi-UAV context

Autonomous Rover Navigation on Unknown Terrains

Monocular-vision based SLAM using Line Segments