Showing papers by "Sebastian Thrun published in 2007"

Simultaneous Localization, Mapping and Moving Object Tracking

Abstract: Simultaneous localization, mapping and moving object tracking (SLAMMOT) involves both simultaneous localization and mapping (SLAM) in dynamic environments and detecting and tracking these dynamic objects. In this paper, a mathematical framework is established to integrate SLAM and moving object tracking. Two solutions are described: SLAM with generalized objects, and SLAM with detection and tracking of moving objects (DATMO). SLAM with generalized objects calculates a joint posterior over all generalized objects and the robot. Such an approach is similar to existing SLAM algorithms, but with additional structure to allow for motion modeling of generalized objects. Unfortunately, it is computationally demanding and generally infeasible. SLAM with DATMO decomposes the estimation problem into two separate estimators. By maintaining separate posteriors for stationary objects and moving objects, the resulting estimation problems are much lower dimensional than SLAM with generalized objects. Both SLAM and moving object tracking from a moving vehicle in crowded urban areas are daunting tasks. Based on the SLAM with DATMO framework, practical algorithms are proposed which deal with issues of perception modeling, data association, and moving object detection. The implementation of SLAM with DATMO was demonstrated using data collected from the CMU Navlab11 vehicle at high speeds in crowded urban environments. Ample experimental results shows the feasibility of the proposed theory and algorithms.

Simultaneous Localization and Mapping

Abstract: Dieses Kapitel gibt eine Einfuhrung in die Kartenerstellung und gleichzeitige Lokalisierung mobiler Sensorplattformen Die gemeinsame Losung dieser beiden Probleme ist eine Voraussetzung fur die Realisierung vieler technischer Systeme von leichten Fluggeraten uber autonome Roboter bis hin zu mobilen Kameras Als Simultaneous Localization and Mapping bezeichnet man die Aufgabe, die Trajektorie samt Orientierungsinformation einer sich bewegenden Plattform aus Beobachtungen zu schatzen und gleichzeitig eine Karte der Umgebung zu erstellen Diese Aufgabe ist in vielen realen Systemen von entscheidender Bedeutung: einerseits stellen hochgenaue Karten mitunter einen Wert an sich fur den Benutzer oder eine spezielle Anwendung dar, andererseits benotigen beispielsweise autonome Roboter ein solches Modell, um zielgerichtet selbststandig navigieren zu konnen Das Simultaneous Localization and Mapping Problem, beziehungsweise Teilprobleme davon, werden, je nach verwendeter Sensorik, auch als Bundelausgleichung, Structure from Motion oder SLAM bezeichnet

Map-Based Precision Vehicle Localization in Urban Environments

Abstract: Many urban navigation applications (e.g., autonomous navigation, driver assistance systems) can benefit greatly from localization with centimeter accuracy. Yet such accuracy cannot be achieved reliably with GPS-based inertial guidance systems, specifically in urban settings. We propose a technique for high-accuracy localization of moving vehicles that utilizes maps of urban environments. Our approach integrates GPS, IMU, wheel odometry, and LIDAR data acquired by an instrumented vehicle, to generate high-resolution environment maps. Offline relaxation techniques similar to recent SLAM methods [2, 10, 13, 14, 21, 30] are employed to bring the map into alignment at intersections and other regions of self-overlap. By reducing the final map to the flat road surface, imprints of other vehicles are removed. The result is a 2-D surface image of ground reflectivity in the infrared spectrum with 5cm pixel resolution. To localize a moving vehicle relative to these maps, we present a particle filter method for correlating LIDAR measurements with this map. As we show by experimentation, the resulting relative accuracies exceed that of conventional GPS-IMU-odometry-based methods by more than an order of magnitude. Specifically, we show that our algorithm is effective in urban environments, achieving reliable real-time localization with accuracy in the 10centimeter range. Experimental results are provided for localization in GPS-denied environments, during bad weather, and in dense traffic. The proposed approach has been used successfully for steering a car through narrow, dynamic urban roads.

Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing

Abstract: This paper presents a nonlinear control law for an automobile to autonomously track a trajectory, provided in real-time, on rapidly varying, off-road terrain. Existing methods can suffer from a lack of global stability, a lack of tracking accuracy, or a dependence on smooth road surfaces, any one of which could lead to the loss of the vehicle in autonomous off-road driving. This work treats automobile trajectory tracking in a new manner, by considering the orientation of the front wheels - not the vehicle's body - with respect to the desired trajectory, enabling collocated control of the system. A steering control law is designed using the kinematic equations of motion, for which global asymptotic stability is proven. This control law is then augmented to handle the dynamics of pneumatic tires and of the servo-actuated steering wheel. To control vehicle speed, the brake and throttle are actuated by a switching proportional integral (PI) controller. The complete control system consumes a negligible fraction of a computer's resources. It was implemented on a Volkswagen Touareg, "Stanley", the Stanford Racing Team's entry in the DARPA Grand Challenge 2005, a 132 mi autonomous off-road race. Experimental results from Stanley demonstrate the ability of the controller to track trajectories between obstacles, over steep and wavy terrain, through deep mud puddles, and along cliff edges, with a typical root mean square (RMS) crosstrack error of under 0.1 m. In the DARPA National Qualification Event 2005, Stanley was the only vehicle out of 40 competitors to not hit an obstacle or miss a gate, and in the DARPA Grand Challenge 2005 Stanley had the fastest course completion time.

FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics (Springer Tracts in Advanced Robotics)

Abstract: This monograph describes a new family of algorithms for the simultaneous localization and mapping (SLAM) problem in robotics, called FastSLAM. The FastSLAM-type algorithms have enabled robots to acquire maps of unprecedented size and accuracy, in a number of robot application domains and have been successfully applied in different dynamic environments, including a solution to the problem of people tracking.

FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics

Abstract: This monograph describes a new family of algorithms for the simultaneous localization and mapping (SLAM) problem in robotics, called FastSLAM. The FastSLAM-type algorithms have enabled robots to acquire maps of unprecedented size and accuracy, in a number of robot application domains and have been successfully applied in different dynamic environments, including a solution to the problem of people tracking.

Shape completion, animation and marker-less motion capture of people, animals or characters

Abstract: Motion capture animation, shape completion and markerless motion capture methods are provided. A pose deformation space model encoding variability in pose is learnt from a three-dimensional (3D) dataset. Body shape deformation space model encoding variability in pose and shape is learnt from another 3D dataset. The learnt pose model is combined with the learnt body shape model. For motion capture animation, given parameter set, the combined model generates a 3D shape surface of a body in a pose and shape. For shape completion, given partial surface of a body defined as 3D points, the combined model generates a 3D surface model in the combined spaces that fits the 3D points. For markerless motion capture, given 3D information of a body, the combined model traces the movement of the body using the combined spaces that fits the 3D information or reconstructing the body's shape or deformations that fits the 3D information.

Reverse Optical Flow for Self-Supervised Adaptive Autonomous Robot Navigation

Abstract: Autonomous mobile robot navigation, either off-road or on ill-structured roads, presents unique challenges for machine perception. A successful terrain or roadway classifier must be able to learn in a self-supervised manner and adapt to inter- and intra-run changes in the local environment. This paper demonstrates the improvements achieved by augmenting an existing self-supervised image segmentation procedure with an additional supervisory input. Obstacles and roads may differ in appearance at distance because of illumination and texture frequency properties. Reverse optical flow is added as an input to the image segmentation technique to find examples of a region of interest at previous times in the past. This provides representations of this region at multiple scales and allows the robot to better determine where more examples of this class appear in the image.

System and process for synthesizing location-referenced panoramic images and video

Abstract: A system and process of morphing location-referenced panoramic images into views at nearby locations. When using panoramic images in an interactive tour, a user might desire to see the environment from viewpoints for which no panoramic images are available. This invention makes this possible. It enables a user to experience views from arbitrary locations in the environment, so as long as one or more panoramic images are available at nearby locations. In particular, this invention makes it possible to combine two non-overlapping geo-referenced panoramic video streams into a new video stream which seamlessly transitions between these streams. When used in a client-server architecture, this invention also makes it possible for the server to transmit a sparse sequence of panoramic images, and provide the user with a dense panoramic video stream, by synthesizing the missing panoramic images. Said system and process is also applicable to incomplete panoramic images, photographs, and video.

System and method for tagging objects in a panoramic video and associating functions and indexing panoramic images with same

Abstract: A system and process for attaching tags to panoramic video. Tags provide information when viewing panoramic images, serve as references to specific actions and serve as reference for outside systems into a panoramic image database. Objects in a video can be tagged. It defines tags through 4-D time-space curves, which specify the 3-D location of a tagged object over time. It provides a user-friendly mechanism for defining said curves in panoramic video, which rely on curve fitting techniques to minimize the number of images to be labeled. It provides a mechanism for annotating tags with further information. When displaying tagged panoramic video, tags are graphically superimposed on the panoramic video feed using projective projection techniques. From this visualization, a user can then select a given tag and invoke an associated action. Additionally a mechanism whereby tags and associated user-provided information are used as index into panoramic image databases is provided.

Recognizing activities with multiple cues

Abstract: In this paper, we introduce a first-order probabilistic model that combines multiple cues to classify human activities from video data accurately and robustly. Our system works in a realistic office setting with background clutter, natural illumination, different people, and partial occlusion. The model we present is compact, requires only fifteen sentences of first-order logic grouped as a Dynamic Markov Logic Network (DMLNs) to implement the probabilistic model and leverages existing state-of-the-art work in pose detection and object recognition.

Image mosaicing systems and methods

Abstract: Mosaicing methods and devices are implementing in a variety of manners. One such method is implemented for generation of a continuous image representation of an area from multiple images consecutively received from an image sensor. A location of a currently received image is indicated relative to the image sensor. A position of a currently received image relative to a set of previously received images is indicated with reference to the indicated location. The currently received image is compared to the set of previously received images as a function of the indicated position. Responsive to the comparison, adjustment information is indicated relative to the indicated position. The currently received image is merged with the set of previously received images to generate data representing a new set of images.

Online speed adaptation using supervised learning for high-speed, off-road autonomous driving

Abstract: The mobile robotics community has traditionally addressed motion planning and navigation in terms of steering decisions However, selecting the best speed is also important - beyond its relationship to stopping distance and lateral maneuverability Consider a high-speed (35 mph) autonomous vehicle driving off-road through challenging desert terrain The vehicle should drive slowly on terrain that poses substantial risk However, it should not dawdle on safe terrain In this paper we address one aspect of risk - shock to the vehicle We present an algorithm for trading-off shock and speed in realtime and without human intervention The trade-off is optimized using supervised learning to match human driving The learning process is essential due to the discontinuous and spatially correlated nature of the control problem - classical techniques do not directly apply We evaluate performance over hundreds of miles of autonomous driving, including performance during the 2005 DARPA Grand Challenge This approach was the deciding factor in our vehicle's speed for nearly 20% of the DARPA competition - more than any other constraint except the DARPA-imposed speed limits - and resulted in the fastest finishing time

Why we compete in DARPA's Urban Challenge autonomous robot race

Abstract: The robots will have to find their own way, sensing and predicting their whereabouts through automated perception, planning, and control.

Autonomous Helicopter Tracking and Localization Using a Self-surveying Camera Array

Abstract: A Self-surveying Camera Array (SSCA) is a vision-based local-area positioning system consisting of multiple ground-deployed cameras that are capable of self-surveying their extrinsic parameters while tracking and localizing a moving target. This paper presents the self-surveying algorithm being used to track a target helicopter in each camera frame and to localize the helicopter in an array-fixed frame. Three cameras are deployed independently in an arbitrary arrangement that allows each camera to view the helicopter's flight volume. The helicopter then flies an unplanned path that allows the cameras to calibrate the relative locations and orientations by utilizing a self-surveying algorithm that is extended from the well-known structure from motion algorithm and the bundle adjustment technique. This yields the cameras'extrinsic parameters enabling real-time helicopter positioning via triangulation. This paper also presents results from field trials, which verify the feasibility of the SSCA as a readily-deployable system applicable to helicopter tracking and localization. The results demonstrate that, compared to the differential GPS solution as true reference, the SSCA alone is capable of positioning the helicopter with meter-level accuracy. The SSCA has been integrated with onboard inertial sensors providing a reliable positioning system to enable successful autonomous hovering.

Deformable Image Mosaicing for Optical Biopsy

Abstract: Traditional image mosaicing usually relies on rigid image transformations. In many medical applications, however, tissue deformation during image acquisition or 3D parallax effects may require nonrigid transformations in the mosaicing process. This paper presents a new method that integrates deformable surface models into the image mosaicing algorithms. Our approach has two main contributions. First, we present a global alignment algorithm to efficiently deal with accumulated image registration errors. Second, we introduce a local alignment algorithm to accommodate local scene deformations. These two problems are integrated into a single optimization problem that simultaneously recovers the motion of the camera as well as the structure of the scene. Our approach is demonstrated on simulations, images from a hand-held digital camera, and microscopic images acquired with a micro-endoscope.

Optical Flow Approaches for Self-supervised Learning in Autonomous Mobile Robot Navigation

Abstract: A common theme in autonomous mobile robotics is the desire to sense farther ahead of the robot than current approaches allow This greater range would enable earlier recognition of hazards, better path planning, and higher speeds In scenarios where the long range sensor modality is computer vision this has led to interest in developing techniques that can effectively identify and respond to obstacles at greater distances than those for which stereo vision methods are useful This paper presents work on optical flow techniques that leverage the difference in appearance between objects at close range and the same objects at more distant locations in order to interpret monocular video streams in a useful manner In particular, two applications are discussed: self-supervised off-road autonomous navigation, and adaptive road following in unstructured environments Examples of the utility of the optical flow techniques discussed here in both arenas are provided

Robotics research : results of the 12th International Symposium ISRR

Abstract: Physical Human Robot Interaction and Haptics.- Session Overview Physical Human-Robot Integration and Haptics.- A Unified Passivity Based Control Framework for Position, Torque and Impedance Control of Flexible Joint Robots.- Wave Haptics: Encoderless Virtual Stiffnesses.- Reality-Based Estimation of Needle and Soft-Tissue Interaction for Accurate Haptic Feedback in Prostate Brachytherapy Simulation.- Haptic Virtual Fixtures for Robot-Assisted Manipulation.- Planning.- Session Overview Planning.- POMDP Planning for Robust Robot Control.- On the Probabilistic Foundations of Probabilistic Roadmap Planning.- Humanoids.- Session Overview Humanoids.- Humanoid HRP2-DHRC for Autonomous and Interactive Behavior.- Android Science.- Mimetic Communication Theory for Humanoid Robots Interacting with Humans.- Mechanism and Design.- Session Overview Mechanisms and Design.- Design of a Compact 6-DOF Haptic Device to Use Parallel Mechanisms.- Hybrid Nanorobotic Approaches to NEMS.- Jacobian, Manipulability, Condition Number and Accuracy of Parallel Robots.- SLAM.- Session Overview Simultaneous Localisation and Mapping.- Subjective Localization with Action Respecting Embedding.- D-SLAM: Decoupled Localization and Mapping for Autonomous Robots.- A Provably Consistent Method for Imposing Sparsity in Feature-Based SLAM Information Filters.- Field Robots.- Session Overview Field Robotics.- Field D*: An Interpolation-Based Path Planner and Replanner.- Tradeoffs Between Directed and Autonomous Driving on the Mars Exploration Rovers.- Surface Mining: Main Research Issues for Autonomous Operations.- Robotic Vision.- Session Overview Robotic Vision.- Bias Reduction and Filter Convergence for Long Range Stereo.- Fusion of Stereo, Colour and Contrast.- Automatic Single-Image 3d Reconstructions of Indoor Manhattan World Scenes.- Robot Design and Control.- Session Overview Robot Design and Control.- One Is Enough!.- A Steerable, Untethered, 250 x 60 m MEMS Mobile Micro-Robot.- Some Issues in Humanoid Robot Design.- That Which Does Not Stabilize, Will Only Make Us Stronger.- Underwater Robotics.- Session Overview Underwater Robotics.- Improved Estimation of Target Velocity Using Multiple Model Estimation and a Dynamic Bayesian Network for a Robotic Tracker of Ocean Animals.- Techniques for Deep Sea Near Bottom Survey Using an Autonomous Underwater Vehicle.- Advances in High Resolution Imaging from Underwater Vehicles.- Learning and Adaptive Behavior.- Session Overview Learning and Adaptive Behavior.- Using AdaBoost for Place Labeling and Topological Map Building.- Emergence, Exploration and Learning of Embodied Behavior.- Hierarchical Conditional Random Fields for GPS-Based Activity Recognition.- Networked Robotics.- Session Overview Networked Robotics.- Networked Robotic Cameras for Collaborative Observation of Natural Environments.- Interfaces and Interaction.- Session Overview Interfaces and Interaction.- Haptic Communication Between Humans and Robots.- A Vestibular Interface for Natural Control of Steering in the Locomotion of Robotic Artifacts: Preliminary Experiments.- How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism.- Invited Overview Talk.- Expo 2005 Robotics Project.- Robotics Science (Panel Discussion).- Position Statement: Robotics Science.

System and process for projecting location-referenced panoramic images into a 3-D environment model and rendering panoramic images from arbitrary viewpoints within the 3-D environment model

Abstract: A system and process of morphing location-referenced panoramic images into views at nearby locations. When using panoramic images in an interactive tour, a user might desire to see the environment from viewpoints for which no panoramic images are available. This invention makes this possible. It enables a user to experience views from arbitrary locations in the environment, so as long as one or more panoramic images are available at nearby locations. In particular, this invention makes it possible to combine two non-overlapping geo-referenced panoramic video streams into a new video stream which seamlessly transitions between these streams. When used in a client-server architecture, this invention also makes it possible for the server to transmit a sparse sequence of panoramic images, and provide the user with a dense panoramic video stream, by synthesizing the missing panoramic images. Said system and process is also applicable to incomplete panoramic images, photographs, and video.