Dynamic Bayesian Networks: Representation, Inference and Learning by Kevin Patrick Murphy Doctor of Philosophy in Computer Science University of California, Berkeley Professor Stuart Russell, Chair Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and bio-sequence analysis, and KFMs have been used for problems ranging from tracking planes and missiles to predicting the economy. However, HMMs and KFMs are limited in their “expressive power”. Dynamic Bayesian Networks (DBNs) generalize HMMs by allowing the state space to be represented in factored form, instead of as a single discrete random variable. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linear-Gaussian. In this thesis, I will discuss how to represent many different kinds of models as DBNs, how to perform exact and approximate inference in DBNs, and how to learn DBN models from sequential data. In particular, the main novel technical contributions of this thesis are as follows: a way of representing Hierarchical HMMs as DBNs, which enables inference to be done in O(T ) time instead of O(T ), where T is the length of the sequence; an exact smoothing algorithm that takes O(log T ) space instead of O(T ); a simple way of using the junction tree algorithm for online inference in DBNs; new complexity bounds on exact online inference in DBNs; a new deterministic approximate inference algorithm called factored frontier; an analysis of the relationship between the BK algorithm and loopy belief propagation; a way of applying Rao-Blackwellised particle filtering to DBNs in general, and the SLAM (simultaneous localization and mapping) problem in particular; a way of extending the structural EM algorithm to DBNs; and a variety of different applications of DBNs. However, perhaps the main value of the thesis is its catholic presentation of the field of sequential data modelling.

/pdf/dynamic-bayesian-networks-representation-inference-and-yy4fhfxdcj.pdf

Dynamic bayesian networks: representation, inference and learning

Human-Robot Interaction (HRI) has recently received considerable attention in the academic community, in labs, in technology companies, and through the media. Because of this attention, it is desirable to present a survey of HRI to serve as a tutorial to people outside the field and to promote discussion of a unified vision of HRI within the field. The goal of this review is to present a unified treatment of HRI-related problems, to identify key themes, and discuss challenge problems that are likely to shape the field in the near future. Although the review follows a survey structure, the goal of presenting a coherent "story" of HRI means that there are necessarily some well-written, intriguing, and influential papers that are not referenced. Instead of trying to survey every paper, we describe the HRI story from multiple perspectives with an eye toward identifying themes that cross applications. The survey attempts to include papers that represent a fair cross section of the universities, government efforts, industry labs, and countries that contribute to HRI, and a cross section of the disciplines that contribute to the field, such as human, factors, robotics, cognitive psychology, and design.

/pdf/human-robot-interaction-a-survey-4q8yi606a9.pdf

Human-Robot Interaction: A Survey

Probabilistic inference is the problem of estimating the hidden variables (states or parameters) of a system in an optimal and consistent fashion as a set of noisy or incomplete observations of the system becomes available online. The optimal solution to this problem is given by the recursive Bayesian estimation algorithm which recursively updates the posterior density of the system state as new observations arrive. This posterior density constitutes the complete solution to the probabilistic inference problem, and allows us to calculate any “optimal” estimate of the state. Unfortunately, for most real-world problems, the optimal Bayesian recursion is intractable and approximate solutions must be used. Within the space of approximate solutions, the extended Kalman filter (EKF) has become one of the most widely used algorithms with applications in state, parameter and dual estimation. Unfortunately, the EKF is based on a sub-optimal implementation of the recursive Bayesian estimation framework applied to Gaussian random variables. This can seriously affect the accuracy or even lead to divergence of any inference system that is based on the EKF or that uses the EKF as a component part. Recently a number of related novel, more accurate and theoretically better motivated algorithmic alternatives to the EKF have surfaced in the literature, with specific application to state estimation for automatic control. We have extended these algorithms, all based on derivativeless deterministic sampling based approximations of the relevant Gaussian statistics, to a family of algorithms called Sigma-Point Kalman Filters (SPKF). Furthermore, we successfully expanded the use of this group of algorithms (SPKFs) within the general field of probabilistic inference and machine learning, both as stand-alone filters and as subcomponents of more powerful sequential Monte Carlo methods (particle filters). We have consistently shown that there are large performance benefits to be gained by applying Sigma-Point Kalman filters to areas where EKFs have been used as the de facto standard in the past, as well as in new areas where the use of the EKF is impossible.

/pdf/sigma-point-kalman-filters-for-probabilistic-inference-in-1affrc62j6.pdf

Sigma-point kalman filters for probabilistic inference in dynamic state-space models

A review of smart homes-Present state and future challenges

An intelligent trial-and-error learning algorithm is presented that allows robots to adapt in minutes to compensate for a wide variety of types of damage. Autonomous mobile robots would be extremely useful in remote or hostile environments such as space, deep oceans or disaster areas. An outstanding challenge is to make such robots able to recover after damage. Jean-Baptiste Mouret and colleagues have developed a machine learning algorithm that enables damaged robots to quickly regain their ability to perform tasks. When they sustain damage — such as broken or even missing legs — the robots adopt an intelligent trial-and-error approach, trying out possible behaviours that they calculate to be potentially high-performing. After a handful of such experiments they discover, in less than two minutes, a compensatory behaviour that works in spite of the damage. Robots have transformed many industries, most notably manufacturing1, and have the power to deliver tremendous benefits to society, such as in search and rescue2, disaster response3, health care4 and transportation5. They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets6 to deep oceans7. A major obstacle to their widespread adoption in more complex environments outside factories is their fragility6,8. Whereas animals can quickly adapt to injuries, current robots cannot ‘think outside the box’ to find a compensatory behaviour when they are damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes9, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots6,8. A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage10,11, but current techniques are slow even with small, constrained search spaces12. Here we introduce an intelligent trial-and-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot’s prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury.

/pdf/robots-that-can-adapt-like-animals-21owdbvqr7.pdf

Robots that can adapt like animals

This paper describes an implemented robot system, which relies heavily on probabilistic AI techniques for acting under uncertainty. The robot Pearl and its predecessor Flo have been developed by a multi-disciplinary team of researchers over the past three years. The goal of this research is to investigate the feasibility of assisting elderly people with cognitive and physical activity limitations through interactive robotic devices, thereby improving their quality of life. The robot's task involves escorting people in an assisted living facility-a time-consuming task currently carried out by nurses. Its software architecture employs probabilistic techniques at virtually all levels of perception and decision making. During the course of experiments conducted in an assisted living facility, the robot successfully demonstrated that it could autonomously provide guidance for elderly residents. While previous experiments with fielded robot systems have provided evidence that probabilistic techniques work well in the context of navigation, we found the same to be true of human robot interaction with elderly people.

/pdf/experiences-with-a-mobile-robotic-guide-for-the-elderly-4bdhdz36kk.pdf

Experiences with a mobile robotic guide for the elderly

Autonomous planetary rovers operating in vast unknown environments must operate efficiently because of size, power and computing limitations. Recently, we have developed a rover capable of efficient obstacle avoidance and path planning. The rover uses binocular stereo vision to sense potentially cluttered outdoor environments. Navigation is performed by a combination of several modules that each "vote" for the next best action for the robot to execute. The key distinction of our system is that it produces globally intelligent behavior with a small computational resource - all processing and decision making are done on a single processor. These algorithms have been tested on our outdoor prototype rover, Bullwinkle, and have recently driven the rover 100 m at a speed of 15 cm/sec. In this paper we report on the extension on the systems that we have previously developed that were necessary to achieve autonomous navigation in this domain.

/pdf/recent-progress-in-local-and-global-traversability-for-2ucxr845po.pdf

Recent progress in local and global traversability for planetary rovers

This article presents a number of complementary algorithms for detecting faults on-board operating robots, where a fault is defined as a deviation from expected behavior. The algorithms focus on faults that cannot directly be detected from current sensor values but require inference from a sequence of time-varying sensor values. Each algorithm provides an independent improvement over the basic approach. These improvements are not mutually exclusive, and the algorithms may be combined to suit the application domain. All the approaches presented require dynamic models representing the behavior of each of the fault and operational states. These models can be built from analytical models of the robot dynamics, data from simulation, or from the real robot. All the approaches presented detect faults from a finite number of known fault conditions, although there may potentially be a very large number of these faults.

Real-time fault diagnosis [robot fault diagnosis]

Limitations on interplanetary communications create operations latencies and slow progress in planetary surface missions, with particular challenges to narrow-field-of-view science instruments requiring precise targeting. The AEGIS (Autonomous Exploration for Gathering Increased Science) autonomous targeting system has been in routine use on NASA's Curiosity Mars rover since May 2016, selecting targets for the ChemCam remote geochemical spectrometer instrument. AEGIS operates in two modes; in autonomous target selection, it identifies geological targets in images from the rover's navigation cameras, choosing for itself targets that match the parameters specified by mission scientists the most, and immediately measures them with ChemCam, without Earth in the loop. In autonomous pointing refinement, the system corrects small pointing errors on the order of a few milliradians in observations targeted by operators on Earth, allowing very small features to be observed reliably on the first attempt. AEGIS consistently recognizes and selects the geological materials requested of it, parsing and interpreting geological scenes in tens to hundreds of seconds with very limited computing resources. Performance in autonomously selecting the most desired target material over the last 2.5 kilometers of driving into previously unexplored terrain exceeds 93% (where ~24% is expected without intelligent targeting), and all observations resulted in a successful geochemical observation. The system has substantially reduced lost time on the mission and markedly increased the pace of data collection with ChemCam. AEGIS autonomy has rapidly been adopted as an exploration tool by the mission scientists and has influenced their strategy for exploring the rover's environment.

https://robotics.sciencemag.org/content/robotics/2/7/eaan4582.full-text.pdf

AEGIS autonomous targeting for ChemCam on Mars Science Laboratory: Deployment and results of initial science team use.

We propose a new particle filter that incorporates a model of costs when generating particles. The approach is motivated by the observation that the costs of accidentally not tracking hypotheses might be significant in some areas of state space, and next to irrelevant in others. By incorporating a cost model into particle filtering, states that are more critical to the system performance are more likely to be tracked. Automatic calculation of the cost model is implemented using an MDP value function calculation that estimates the value of tracking a particular state. Experiments in two mobile robot domains illustrate the appropriateness of the approach.

/pdf/risk-sensitive-particle-filters-35xbbj7dt4.pdf

Vandi Verma

Papers

Experiences with a mobile robotic guide for the elderly

Recent progress in local and global traversability for planetary rovers

Real-time fault diagnosis [robot fault diagnosis]

AEGIS autonomous targeting for ChemCam on Mars Science Laboratory: Deployment and results of initial science team use.

Risk Sensitive Particle Filters