Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Introduction to stochastic programming

Consider a given dynamical system, described by ẋ = f(x) (wheref is a nonlinear function) and [x0] a subset ofR. We present an algorithm, based on interval analysis, able to show that there exists a unique equilibrium statex∞ ∈ [x0] which is asymptotically stable. The effective method also provides a set[x] (subset of[x0]) which is included in the attraction domain of x∞.

Ieee transactions on automatic control

Ieee transactions® on aerospace and electronic systems

Intelligent autonomous agents, designed to automate and simplify many aspects of our society, will increasingly be required to also interact with other agents autonomously. Where agents interact, they are likely to encounter resource constraints. For example, agents managing household appliances to optimize electricity usage might need to share the limited capacity of the distribution grid. This thesis describes research into new algorithms for optimizing the behavior of agents operating in constrained environments, when these agents have significant uncertainty about the effects of their actions on their state. Such systems are effectively modeled in a framework of constrained multi-agent Markov decision processes (MDPs). A single-agent MDP model captures the uncertainty in the outcome of the actions chosen by a specific agent. It does so by providing a probabilistic model of state transitions, describing the likelihood of arriving in a future state, conditional on the current state and action. Agents collect different rewards or penalties depending on the current state and chosen action, informing their objective of maximizing their expected reward. To include constraints, resource consumption functions are added to the actions, and the agents' (shared) objective is modified with a condition restricting their (cumulative) resource consumption. We propose novel algorithms to advance the state of the art in three challenging settings: computing static preallocations off-line, computing dynamic (re)allocations on-line, and optimally learning model dynamics through safe reinforcement learning under the constraints. Taken together, these algorithms show how agents can coordinate their actions under uncertainty and shared resource constraints in a broad range of conditions. Furthermore, the proposed solutions are complementary: static preallocations can be used as back-up strategy for when a communication disruption prevents the use of dynamic allocations.

/pdf/resource-constrained-multi-agent-markov-decision-processes-26r3nospx7.pdf

Resource-constrained Multi-agent Markov Decision Processes

Innovative methods have been developed for diagnosis, activity monitoring, and state estimation that achieve high accuracy through the use of stochastic models involving hybrid discrete and continuous behaviors. A key bottleneck is the automated acquisition of these hybrid models, and recent methods have focused predominantly on Jump Markov processes and piecewise autoregressive models. In this paper, we present a novel algorithm capable of performing unsupervised learning of guarded Probabilistic Hybrid Automata (PHA) models, which extends prior work by allowing stochastic discrete mode transitions in a hybrid system to have a functional dependence on its continuous state. Our experiments indicate that guarded PHA models can yield significant performance improvements when used by hybrid state estimators, particularly when diagnosing the true discrete mode of the system, without any noticeable impact on their real-time performance.

/pdf/learning-hybrid-models-with-guarded-transitions-3bher12e2s.pdf

Learning hybrid models with guarded transitions

MIT is developing an Earth-based hopper prototype for autonomous planetary exploration. This paper describes the initial development of the hopper prototype and its application as a testbed for lunar hopper techno logies. The prototype hopper consists of a gravity-canceling system, composed of battery-powered electric ducted fans, and an integrated lunar hopper system, which operates using cold gas propulsion. The gravity- canceling system is designed to allow the luna r hopper system to operate in an environment which is dynamically similar to the surface of the Moon, although it also allows for recovery and safe touchdown if the cold gas system experiences a failure. The environment provided by the electric ducted fans applies the fan thrust against Earth's gravity to obtain a desired upward acceleration, allowing the cold gas jets to operate as if they were in a fractional- gravity environment. The key design features of the integrated gravity-canceling and lunar hopping systems are presented in this paper.

Initial Development of an Earth-Based Prototype for a Lunar Hopper Autonomous Exploration System

A wide range of robotic missions contain actions that exhibit looping behavior. Examples of these actions include picking fruit in agriculture, pick-and-place tasks in manufacturing and search patterns in robotic search or survey missions. These looping actions often have a range of acceptable values for the number of loops and a preference function over them. For example, during robotic survey missions, the information gain is expected to increase with the number of loops in a search pattern. Since these looping actions also take time, which is typically bounded, there is a challenge of maximizing utility while respecting time constraints. In this paper, we introduce the Looping Temporal Problem with Preference (LTPP) as a simple parameterized extension of a simple temporal problem. In addition, we introduce a scheduling algorithm for LTPPs which leverages the structure of the problem to find the optimal solution efficiently. We show more than an order of magnitude improvement in run-time over current scheduling techniques and framing a LTPP as a MINLP.

A scheduler for actions with iterated durations

Risk-aware Planning in Hybrid Domains: An Application to Autonomous Planetary Rovers

There is an increasing desire for autonomous systems to have high levels of robustness and safety, attained through continuously planning and self-repairing online. Underlying this is the need to accurately estimate the system state and diagnose subtle failures. Estimation methods based on hybrid discrete and continuous state models have emerged as a method of precisely computing these estimates. However, existing methods have difficulty scaling to systems with more than a handful of components. Discrete, consistency based state estimation capabilities can scale to this level by combining best-first enumeration and conflict-directed search. While best-first methods have been developed for hybrid estimation, conflict-directed methods have thus far been elusive as conflicts learn inconsistencies from constraint violation, but probabilistic hybrid estimation is relatively unconstrained. In this paper we present an approach to hybrid estimation that unifies best-first enumeration and conflict-directed search through the concept of “bounding” conflicts, an extension of conflicts that represent tighter bounds on the cost of regions of the search space. This paper presents a general best-first enumeration algorithm based on bounding conflicts (A*BC) and a hybrid estimation method using this enumeration algorithm. Experiments show that an A*BC powered state estimator produces estimates up to an order of magnitude faster than the current state of the art, particularly on large systems.

/pdf/best-first-enumeration-based-on-bounding-conflicts-and-its-3iva8s61sk.pdf

Eric Timmons

Papers

Learning hybrid models with guarded transitions

Initial Development of an Earth-Based Prototype for a Lunar Hopper Autonomous Exploration System

A scheduler for actions with iterated durations

Risk-aware Planning in Hybrid Domains: An Application to Autonomous Planetary Rovers

Best-first Enumeration Based on Bounding Conflicts, and its Application to Large-scale Hybrid Estimation