This paper presents control and coordination algorithms for groups of vehicles. The focus is on autonomous vehicle networks performing distributed sensing tasks where each vehicle plays the role of a mobile tunable sensor. The paper proposes gradient descent algorithms for a class of utility functions which encode optimal coverage and sensing policies. The resulting closed-loop behavior is adaptive, distributed, asynchronous, and verifiably correct.

Coverage control for mobile sensing networks

We present a reinforcement learning based framework for human-centered collaborative systems. The framework is proactive and balances the benefits of timely actions with the risk of taking improper actions by minimizing the total time spent to complete the task. The framework is learned end-to-end in an unsupervised fashion addressing the perception uncertainties and decision making in an integrated manner. The framework is shown to provide more time-efficient coordination between human and robot partners on an example task of packaging compared to alternatives for which perception and decision-making systems are learned independently, using supervised learning. Two important benefits of the proposed approach are that tedious annotation of motion data is avoided, and the learning is performed on-line.

Human-Centered Collaborative Robots With Deep Reinforcement Learning

The expected adoption of robots able to interact with people in real-world scenarios brings the need to move from easy-to-use to easy-to-develop robot architectures. In this paper, we discuss how a component-based methodology, a block and web-based interface, and a behavior tree approach to the development of robot behaviors can be used together to enable the adoption of the end-user development (EUD) paradigm. The proposed framework (i) mainly uses easy-to-install, cross-platform and modular tools, (ii) is ROS, ZeroMQ and nanomsg compatible, (iii) enables the creation of platform-independent applications, and (iv) can be easily expanded with new sensory devices or robots. In order to evaluate the proposed software framework, an example of a human-robot interaction application using a NAO robot is presented and discussed.

Development of Intelligent Behaviors for Social Robots via User-Friendly and Modular Programming Tools

There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures. While there is broad consensus on how BTs work, some characteristics rely on the implementation choices done in the specific software library used. In this letter, we outline practical aspects in the adoption of BTs and the solutions devised by the robotics community to fully exploit the advantages of BTs in real robots. We also overview the solutions proposed in open-source libraries used in robotics, we show how BTs fit in a robotic software architecture, and we present a use case example.

On the Implementation of Behavior Trees in Robotics

We present a reinforcement learning based framework for human-centered collaborative systems. The framework is proactive and balances the benefits of timely actions with the risk of taking improper actions by minimizing the total time spent to complete the task. The framework is learned end-to-end in an unsupervised fashion addressing the perception uncertainties and decision making in an integrated manner. The framework is shown to provide more fluent coordination between human and robot partners on an example task of packaging compared to alternatives for which perception and decision-making systems are learned independently, using supervised learning. The foremost benefit of the proposed approach is that it allows for fast adaptation to new human partners and tasks since tedious annotation of motion data is avoided and the learning is performed on-line.

Human-centered collaborative robots with deep reinforcement learning

Behavior Trees (BTs) were invented as a tool to enable modular AI in computer games, but have received an increasing amount of attention in the robotics community in the last decade. With rising demands on agent AI complexity, game programmers found that the Finite State Machines (FSM) that they used scaled poorly and were difficult to extend, adapt and reuse. In BTs, the state transition logic is not dispersed across the individual states, but organized in a hierarchical tree structure, with the states as leaves. This has a significant effect on modularity, which in turn simplifies both synthesis and analysis by humans and algorithms alike. These advantages are needed not only in game AI design, but also in robotics, as is evident from the research being done. In this paper we present a comprehensive survey of the topic of BTs in Artificial Intelligence and Robotic applications. The existing literature is described and categorized based on methods, application areas and contributions, and the paper is concluded with a list of open research challenges.

A Survey of Behavior Trees in Robotics and AI.

Air Combat is a high-risk activity carried out by trained professionals operating sophisticated equipment. During this activity, a number of trade-offs have to be made, such as the balance between risk and efficiency. A policy that minimizes risk could have very low efficiency, and one that maximizes efficiency may involve very high risk.In this study, we use Reinforcement Learning (RL) to create Control Barrier Functions (CBF) that captures the current risk, in terms of worst-case future separation between the aircraft and an enemy missile. CBFs are usually designed manually as closed-form expressions, but for a complex system such as a guided missile, this is not possible. Instead, we solve an RL problem using high fidelity simulation models to find value functions with CBF properties, that can then be used to guarantee safety in real air combat situations. We also provide a theoretical analysis of what family of RL problems result in value functions that can be used as CBFs in this way.The proposed approach allows the pilot in an air combat scenario to set the exposure level deemed acceptable and continuously monitor the risk related to his/her own safety. Given input regarding acceptable risk, the system limits the choices of the pilot to those that guarantee future satisfaction of the provided bound.

/pdf/using-reinforcement-learning-to-create-control-barrier-4ffa6xeflh.pdf

Using Reinforcement Learning to Create Control Barrier Functions for Explicit Risk Mitigation in Adversarial Environments

Sorption fixed-bed column experiments were performed using a titanium phosphate ion-exchanger composed of −H2PO4 units [TiO(OH)(H2PO4)·H2O]. Model mine water containing five divalent metal ions (Cu...

Competitive Sorption of Metal Ions on Titanium Phosphate Sorbent (TiP1) in Fixed-Bed Columns: A Closed-Mine Waters Study

Military aircraft pilots need to adjust to a constantly changing battlefield. A system that aids in understanding challenging combat circumstances and suggests appropriate responses can considerably improve the effectiveness of pilots. In this paper, we provide a Reinforcement Learning (RL) based system that acts as an aid in determining if an afterburner should be turned on to escape an incoming air-to-air missile. An afterburner is a component of a jet engine that increases thrust at the expense of exceptionally high fuel consumption. Thus it provides a short-term advantage, at the cost of a longterm disadvantage, in terms of reduced mission time. Helping to choose when to use the afterburner may significantly lengthen the flight duration, allowing aircraft to support friendly aircraft for longer and suffer fewer friendly fatalities due to this extended ability to provide support. We propose an RL-based risk estimation approach to help determine whether additional thrust is required to escape an incoming missile and study the benefits of thrust-aided evasive maneuvers. The suggested technique gives pilots a risk estimate for the scenario and a recommended course of action. We create an environment in which a pilot must decide whether or not to activate additional thrust to achieve the intended aim at a potentially high fuel consumption cost. Additionally, we investigate various trade-offs of the generated evasive maneuver policies.

Enhancing Situation Awareness in Beyond Visual Range Air Combat with Reinforcement Learning-based Decision Support

Air combat is a high-risk activity where pilots must be aware of the surrounding situation to outperform the opposing team. The chances of beating the opposing team improve when the pilots have superior situation awareness, thus allowing them to act before the opposing team can do counteractions. In a highly dynamic environment, such as air combat, it can be difficult for pilots to track all adversarial units and their capabilities. In this work, we propose a combination of Monte Carlo Tree Search (MCTS) and Convex optimization to help pilots analyze the situation and be aware of any potential risks associated with missile guidance in Beyond Visual Range air combat. Our process uses MCTS to assess the best action from an opposing aircraft perspective. At the same time, the convex optimization problem searches available aircraft trajectories that enable missile guidance in relation to the opponent’s potential actions. The proposed system is intended to support human decisions made by a pilot inside the aircraft or by a remote pilot operating an unmanned aerial system (UAS).

Edvards Scukins

Papers

A Survey of Behavior Trees in Robotics and AI.

Using Reinforcement Learning to Create Control Barrier Functions for Explicit Risk Mitigation in Adversarial Environments

Competitive Sorption of Metal Ions on Titanium Phosphate Sorbent (TiP1) in Fixed-Bed Columns: A Closed-Mine Waters Study

Enhancing Situation Awareness in Beyond Visual Range Air Combat with Reinforcement Learning-based Decision Support

Monte Carlo Tree Search and Convex Optimization for Decision Support in Beyond-Visual-Range Air Combat