There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

This paper gives an overview of ROS, an opensource robot operating system. ROS is not an operating system in the traditional sense of process management and scheduling; rather, it provides a structured communications layer above the host operating systems of a heterogenous compute cluster. In this paper, we discuss how ROS relates to existing robot software frameworks, and briefly overview some of the available application software which uses ROS.

ROS: an open-source Robot Operating System

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

https://web.cs.wpi.edu/~rail/files/ArgallEtAlRAS09.pdf

A survey of robot learning from demonstration

Autonomous helicopter flight is widely regarded to be a highly challenging control problem. This paper presents the first successful autonomous completion on a real RC helicopter of the following four aerobatic maneuvers: forward flip and sideways roll at low speed, tail-in funnel, and nose-in funnel. Our experimental results significantly extend the state of the art in autonomous helicopter flight. We used the following approach: First we had a pilot fly the helicopter to help us find a helicopter dynamics model and a reward (cost) function. Then we used a reinforcement learning (optimal control) algorithm to find a controller that is optimized for the resulting model and reward function. More specifically, we used differential dynamic programming (DDP), an extension of the linear quadratic regulator (LQR).

/pdf/an-application-of-reinforcement-learning-to-aerobatic-3shntu58t0.pdf

An Application of Reinforcement Learning to Aerobatic Helicopter Flight

Autonomous unmanned air vehicle ∞ight control systems require robust path generation to account for terrain obstructions, weather, and moving threats such as radar, jammers, and unfriendly aircraft. In this paper, we outline a feasible, hierarchal approach for real-time motion planning of small autonomous flxed-wing UAVs. The approach divides the trajectory generation into four tasks: waypoint path planning, dynamic trajectory smoothing, trajectory tracking, and low-level autopilot compensation. The waypoint path planner determines the vehicle’s route without regard for the dynamic constraints of the vehicle. This results in a signiflcant reduction in the path search space, enabling the generation of complicated paths that account for pop-up and dynamically moving threats. Kinematic constraints are satisfled using a trajectory smoother which has the same kinematic structure as the physical vehicle. The third step of the approach uses a novel tracking algorithm to generate a feasible state trajectory that can be followed by a standard autopilot. Monte-Carlo simulations were done to analyze the performance and feasibility of the approach and determine real-time computation requirements. A planar version of the algorithm has also been implemented and tested in a low-cost micro-controller. The paper describes a custom UAV built to test the algorithms.

/pdf/autonomous-vehicle-technologies-for-small-fixed-wing-uavs-1lr64kzlwl.pdf

Autonomous Vehicle Technologies for Small Fixed-Wing UAVs

Wilderness Search and Rescue (WiSAR) entails searching over large regions in often rugged remote areas. Because of the large regions and potentially limited mobility of ground searchers, WiSAR is an ideal application for using small (human-packable) unmanned aerial vehicles (UAVs) to provide aerial imagery of the search region. This paper presents a brief analysis of the WiSAR problem with emphasis on practical aspects of visual-based aerial search. As part of this analysis, we present and analyze a generalized contour search algorithm, and relate this search to existing coverage searches. Extending beyond laboratory analysis, lessons from field trials with search and rescue personnel indicated the immediate need to improve two aspects of UAV-enabled search: How video information is presented to searchers and how UAV technology is integrated into existing WiSAR teams. In response to the first need, three computer vision algorithms for improving video display presentation are compared; results indicate that constructing temporally localized image mosaics is more useful than stabilizing video imagery. In response to the second need, a goal-directed task analysis of the WiSAR domain was conducted and combined with field observations to identify operational paradigms and field tactics for coordinating the UAV operator, the payload operator, the mission manager, and ground searchers. © 2008 Wiley Periodicals, Inc.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/rob.20226

Supporting wilderness search and rescue using a camera‐equipped mini UAV

In the model-based policy search approach to reinforcement learning (RL), policies are found using a model (or "simulator") of the Markov decision process. However, for high-dimensional continuous-state tasks, it can be extremely difficult to build an accurate model, and thus often the algorithm returns a policy that works in simulation but not in real-life. The other extreme, model-free RL, tends to require infeasibly large numbers of real-life trials. In this paper, we present a hybrid algorithm that requires only an approximate model, and only a small number of real-life trials. The key idea is to successively "ground" the policy evaluations using real-life trials, but to rely on the approximate model to suggest local changes. Our theoretical results show that this algorithm achieves near-optimal performance in the real system, even when the model is only approximate. Empirical results also demonstrate that---when given only a crude model and a small number of real-life trials---our algorithm can obtain near-optimal performance in the real system.

/pdf/using-inaccurate-models-in-reinforcement-learning-3e8gqy2hrh.pdf

Morgan Quigley

Papers

ROS: an open-source Robot Operating System

An Application of Reinforcement Learning to Aerobatic Helicopter Flight

Autonomous Vehicle Technologies for Small Fixed-Wing UAVs

Supporting wilderness search and rescue using a camera‐equipped mini UAV

Using inaccurate models in reinforcement learning