Marcus Johnson

Bio: Marcus Johnson is an academic researcher from Ames Research Center. The author has contributed to research in topics: Adaptive control & Lyapunov function. The author has an hindex of 15, co-authored 47 publications receiving 1246 citations. Previous affiliations of Marcus Johnson include University of Florida.

Unmanned Aircraft System Traffic Management (UTM) Concept of Operations

Abstract: Many applications of small Unmanned Aircraft System (sUAS) have been envisioned. These include surveillance of key assets such as pipelines, rail, or electric wires, deliveries, search and rescue, traffic monitoring, videography, and precision agriculture. These operations are likely to occur in the same airspace in presence of many static and dynamic constraints such as airports, and high wind areas. Therefore, small UAS, typically 55 pounds and below, operations need to be managed to ensure safety and efficiency of operations is maintained. This paper will describe the Concept of Operations (ConOps) for NASA's UAS Traffic Management (UTM) research initiative. The UTM ConOps is focused on safely enabling large-scale small UAS (sUAS) operations in low altitude airspace. The UTM construct supports large-scale visual line of sight and beyond visual line of sight operations. It is based on two primary mantras: (1) flexibility where possible and structure where necessary (2) a risk-based approach where geographical needs and use case indicate the airspace performance requirements. Preliminary stakeholder feedback and initial UTM tests conducted by NASA show promise of UTM to enable large-scale low altitude UAS operations safely.

Closed-Loop Neural Network-Based NMES Control for Human Limb Tracking

Abstract: Closed-loop control of skeletal muscle is complicated by the nonlinear muscle force to length and velocity relationships and the inherent unstructured and time-varying uncertainties in available models. Some pure feedback methods have been developed with some success, but the most promising and popular control methods for neuromuscular electrical stimulation (NMES) are neural network (NN)-based methods. Efforts in this paper focus on the use of a NN feedforward controller that is augmented with a continuous robust feedback term to yield an asymptotic result (in lieu of typical uniformly ultimately bounded stability). Specifically, an NN-based controller and Lyapunov-based stability analysis are provided to enable semi-global asymptotic tracking of a desired limb time-varying trajectory (i.e., non-isometric contractions). The developed controller is applied as an amplitude modulated voltage to external electrodes attached to the distal-medial and proximal-lateral portion of the quadriceps femoris muscle group in non-impaired volunteers. The added value of incorporating a NN feedforward term is illustrated through experiments that compare the developed controller with and without the NN feedforward component.

SET-Valued Analysis

Abstract: “Multivalued Analysis” is the theory of set-valued maps (called multifonctions) and has important applications in many different areas. Multivalued analysis is a remarkable mixture of many different parts of mathematics such as point-set topology, measure theory and nonlinear functional analysis. It is also closely related to “Nonsmooth Analysis” (Chapter 5) and in fact one of the main motivations behind the development of the theory, was in order to provide necessary analytical tools for the study of problems in nonsmooth analysis. It is not a coincidence that the development of the two fields coincide chronologically and follow parallel paths. Today multivalued analysis is a mature mathematical field with its own methods, techniques and applications that range from social and economic sciences to biological sciences and engineering. There is no doubt that a modern treatise on “Nonlinear Functional Analysis” can not afford the luxury of ignoring multivalued analysis. The omission of the theory of multifunctions will drastically limit the possible applications.

Optimal and Autonomous Control Using Reinforcement Learning: A Survey

Abstract: This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal $\mathcal {H}_{2}$ and $\mathcal {H}_\infty $ control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.

Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems

Abstract: This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming (ADP) method for solving the infinite horizon optimal control problem of nonlinear systems. The idea is to use an iterative ADP technique to obtain the iterative control law, which optimizes the iterative performance index function. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time. It shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation. It is also proven that any of the iterative control laws can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control law, respectively, for facilitating the implementation of the iterative ADP algorithm, where the convergence of the weight matrices is analyzed. Finally, the numerical results and analysis are presented to illustrate the performance of the developed method.