Showing papers in "IEEE Control Systems Magazine in 2012"

Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers

Abstract: This article describes the use of principles of reinforcement learning to design feedback controllers for discrete- and continuous-time dynamical systems that combine features of adaptive control and optimal control. Adaptive control [1], [2] and optimal control [3] represent different philosophies for designing feedback controllers. Optimal controllers are normally designed of ine by solving Hamilton JacobiBellman (HJB) equations, for example, the Riccati equation, using complete knowledge of the system dynamics. Determining optimal control policies for nonlinear systems requires the offline solution of nonlinear HJB equations, which are often difficult or impossible to solve. By contrast, adaptive controllers learn online to control unknown systems using data measured in real time along the system trajectories. Adaptive controllers are not usually designed to be optimal in the sense of minimizing user-prescribed performance functions. Indirect adaptive controllers use system identification techniques to first identify the system parameters and then use the obtained model to solve optimal design equations [1]. Adaptive controllers may satisfy certain inverse optimality conditions [4].

Predictive Control for Energy Efficient Buildings with Thermal Storage: Modeling, Stimulation, and Experiments

Abstract: The building sector is the largest energy consumer in the world. Therefore, it is economically, socially, and environmentally significant to reduce the energy consumption of buildings. Achieving substantial energy reduction in buildings may require rethinking the whole processes of design, construction, and operation of a building. This article focuses on the specific issue of advanced control system design for energy efficient buildings.

Iterative Learning Control in Health Care: Electrical Stimulation and Robotic-Assisted Upper-Limb Stroke Rehabilitation

Abstract: Annually, 15 million people worldwide suffer a stroke, and 5 million are left permanently disabled. A stroke is usually caused when a blood clot blocks a vessel in the brain and acts like a dam, stopping the blood reaching the regions downstream. Alternatively, it may be caused by a hemorrhage, in which a vessel ruptures and leaks blood into surrounding areas. As a result, some of the connecting nerve cells die, and the person commonly suffers partial paralysis on one side of the body, termed hemiplegia. Cells killed in this way cannot regrow, but the brain has some spare capacity and, hence, new connections can be made. The brain is continually and rapidly changing as new skills are learned, new connections are formed, and redundant ones disappear. A person who relearns skills after a stroke goes through the same process as someone learning to play tennis or a baby learning to walk, requiring sensory feedback during the repeated practice of a task. Unfortunately, the problem is that they can hardly move and, therefore, do not receive feedback on their performance.

Interacting with Networks: How Does Structure Relate to Controllability in Single-Leader, Consensus Networks?

Abstract: As networked dynamical systems appear around us at an increasing rate, questions concerning how to manage and control such systems are becoming more important. Examples include multiagent robotics, distributed sensor networks, interconnected manufacturing chains, and data networks. In response to this growth, a significant body of work has emerged focusing on how to organize such networks to facilitate their control and make them amenable to human interactions. In this article, we summarize these activities by connecting the network topology, that is, the layout of the interconnections in the network, to the classic notion of controllability.

Towards Control of Magnetic Fluids in Patients: Directing Therapeutic Nanoparticles to Disease Locations

Abstract: This article describes a range of results, from passive magnet design to optimal feedback control of a distributed ferrofluid. Representing magnetic forces as the gradient of a magnetic energy allowed design and demonstration, in animal experiments, of a simple open-loop two-magnet system to inject nanoparticles into the inner ear. Then a petri-dish test bed was used to develop and test optimal closed-loop manipulation of a single ferrofluid droplet.The demonstrated algorithms exploited the quadratic dependence of the magnetic forces on control inputs while accounting for magnet time delays and spatial discontinuities in the optimal control. All dimensional parameters were reduced to three essential nondimensional numbers, and ferrofluid behavior was mapped across the entire feasible drug delivery parameter space.

Time-Critical Cooperative Control of Multiple Autonomous Vehicles: Robust Distributed Strategies for Path-Following Control and Time-Coordination over Dynamic Communications Networks

Abstract: Worldwide, there has been growing interest in the use of autonomous vehicles to execute missions of increasing complexity without constant supervision of human operators. A key enabling element for the execution of such missions is the availability of advanced systems for motion control of autonomous vehicles. Usually, the problems of motion control for a single autonomous vehicle are roughly classified into three groups:.

Control of Aerial Robots: Hybrid Force and Position Feedback for a Ducted Fan

Abstract: This article has dealt with the modeling and control of a class of aerial robots capable of interacting with the environment to accomplish robotic operations midair rather than being constrained to the ground. Discussed in detail are the design of hybrid force and position control laws for a ducted fan aeriaI vehicle. Particular attention has been placed on keeping track of how the stability properties of the system's zero dynamics are affected by the position of the vehicle's center of gravity, which can be mechanically designed to infer desired stability properties. The described control laws are state feedback and rely upon partial feedback linearizing techniques. In this respect, future extensions concern the development of output feedback control strategies, whereby the nonminimum-phase behavior of the system imposes fundamental limitations to the achievable tracking performance regardless of the kind of control strategy adopted. The present article fits in a broader research context in which aerial vehicles are considered to he a support to human beings in all those activities that require the ability to interact safely with airborne environments. In this context, future research attempts are directed to develop teleoperation algorithms, according to which x human operator can remotely supervise the motion of the CAV by means of haptic devices. Experimental activities in the above interaction scenarios are also planned in the near future.

Flow Control of Small Objects on Chip: Manipulating Live Cells, Quantum Dots, and Nanowires

Abstract: This article is on microscale flow control, on dynamically shaping flow fields in microfluidic devices to precisely manipulate cells, quantum dots (QDs), and nanowires. Compared to prior methods, manipulating microscopic and nanoscopic objects by flow control can be achieved with simpler and easy-to-fabricate devices, can steer a wider variety of objects, and enables entirely new capabilities such as placement and immobilization of specific quantum dots to desired on-chip locations with nanoscale precision. A companion article investigates flow control in the body and develops methods to shape magnetic fields to direct ferrofluids of therapeutic magnetic nanoparticles to disease locations in patients.

The Balancing Cube: A Dynamic Sculpture As Test Bed for Distributed Estimation and Control

Abstract: The balancing cube is a dynamic sculpture that can balance autonomously on any of its edges or corners (see Figures 14). When standing on a corner, the cube represents a three-dimensional (3-D) inverted pendulum with multiple actuation, sensing, and control units that are interconnected over a communication network. The main structural components are the cube body (a rigid aluminum structure with a cubic shape) and six identical rotating arms located on each of the cube's inner faces. The rotating arms are self-contained units carrying sensors, actuation, a computer, and a battery. Due to their modular design, these units are referred to as modules. As they rotate, they shift the overall center of mass of the system, exert forces on the cube structure, and can, as a result, influence the cube's motion. The modules constitute the agents in the distributed and networked control system; their joint objective is the stabilization of the cube. A video of the cube can be found on the project Web site [1].

Tracking and Data Fusion: A Handbook of Algorithms (Bar-Shalom, Y. et al; 2011) [Bookshelf]

Abstract: This book covers one of the most important applications of estimation theory - multiple object tracking or multi-target tracking. It contains 16 chapters and an extensive bibliography. Additions to the 1995 version of this book include a more thorough treatment of multisensor fusion and multiple hypothesis tracking, attribute-aide tracking, tracking with imaging sensors, unresolved targets, sensor management, and radar biases.

Autopilots for Ultra Lightweight Robotic Birds: Automatic Altitude Control and System Integration of a Sub-10 g Weight Flapping-Wing Micro Air Vehicle

Abstract: This article described a control law for stabilizing the vertical motion of a flapping-wing MAV and developed a system architecture that is potentially beneficial in realizing the autonomous flight of flapping-wing MAVs fewer than 10 g. The article began with a brief introduction to the Golden Snitch, including its development history and the development of the overall system. The vertical dynamics were given for altitude control. The use of wind tunnel tests to obtain aerodynamical parameters was described. Due to the limited payload-carrying capability, the control architecture was modified so that automatic control of flight altitude of a flapping-wing MAV fewer than 10 g is possible using current technology. Taking the hardware constraint into account, it was shown that the modified P-control can stabilize the vertical motion and track altitude commands. Numerical simulations and flight tests were presented that demonstrate the function of the developed control law and the system architecture. We believe this flapping-wing MAV to be the first under 10 g able to automatically maintain its flight altitude.

Distributed Linear-Quadratic Control of Serially Chained Systems: Application to a Water Delivery Canal [Applications of Control]

Abstract: Networks are prevalent in many human activities such as communication, transport, energy, and water delivery and drainage. Large networks that spread over wide-space areas cross the borders of different administrative regions, or even countries, and are managed by regional decision centers that may have to comply with different legal requirements. With networking comes complexity and the need to coordinate local decisions, which is an issue for which control and optimization can provide significant contributions. The sources of complexity are many, including the heterogeneous nature of some information patterns and the need to perform control over communication networks that may fail or have varying delays. This article does not attempt to address all of these issues. Instead, the attention is concentrated on how to design distributed controllers for plants that can be modeled by the serial interconnection of subsystems.

Autonomous Control of Unmanned Combat Air Vehicles: Design of a Multimodal Control and Flight Planning Framework for Agile Maneuvering

Abstract: There is a growing demand for unmanned air vehicles (UAVs) with combat capabilities in battlefield scenarios [1]. Whether this capability is for evasive maneuvers or for flying attack patterns, unmanned combat air vehicles (UCAVs) are expected to operate in dense and often threatening environments that require aggressive trajectory planning and controls [1]. These trajectories often require the use of maneuvering capability over the full flight envelope of the aircraft. Examples of such trajectories are high-g turns and high angle-of-attack maneuvers. This article presents the development of a multimodal flight control and flight path planning scheme that allows the vehicle to autonomously perform agile maneuvers over its full flight envelope. The key element of this scheme is the maneuver decomposition methodology, which aims to reduce the complexity of the planning and control problems for UCAVs. This article demonstrates how a parameterized family of maneuver modes for a UCAV can be developed systematically and how an arbitrary agile maneuver can be decomposed into simpler segments. In addition, the article presents a multimodal flight control scheme in which each of the maneuver modes is controlled locally by a sliding mode controller. The overall capability of the system is demonstrated in challenging scenarios such as navigation in dense environments and autonomous execution of aerobatics competition sequences.

Robust Adaptive Markov Decision Processes: Planning with Model Uncertainty

Abstract: The ability of autonomous systems to make complex decisions is becoming an increasingly commonplace requirement for many cooperative control operations, including the management of teams of robots such as unmanned aerial vehicles (UAV). Central to this research is the requirement to optimize the vehicle decisions, such as route planning and allocation of team resources, while operating in a dynamic and uncertain environment. Even with the advent of increasingly sophisticated vehicle sensors that can improve the information about the surroundings, uncertainty remains a ubiquitous feature of UAV applications and a key issue in UAV research.

Unmanned Aerial Vehicles and Control: Lockheed Martin Advanced Technology Laboratories

Abstract: Unmanned aerial vehicles (UAVs) are playing increasingly important roles on the battlefield and in our everyday lives. In military missions, they perform intelligence gathering, surveillance, and reconnaissance. In the civilian arena, UAVs provide climate monitoring [1], [2]; pipeline inspection [3]; farm management [4], and pollution monitoring [5]. Some enterprising realtors have even used them for virtual home tours [6]! Recent legislation will further promote the civilian use of UAVs by requiring the Federal Aviation Authority (FAA) to open the skies to UAV use in a variety of commercial applications. While the international community has not come to a consensus on permissible civilian UAV use, the European Union has a number of ongoing initiatives to mirror the FAA's actions [7].

A Monoball Robot Based on LEGO Mindstorms [Focus on Education]

Abstract: The LEGO Mindstorms kit is widely used in many areas of education and research, covering a number of applications from benchtop demonstrations to postgraduate research, laboratories, and projects [1][3]. Although the kit was initially designed as a toy for children over 12 years of age, its use in university courses is increasing yearly. The kit has attracted the interest of people working in many areas, including artificial intelligence, embedded systems, control systems, robotics, and operating systems. The LEGO Mindstorms kit is inexpensive, easily reconfigurable, reprogrammable, versatile, and robust, which makes it well suited for use in teaching [4].

Hamiltonian-Based Clustering: Algorithms for Static and Dynamic Clustering in Data Mining and Image Processing

Abstract: The large amount of data available for analysis and management raises the need for defining, determining, and extracting meaningful information from the data. Hence in scientific, engineering, and economics studies, the practice of clustering data arises naturally when sets of data have to be divided into subgroups with the aim of possibly deducting common features for data belonging to the same subgroup. For instance, the innovation scoreboard [1] (see Figure 1) allows for the classification of the countries into four main clusters corresponding to the level of innovation defining the “leaders,” the “followers,” the “trailing,” and the “catching up” countries. Many other disciplines may require or take advantage of a clustering of data, from market research [2] to gene expression analysis [3], from biology to image processing [4][7]. Therefore, several clustering techniques have been developed (for details see “Review of Clustering Algorithms”).

Modeling and Control of Exhaust Recompression HCCI: Split Fuel Injection for Cylinder-Individual Combustion Control

Abstract: The 21st century has brought with it a strong push in the automotive industry to develop energy technologies that can better address the global challenges posed by the depletion of finite fossil fuel resources and the pollution caused by using these fossil fuels as an energy resource. The two key words dominating automotive powertrain research today, therefore, are efficiency and emissions-There is a strong push for cars that burn less fuel and emit fewer pollutants. Some of the technologies in the spotlight include fuel cells, batteries, hybrid powertrains, and advanced IC engine strategies. Though fuel cells and battery technologies are very promising in the long-term future, they are currently more expensive and less practical than hybrid and advanced combustion engine strategies and are expected to remain so in the near future. In addition, technological advances that have made electronic control systems ubiquitous in todays automobiles have enabled combustion strategies that would not have been possible even a decade ago. One such advanced engine strategy that presents significant opportunities for reduced emissions and higher efficiencies is homogeneous charge compression ignition (HCCI).

On the Analysis of Robust Stability of Metabolic Pathways [Focus on Education]

Abstract: It is common to encounter proposed solutions to analysis problems posed in other fields that ignore the methodologies, results, and tools published in the control literature. Below is an example where a result adopted from a robust control paper shows that a proposed approach for analyzing a metabolic network can produce misleading results.

Short Online Videos to Excite and Engage Students About Control [Focus on Education]

Abstract: In this issue of “Focus on Education,” K.K. Leang discusses the use of online videos to engage students in control engineering, and S. Sanchez, T. Arribas, M. Gomez, and O.R. Polo describe the application of the LEGO Mindstorms kit in controlling a mobile robot that balances itself on a single spherical wheel while staying in place or moving in a desired direction.

Second Workshop on Dynamics and Control of Micro- and Nanoscale Systems [Conference Reports]

Abstract: The authors briefly describe the events of the Second Workshop on Dynamics and Control of Micro- and Nanoscale Systems was held at the University of Newcastle, Australia, February 23? 24, 2012. The purpose of this workshop was to bring together some of the leading experts in the field of dynamics and control of micro- and nanoscale systems to discuss emerging research problems and recent advances in the field. The two-day program comprised two keynote speeches and 17 invited presentations.

An Experiment for Teaching Students About Control at the Nanoscale [Focus on Education]

Abstract: One billionth of a meter is small. Conveying how small 1 nm is to students can be challenging. In class when this topic comes up for the first time, I always rely on the marble and Earth example. I ask my students that, if the Earth (with a diameter of approximately 12,756 km) was scaled to 1 m, then what normal-sized object on Earth would be representative of 1 nm? The responses vary from a 747 passenger plane to a small car. Rarely is suggested an object smaller than a soccer ball. The students are then told the answer, “It's a marble!,” which usually triggers a reaction in which many students' jaws drop wide open. This response is short lived, however, because the next question the students ask is, “How do controls fit in with nano?” This column describes an approach used to introduce undergraduate students to control at the nanoscale.

Fifty Years of CDCs: Rises, Falls, and Trends in Systems and Control [President's Message]

Abstract: December 2011, the society celebrated the 50th anniversary of our flagship annual event - the IEEE Conference on Decision and Control?known to most of us as "the CDC." During the 1960s, it was actually called the IEEE Symposium on Adaptive Processes, and the first time that it took place as IEEE Conference on Decision and Control was in 1971. The author decided to take a quick look at past CDC programs and proceedings to get a better sense of how specific topics emerged and flourished and how interest in them evolved through the years. As he did so, he observed some trends, many of them predictable, others a little surprising. He eventually decided to collect some real data to quantify his observations, which has led to the main content of this column.

Teaching the Difference Between Stiffness and Damping [Lecture Notes]

Abstract: A necessary step in the design of an effective control system is to understand its dynamics. It is not uncommon for a well-trained control engineer to use such an understanding to redesign the system to become easier to control, which requires an even deeper understanding of how the components of a system influence its overall dynamics. In this issue Henry C. Fu and Kam K. Leang discuss the difference between stiffness and damping when understanding the dynamics of mechanical systems.

Chasing Impact Factors, or Making an Impact on Technology? [From the Editor]

Abstract: As an editor-in-chief, the author receives many e-mails on the impact factor of various journals so he looked up the "new" impact factor for IEEE Control Systems Magazine (CSM). While its 2011 impact factor is high for a control publication, the value of this metric is unclear. Many people quantify the quality of a journal in terms of this metric, although it is well known that the impact factor metric can be highly misleading. He feels the main problem is not the metric per se, but that the metric is stated as a measure of the quality of the journal. The author maintains that there is no aspect of the calculation of the impact factor that is a measure of quality. While it could be argued that the measure of the number of times that an article has been cited is a measure of popularity, our everyday experience is that popularity is not necessarily correlated with quality. He concludes that, all other things being equal, he would prefer an article that makes a positive impact on technology and society that lowers the associated journal's impact factor than a paper that raises a journal's impact factor but makes no positive longterm impact.