This paper first offers a transversal view through microfluidics theory and applications, starting from a brief overview on microfluidic systems and related theoretical issues, covering different kinds of phenomena, from continuous to multiphase flow. Multidimensional models, from lumped parameters to numerical models and computational solutions, are then considered as preliminary tools for the characterization of spatio-temporal dynamics in microfluidic flows. Following these, experimental approaches through original monitoring opto-electronic interfaces and systems are discussed. Finally, a vision of two phase microfluidic phenomena is given through nonlinear analyses applied to experimental time series.

Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control

Over the last few decades, extensive use of flexible manipulators in various robotic applications has made it as one of the research interests for many scholars over the world. Recent studies on the modeling, sensor systems and controllers for the applications of flexible robotic manipulators are reviewed in order to complement the previous literature surveyed by Benosman & Vey (Robotica 22:533---545, 2004) and Dwivedy & Eberhard (Mech. Mach. Theory 41:749---777, 2006) . A brief introduction of the essential modeling techniques is first presented, followed by a review of the practical alternatives of sensor systems that can help scientists or engineers to choose the appropriate sensors for their applications. It followed by the main goal of this paper with a comprehensive review of the control strategies for the flexible manipulators and flexible joints that were studied in recent literatures. The issues for controlling flexible manipulators are highlighted. Most of the noteworthy control techniques that were not covered in the recent surveys in references (Benosman & Vey Robotica 22:533---545, 2004; Dwivedy & Eberhard Mech. Mach. Theory 41:749---777, 2006) are then reviewed. It concludes by providing some possible issues for future research works.

Review of Control and Sensor System of Flexible Manipulator

This article provides a study of modern and classical approaches used for PID tuning and its applications in various domains. Most of the control systems that are implemented to date with the use of PID control because of its simple structure, ease of implementation, and active research in tuning the PID for a long time. The techniques reviewed in the paper are in the order from classical to modern optimization rules used for the PID tuning. This paper attempts to address the literature review of PID control in an era of control system and bio-medical applications. The development of classical PID to the integration of intelligent control to it, has been surveyed by consideration of various application domains. The primary purpose of this document is to provide a detailed point of information for the people to understand the command of PID in different application areas.

A review of PID control, tuning methods and applications

In this paper, a novel iterative two-stage dual heuristic programming (DHP) is proposed to solve the optimal control problems for a class of discrete-time switched nonlinear systems subject to actuators saturation. First, a novel nonquadratic performance functional is introduced to confront control constraints of the saturating actuator. Then, the iterative two-stage DHP algorithm is developed to solve the Hamilton-Jacobi-Bellman (HJB) equation of the switched system with the saturating actuator. Moreover, the convergence and optimality of the two-stage DHP algorithm are strictly proven. To implement this algorithm efficiently, there are two neural networks used as parametric structure to approximate the costate function and the corresponding control law, respectively. Finally, simulation results are given to verify the effectiveness of the proposed algorithm.

Neural-Network-Based Constrained Optimal Control Scheme for Discrete-Time Switched Nonlinear System Using Dual Heuristic Programming

In this paper, a novel strategy is established to design the robust controller for a class of continuous-time nonlinear systems with uncertainties based on the online policy iteration algorithm. The robust control problem is transformed into the optimal control problem by properly choosing a cost function that reflects the uncertainties, regulation, and control. An online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman (HJB) equation by constructing a critic neural network. The approximate expression of the optimal control policy can be derived directly. The closed-loop system is proved to possess the uniform ultimate boundedness. The equivalence of the neural-network-based HJB solution of the optimal control problem and the solution of the robust control problem is established as well. Two simulation examples are provided to verify the effectiveness of the present robust control scheme.

Policy Iteration Algorithm for Online Design of Robust Control for a Class of Continuous-Time Nonlinear Systems

This paper exploits reinforcement learning (RL) for developing real-time adaptive control of tip trajectory and deflection of a two-link flexible manipulator handling variable payloads. This proposed adaptive controller consists of a proportional derivative (PD) tracking loop and an actor-critic-based RL loop that adapts the actor and critic weights in response to payload variations while suppressing the tip deflection and tracking the desired trajectory. The actor-critic-based RL loop uses a recursive least square (RLS)-based temporal difference (TD) learning with eligibility trace and an adaptive memory to estimate the critic weights and a gradient-based estimator for estimating actor weights. Tip trajectory tracking and suppression of tip deflection performances of the proposed RL-based adaptive controller (RLAC) are compared with that of a nonlinear regression-based direct adaptive controller (DAC) and a fuzzy learning-based adaptive controller (FLAC). Simulation and experimental results envisage that the RLAC outperforms both the DAC and FLAC.

Real-Time Adaptive Control of a Flexible Manipulator Using Reinforcement Learning

This paper presents a new nonlinear adaptive model predictive controller (NAMPC) for tip position control of a flexible manipulator (FLM) when subjected to handle different payloads. The proposed adaptive control strategy consists of an online FLM dynamics identifier in the form of a nonlinear autoregressive moving average with exogenous input model and a control signal generator based on the above identified dynamics. The effectiveness of the proposed control algorithm is then verified by comparing its performance with that of a self-tuning controller (STC) and a nonlinear adaptive controller (NAC). The metrics of the controller performance are chosen as tip trajectory tracking accuracy and fast suppression of tip deflection. From the simulation and experimental results, it is observed that the proposed controller exhibits superior performance compared with the STC and NAC.

Nonlinear Adaptive Model Predictive Controller for a Flexible Manipulator: An Experimental Study

In this paper, a new nonlinear self-tuning PID controller &#x0028 NSPIDC &#x0029 is proposed to control the joint position and link deflection of a flexible-link manipulator &#x0028 FLM &#x0029 while it is subjected to carry different payloads. Since, payload is a critical parameter of the FLM whose variation greatly influences the controller performance. The proposed controller guarantees stability under change in payload by attenuating the non-modeled higher order dynamics using a new nonlinear autoregressive moving average with exogenous-input &#x0028 NARMAX &#x0029 model of the FLM. The parameters of the FLM are identified on-line using recursive least square &#x0028 RLS &#x0029 algorithm and using minimum variance control &#x0028 MVC &#x0029 laws the control parameters are updated in real-time. This proposed NSPID controller has been implemented in real-time on an experimental set-up. The joint tracking and link deflection performances of the proposed adaptive controller are compared with that of a popular direct adaptive controller &#x0028 DAC &#x0029 . From the obtained results, it is confirmed that the proposed controller exhibits improved performance over the DAC both in terms of accurate position tracking and quick damping of link deflections when subjected to variable payloads.

Position control of a flexible manipulator using a new nonlinear self-tuning PID controller

Teaching a subject on system identification and adaptive control to Master’s students is a challenging task. In view of providing a platform to achieve the teaching-learning on adaptive control eff...

A flexible robotic control experiment for teaching nonlinear adaptive control

This paper proposes a new adaptive control using the concept of reinforcement learning to address adaptivity for varied payload conditions for a two-link flexible manipulator (TLFM). The application of reinforcement learning has been implemented using a method called adaptive dynamic programming. Decentralized controllers for the decoupled system have been also designed using LQR technique. Then the reinforcement learning is used to tune the gains of the optimal control to adapt in terms of different payload to the manipulator end effecter. Simulation results show that proposed controller provides better end point tracking then LQR fixed gain controller.

Santanu Kumar Pradhan

Papers

Real-Time Adaptive Control of a Flexible Manipulator Using Reinforcement Learning

Nonlinear Adaptive Model Predictive Controller for a Flexible Manipulator: An Experimental Study

Position control of a flexible manipulator using a new nonlinear self-tuning PID controller

A flexible robotic control experiment for teaching nonlinear adaptive control

Direct adaptive control of a flexible robot using reinforcement learning