There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Filtering and smoothing methods are used to produce an accurate estimate of the state of a time-varying system based on multiple observational inputs (data). Interest in these methods has exploded in recent years, with numerous applications emerging in fields such as navigation, aerospace engineering, telecommunications, and medicine. This compact, informal introduction for graduate students and advanced undergraduates presents the current state-of-the-art filtering and smoothing methods in a unified Bayesian framework. Readers learn what non-linear Kalman filters and particle filters are, how they are related, and their relative advantages and disadvantages. They also discover how state-of-the-art Bayesian parameter estimation methods can be combined with state-of-the-art filtering and smoothing algorithms. The book’s practical and algorithmic approach assumes only modest mathematical prerequisites. Examples include MATLAB computations, and the numerous end-of-chapter exercises include computational assignments. MATLAB/GNU Octave source code is available for download at www.cambridge.org/sarkka, promoting hands-on work with the methods.

贝叶斯滤波与平滑 (Bayesian filtering and smoothing)

Entropy is a powerful tool for the analysis of time series, as it allows describing the probability distributions of the possible state of a system, and therefore the information encoded in it. Nevertheless, important information may be codified also in the temporal dynamics, an aspect which is not usually taken into account. The idea of calculating entropy based on permutation patterns (that is, permutations defined by the order relations among values of a time series) has received a lot of attention in the last years, especially for the understanding of complex and chaotic systems. Permutation entropy directly accounts for the temporal information contained in the time series; furthermore, it has the quality of simplicity, robustness and very low computational cost. To celebrate the tenth anniversary of the original work, here we analyze the theoretical foundations of the permutation entropy, as well as the main recent applications to the analysis of economical markets and to the understanding of biomedical systems.

Permutation Entropy and Its Main Biomedical and Econophysics Applications: A Review

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal $\mathcal {H}_{2}$ and $\mathcal {H}_\infty $ control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.

Optimal and Autonomous Control Using Reinforcement Learning: A Survey

Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning

Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics

In this paper, an output-feedback solution to the infinite-horizon linear quadratic tracking (LQT) problem for unknown discrete-time systems is proposed. An augmented system composed of the system dynamics and the reference trajectory dynamics is constructed. The state of the augmented system is constructed from a limited number of measurements of the past input, output, and reference trajectory in the history of the augmented system. A novel Bellman equation is developed that evaluates the value function related to a fixed policy by using only the input, output, and reference trajectory data from the augmented system. By using approximate dynamic programming, a class of reinforcement learning methods, the LQT problem is solved online without requiring knowledge of the augmented system dynamics only by measuring the input, output, and reference trajectory from the augmented system. We develop both policy iteration (PI) and value iteration (VI) algorithms that converge to an optimal controller that require only measuring the input, output, and reference trajectory data. The convergence of the proposed PI and VI algorithms is shown. A simulation example is used to verify the effectiveness of the proposed control scheme.

Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data

A dc-dc resonant converter has the advantage of overcoming switching losses and electromagnetic interference which are the main limitations of high frequency power converters. Nevertheless, the modeling and stability analysis of dc-dc resonant converters are considerably more complex than pulsewidth modulation counterparts. The conventional averaged linearized model of the resonant converter has limitations due to averaging and linearization. First of all, the linearized model has large modeling error in presence of large variations of reference voltage and input voltage. Furthermore, Converging area for stabilizing controllers is smaller in the averaged model. In order to overcome these limitations, this paper presents a novel framework for modeling of dc-dc resonant converters. As dc-dc resonant converters are naturally switched systems with affine subsystems, a piecewise affine model is derived directly from the converter model. The new model is based on analysis of the resonant converter on state plane trajectories. It is also suitable for precise simulation and high performance controller design of resonant converters. In addition, a stability analysis theorem is provided for the proposed model. The simulation and experimental results on a dc-dc series resonant converter show the effectiveness of this modeling approach.

Hybrid Modeling of a DC-DC Series Resonant Converter: Direct Piecewise Affine Approach

This paper proposes a maximum power point tracking (MPPT) technique based on the tip speed ratio control for small scale wind turbines (WTs). In this paper, artificial neural network based particle swarm optimization has been trained offline to learn the characteristic of the turbine power as a function of wind and machine speeds. Afterwards, it has been realized online to estimate the varying wind speed. It is essential to design a controller that can track the maximum peak of energy regardless of wind speed changes. Therefore, this work provides a novel robust direct adaptive fuzzy–Proportional-Integral (PI) controller during the MPPT process through tuning duty cycle of the boost converter for permanent magnet synchronous generator driven by a WT. The proposed method has successfully decreased the ripples of coefficient of power (Cp) which is the index of MPPT mode, under variations of the wind speed in comparison with conventional controller. Finally, a systematic analysis is presented which is in goo...

MPPT control of wind turbines by direct adaptive fuzzy-PI controller and using ANN-PSO wind speed estimator

Permutation entropy (PE) is a new complexity measure which can extract important information from long, complex and high-dimensional time series. The advantages of this measure such as its fast calculation and robustness with respect to additive noise make it suitable for biomedical signal analysis. In this paper the ability of PE for characterizing the normal and epileptic EEG signals is investigated. Classification is performed using discriminant analysis. The effect of additive Gaussian noise on the discrimination performance is also studied and some parameters derived from PE are suggested to improve the classification accuracy when the signal is contaminated with noise. The results indicate that the proposed measures can distinguish normal and epileptic EEG signals with an accuracy of more than 97% for clean EEG and more than 85% for highly noised EEG signals.

/pdf/fast-and-robust-detection-of-epilepsy-in-noisy-eeg-signals-1z6wkvwcdf.pdf

Ali Karimpour

Papers

Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics

Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data

Hybrid Modeling of a DC-DC Series Resonant Converter: Direct Piecewise Affine Approach

MPPT control of wind turbines by direct adaptive fuzzy-PI controller and using ANN-PSO wind speed estimator

Fast and Robust Detection of Epilepsy in Noisy EEG Signals Using Permutation Entropy