Showing papers in "IEEE Transactions on Neural Networks in 2012"

$L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver

Abstract: The special importance of L1/2 regularization has been recognized in recent studies on sparse modeling (particularly on compressed sensing). The L1/2 regularization, however, leads to a nonconvex, nonsmooth, and non-Lipschitz optimization problem that is difficult to solve fast and efficiently. In this paper, through developing a threshoding representation theory for L1/2 regularization, we propose an iterative half thresholding algorithm for fast solution of L1/2 regularization, corresponding to the well-known iterative soft thresholding algorithm for L1 regularization, and the iterative hard thresholding algorithm for L0 regularization. We prove the existence of the resolvent of gradient of ||x||1/21/2, calculate its analytic expression, and establish an alternative feature theorem on solutions of L1/2 regularization, based on which a thresholding representation of solutions of L1/2 regularization is derived and an optimal regularization parameter setting rule is formulated. The developed theory provides a successful practice of extension of the well- known Moreau's proximity forward-backward splitting theory to the L1/2 regularization case. We verify the convergence of the iterative half thresholding algorithm and provide a series of experiments to assess performance of the algorithm. The experiments show that the half algorithm is effective, efficient, and can be accepted as a fast solver for L1/2 regularization. With the new algorithm, we conduct a phase diagram study to further demonstrate the superiority of L1/2 regularization over L1 regularization.

Twenty Years of Mixture of Experts

Abstract: In this paper, we provide a comprehensive survey of the mixture of experts (ME). We discuss the fundamental models for regression and classification and also their training with the expectation-maximization algorithm. We follow the discussion with improvements to the ME model and focus particularly on the mixtures of Gaussian process experts. We provide a review of the literature for other training methods, such as the alternative localized ME training, and cover the variational learning of ME in detail. In addition, we describe the model selection literature which encompasses finding the optimum number of experts, as well as the depth of the tree. We present the advances in ME in the classification area and present some issues concerning the classification model. We list the statistical properties of ME, discuss how the model has been modified over the years, compare ME to some popular algorithms, and list several applications. We conclude our survey with future directions and provide a list of publicly available datasets and a list of publicly available software that implement ME. Finally, we provide examples for regression and classification. We believe that the study described in this paper will provide quick access to the relevant literature for researchers and practitioners who would like to improve or use ME, and that it will stimulate further studies in ME.

Quantized Kernel Least Mean Square Algorithm

Abstract: In this paper, we propose a quantization approach, as an alternative of sparsification, to curb the growth of the radial basis function structure in kernel adaptive filtering. The basic idea behind this method is to quantize and hence compress the input (or feature) space. Different from sparsification, the new approach uses the “redundant” data to update the coefficient of the closest center. In particular, a quantized kernel least mean square (QKLMS) algorithm is developed, which is based on a simple online vector quantization method. The analytical study of the mean square convergence has been carried out. The energy conservation relation for QKLMS is established, and on this basis we arrive at a sufficient condition for mean square convergence, and a lower and upper bound on the theoretical value of the steady-state excess mean square error. Static function estimation and short-term chaotic time-series prediction examples are presented to demonstrate the excellent performance.

Synchronization Control for Nonlinear Stochastic Dynamical Networks: Pinning Impulsive Strategy

Abstract: In this paper, a new control strategy is proposed for the synchronization of stochastic dynamical networks with nonlinear coupling. Pinning state feedback controllers have been proved to be effective for synchronization control of state-coupled dynamical networks. We will show that pinning impulsive controllers are also effective for synchronization control of the above mentioned dynamical networks. Some generic mean square stability criteria are derived in terms of algebraic conditions, which guarantee that the whole state-coupled dynamical network can be forced to some desired trajectory by placing impulsive controllers on a small fraction of nodes. An effective method is given to select the nodes which should be controlled at each impulsive constants. The proportion of the controlled nodes guaranteeing the stability is explicitly obtained, and the synchronization region is also derived and clearly plotted. Numerical simulations are exploited to demonstrate the effectiveness of the pinning impulsive strategy proposed in this paper.

Discriminative Least Squares Regression for Multiclass Classification and Feature Selection

Abstract: This paper presents a framework of discriminative least squares regression (LSR) for multiclass classification and feature selection. The core idea is to enlarge the distance between different classes under the conceptual framework of LSR. First, a technique called e-dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged. Then, the e-draggings are integrated into the LSR model for multiclass classification. Our learning framework, referred to as discriminative LSR, has a compact model form, where there is no need to train two-class machines that are independent of each other. With its compact form, this model can be naturally extended for feature selection. This goal is achieved in terms of L2,1 norm of matrix, generating a sparse learning model for feature selection. The model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently. Experimental evaluation over a range of benchmark datasets indicates the validity of our method.

Global Stability of Complex-Valued Recurrent Neural Networks With Time-Delays

Abstract: Since the last decade, several complex-valued neural networks have been developed and applied in various research areas. As an extension of real-valued recurrent neural networks, complex-valued recurrent neural networks use complex-valued states, connection weights, or activation functions with much more complicated properties than real-valued ones. This paper presents several sufficient conditions derived to ascertain the existence of unique equilibrium, global asymptotic stability, and global exponential stability of delayed complex-valued recurrent neural networks with two classes of complex-valued activation functions. Simulation results of three numerical examples are also delineated to substantiate the effectiveness of the theoretical results.

Domain Adaptation From Multiple Sources: A Domain-Dependent Regularization Approach

Abstract: In this paper, we propose a new framework called domain adaptation machine (DAM) for the multiple source domain adaption problem. Under this framework, we learn a robust decision function (referred to as target classifier) for label prediction of instances from the target domain by leveraging a set of base classifiers which are prelearned by using labeled instances either from the source domains or from the source domains and the target domain. With the base classifiers, we propose a new domain-dependent regularizer based on smoothness assumption, which enforces that the target classifier shares similar decision values with the relevant base classifiers on the unlabeled instances from the target domain. This newly proposed regularizer can be readily incorporated into many kernel methods (e.g., support vector machines (SVM), support vector regression, and least-squares SVM (LS-SVM)). For domain adaptation, we also develop two new domain adaptation methods referred to as FastDAM and UniverDAM. In FastDAM, we introduce our proposed domain-dependent regularizer into LS-SVM as well as employ a sparsity regularizer to learn a sparse target classifier with the support vectors only from the target domain, which thus makes the label prediction on any test instance very fast. In UniverDAM, we additionally make use of the instances from the source domains as Universum to further enhance the generalization ability of the target classifier. We evaluate our two methods on the challenging TRECIVD 2005 dataset for the large-scale video concept detection task as well as on the 20 newsgroups and email spam datasets for document retrieval. Comprehensive experiments demonstrate that FastDAM and UniverDAM outperform the existing multiple source domain adaptation methods for the two applications.

Robust Exponential Stability of Uncertain Delayed Neural Networks With Stochastic Perturbation and Impulse Effects

Abstract: This paper focuses on the hybrid effects of parameter uncertainty, stochastic perturbation, and impulses on global stability of delayed neural networks By using the Ito formula, Lyapunov function, and Halanay inequality, we established several mean-square stability criteria from which we can estimate the feasible bounds of impulses, provided that parameter uncertainty and stochastic perturbations are well-constrained Moreover, the present method can also be applied to general differential systems with stochastic perturbation and impulses

Stability Analysis of Markovian Jump Stochastic BAM Neural Networks With Impulse Control and Mixed Time Delays

Abstract: This paper discusses the issue of stability analysis for a class of impulsive stochastic bidirectional associative memory neural networks with both Markovian jump parameters and mixed time delays. The jumping parameters are modeled as a continuous-time discrete-state Markov chain. Based on a novel Lyapunov-Krasovskii functional, the generalized Ito's formula, mathematical induction, and stochastic analysis theory, a linear matrix inequality approach is developed to derive some novel sufficient conditions that guarantee the exponential stability in the mean square of the equilibrium point. At the same time, we also investigate the robustly exponential stability in the mean square of the corresponding system with unknown parameters. It should be mentioned that our stability results are delay-dependent, which depend on not only the upper bounds of time delays but also their lower bounds. Moreover, the derivatives of time delays are not necessarily zero or smaller than one since several free matrices are introduced in our results. Consequently, the results obtained in this paper are not only less conservative but also generalize and improve many earlier results. Finally, two numerical examples and their simulations are provided to show the effectiveness of the theoretical results.

Memristor Bridge Synapse-Based Neural Network and Its Learning

Abstract: Analog hardware architecture of a memristor bridge synapse-based multilayer neural network and its learning scheme is proposed. The use of memristor bridge synapse in the proposed architecture solves one of the major problems, regarding nonvolatile weight storage in analog neural network implementations. To compensate for the spatial nonuniformity and nonideal response of the memristor bridge synapse, a modified chip-in-the-loop learning scheme suitable for the proposed neural network architecture is also proposed. In the proposed method, the initial learning is conducted in software, and the behavior of the software-trained network is learned by the hardware network by learning each of the single-layered neurons of the network independently. The forward calculation of the single-layered neuron learning is implemented on circuit hardware, and followed by a weight updating phase assisted by a host computer. Unlike conventional chip-in-the-loop learning, the need for the readout of synaptic weights for calculating weight updates in each epoch is eliminated by virtue of the memristor bridge synapse and the proposed learning scheme. The hardware architecture along with the successful implementation of proposed learning on a three-bit parity network, and on a car detection network is also presented.

Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

Abstract: In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.

Online Nonnegative Matrix Factorization With Robust Stochastic Approximation

Abstract: Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-scale or streaming datasets. In this paper, we propose an efficient online RSA-NMF algorithm (OR-NMF) that learns NMF in an incremental fashion and thus solves this problem. In particular, OR-NMF receives one sample or a chunk of samples per step and updates the bases via robust stochastic approximation. Benefitting from the smartly chosen learning rate and averaging technique, OR-NMF converges at the rate of in each update of the bases. Furthermore, we prove that OR-NMF almost surely converges to a local optimal solution by using the quasi-martingale. By using a buffering strategy, we keep both the time and space complexities of one step of the OR-NMF constant and make OR-NMF suitable for large-scale or streaming datasets. Preliminary experimental results on real-world datasets show that OR-NMF outperforms the existing online NMF (ONMF) algorithms in terms of efficiency. Experimental results of face recognition and image annotation on public datasets confirm the effectiveness of OR-NMF compared with the existing ONMF algorithms.

Exponential Stabilization of Memristive Neural Networks With Time Delays

Abstract: In this paper, a general class of memristive neural networks with time delays is formulated and studied. Some sufficient conditions in terms of linear matrix inequalities are obtained, in order to achieve exponential stabilization. The result can be applied to the closed-loop control of memristive systems. In particular, several succinct criteria are given to ascertain the exponential stabilization of memristive cellular neural networks. In addition, a simplified and effective algorithm is considered for design of the optimal controller. These conditions are the improvement and extension of the existing results in the literature. Two numerical examples are given to illustrate the theoretical results via computer simulations.

Study on the Impact of Partition-Induced Dataset Shift on $k$ -Fold Cross-Validation

Abstract: Cross-validation is a very commonly employed technique used to evaluate classifier performance. However, it can potentially introduce dataset shift, a harmful factor that is often not taken into account and can result in inaccurate performance estimation. This paper analyzes the prevalence and impact of partition-induced covariate shift on different k-fold cross-validation schemes. From the experimental results obtained, we conclude that the degree of partition-induced covariate shift depends on the cross-validation scheme considered. In this way, worse schemes may harm the correctness of a single-classifier performance estimation and also increase the needed number of repetitions of cross-validation to reach a stable performance estimation.

Exponential Synchronization of Neural Networks With Discrete and Distributed Delays Under Time-Varying Sampling

Abstract: This paper investigates the problem of master-slave synchronization for neural networks with discrete and distributed delays under variable sampling with a known upper bound on the sampling intervals. An improved method is proposed, which captures the characteristic of sampled-data systems. Some delay-dependent criteria are derived to ensure the exponential stability of the error systems, and thus the master systems synchronize with the slave systems. The desired sampled-data controller can be achieved by solving a set of linear matrix inequalitys, which depend upon the maximum sampling interval and the decay rate. The obtained conditions not only have less conservatism but also have less decision variables than existing results. Simulation results are given to show the effectiveness and benefits of the proposed methods.

Generalization Characteristics of Complex-Valued Feedforward Neural Networks in Relation to Signal Coherence

Abstract: Applications of complex-valued neural networks (CVNNs) have expanded widely in recent years-in particular in radar and coherent imaging systems. In general, the most important merit of neural networks lies in their generalization ability. This paper compares the generalization characteristics of complex-valued and real-valued feedforward neural networks in terms of the coherence of the signals to be dealt with. We assume a task of function approximation such as interpolation of temporal signals. Simulation and real-world experiments demonstrate that CVNNs with amplitude-phase-type activation function show smaller generalization error than real-valued networks, such as bivariate and dual-univariate real-valued neural networks. Based on the results, we discuss how the generalization characteristics are influenced by the coherence of the signals depending on the degree of freedom in the learning and on the circularity in neural dynamics.

Stability and Dissipativity Analysis of Static Neural Networks With Time Delay

Abstract: This paper is concerned with the problems of stability and dissipativity analysis for static neural networks (NNs) with time delay. Some improved delay-dependent stability criteria are established for static NNs with time-varying or time-invariant delay using the delay partitioning technique. Based on these criteria, several delay-dependent sufficient conditions are given to guarantee the dissipativity of static NNs with time delay. All the given results in this paper are not only dependent upon the time delay but also upon the number of delay partitions. Some examples are given to illustrate the effectiveness and reduced conservatism of the proposed results.

Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update

Abstract: In this paper, the Hamilton-Jacobi-Bellman equation is solved forward-in-time for the optimal control of a class of general affine nonlinear discrete-time systems without using value and policy iterations. The proposed approach, referred to as adaptive dynamic programming, uses two neural networks (NNs), to solve the infinite horizon optimal regulation control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. One NN approximates the cost function and is referred to as the critic NN, while the second NN generates the control input and is referred to as the action NN. The cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time-based ADP. Novel update laws for tuning the unknown weights of the NNs online are derived. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small bounded error over time. In the absence of disturbances, an optimal control is demonstrated. Simulation results are included to show the effectiveness of the approach. The end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation.

$H_{\infty}$ State Estimation for Discrete-Time Complex Networks With Randomly Occurring Sensor Saturations and Randomly Varying Sensor Delays

Abstract: In this paper, the state estimation problem is investigated for a class of discrete time-delay nonlinear complex networks with randomly occurring phenomena from sensor measurements. The randomly occurring phenomena include randomly occurring sensor saturations (ROSSs) and randomly varying sensor delays (RVSDs) that result typically from networked environments. A novel sensor model is proposed to describe the ROSSs and the RVSDs within a unified framework via two sets of Bernoulli-distributed white sequences with known conditional probabilities. Rather than employing the commonly used Lipschitz-type function, a more general sector-like nonlinear function is used to describe the nonlinearities existing in the network. The purpose of the addressed problem is to design a state estimator to estimate the network states through available output measurements such that, for all probabilistic sensor saturations and sensor delays, the dynamics of the estimation error is guaranteed to be exponentially mean-square stable and the effect from the exogenous disturbances to the estimation accuracy is attenuated at a given level by means of an H∞-norm. In terms of a novel Lyapunov-Krasovskii functional and the Kronecker product, sufficient conditions are established under which the addressed state estimation problem is recast as solving a convex optimization problem via the semidefinite programming method. A simulation example is provided to show the usefulness of the proposed state estimation conditions.

Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes

Abstract: Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron-like and perform well in both regression and classification applications. In this brief, we propose an ELM with adaptive growth of hidden nodes (AG-ELM), which provides a new approach for the automated design of networks. Different from other incremental ELMs (I-ELMs) whose existing hidden nodes are frozen when the new hidden nodes are added one by one, in AG-ELM the number of hidden nodes is determined in an adaptive way in the sense that the existing networks may be replaced by newly generated networks which have fewer hidden nodes and better generalization performance. We then prove that such an AG-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results demonstrate and verify that this new approach can achieve a more compact network architecture than the I-ELM.

Bidirectional Extreme Learning Machine for Regression Problem and Its Learning Effectiveness

Abstract: It is clear that the learning effectiveness and learning speed of neural networks are in general far slower than required, which has been a major bottleneck for many applications. Recently, a simple and efficient learning method, referred to as extreme learning machine (ELM), was proposed by Huang , which has shown that, compared to some conventional methods, the training time of neural networks can be reduced by a thousand times. However, one of the open problems in ELM research is whether the number of hidden nodes can be further reduced without affecting learning effectiveness. This brief proposes a new learning algorithm, called bidirectional extreme learning machine (B-ELM), in which some hidden nodes are not randomly selected. In theory, this algorithm tends to reduce network output error to 0 at an extremely early learning stage. Furthermore, we find a relationship between the network output error and the network output weights in the proposed B-ELM. Simulation results demonstrate that the proposed method can be tens to hundreds of times faster than other incremental ELM algorithms.

Fault Detection and Diagnosis of Induction Motors Using Motor Current Signature Analysis and a Hybrid FMM–CART Model

Abstract: In this paper, a novel approach to detect and classify comprehensive fault conditions of induction motors using a hybrid fuzzy min-max (FMM) neural network and classification and regression tree (CART) is proposed. The hybrid model, known as FMM-CART, exploits the advantages of both FMM and CART for undertaking data classification and rule extraction problems. A series of real experiments is conducted, whereby the motor current signature analysis method is applied to form a database comprising stator current signatures under different motor conditions. The signal harmonics from the power spectral density are extracted as discriminative input features for fault detection and classification with FMM-CART. A comprehensive list of induction motor fault conditions, viz., broken rotor bars, unbalanced voltages, stator winding faults, and eccentricity problems, has been successfully classified using FMM-CART with good accuracy rates. The results are comparable, if not better, than those reported in the literature. Useful explanatory rules in the form of a decision tree are also elicited from FMM-CART to analyze and understand different fault conditions of induction motors.

Innovative Second-Generation Wavelets Construction With Recurrent Neural Networks for Solar Radiation Forecasting

Abstract: Solar radiation prediction is an important challenge for the electrical engineer because it is used to estimate the power developed by commercial photovoltaic modules. This paper deals with the problem of solar radiation prediction based on observed meteorological data. A 2-day forecast is obtained by using novel wavelet recurrent neural networks (WRNNs). In fact, these WRNNS are used to exploit the correlation between solar radiation and timescale-related variations of wind speed, humidity, and temperature. The input to the selected WRNN is provided by timescale-related bands of wavelet coefficients obtained from meteorological time series. The experimental setup available at the University of Catania, Italy, provided this information. The novelty of this approach is that the proposed WRNN performs the prediction in the wavelet domain and, in addition, also performs the inverse wavelet transform, giving the predicted signal as output. The obtained simulation results show a very low root-mean-square error compared to the results of the solar radiation prediction approaches obtained by hybrid neural networks reported in the recent literature.

Kernel Recursive Least-Squares Tracker for Time-Varying Regression

Abstract: In this paper, we introduce a kernel recursive least-squares (KRLS) algorithm that is able to track nonlinear, time-varying relationships in data. To this purpose, we first derive the standard KRLS equations from a Bayesian perspective (including a sensible approach to pruning) and then take advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in nonstationary scenarios. The resulting method is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner. In addition to its tracking ability, it has a number of appealing properties. It is online, requires a fixed amount of memory and computation per time step, incorporates regularization in a natural manner and provides confidence intervals along with each prediction. We include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm.

Synchronization of Markovian Coupled Neural Networks With Nonidentical Node-Delays and Random Coupling Strengths

Abstract: In this paper, a general model of coupled neural networks with Markovian jumping and random coupling strengths is introduced. In the process of evolution, the proposed model switches from one mode to another according to a Markovian chain, and all the modes have different constant time-delays. The coupling strengths are characterized by mutually independent random variables. When compared with most of existing dynamical network models which share common time-delay for all modes and have constant coupling strengths, our model is more practical because different chaotic neural network models can have different time-delays and coupling strength of complex networks may randomly vary around a constant due to environmental and artificial factors. By designing a novel Lyapunov functional and using some inequalities and the properties of random variables, we derive several new sufficient synchronization criteria formulated by linear matrix inequalities. The obtained criteria depend on mode-delays and mathematical expectations and variances of the random coupling strengths as well. Numerical examples are given to demonstrate the effectiveness of the theoretical results, meanwhile right-continuous Markovian chain is also presented.

Multistability of Neural Networks With Time-Varying Delays and Concave-Convex Characteristics

Abstract: In this paper, stability of multiple equilibria of neural networks with time-varying delays and concave-convex characteristics is formulated and studied. Some sufficient conditions are obtained to ensure that an n-neuron neural network with concave-convex characteristics can have a fixed point located in the appointed region. By means of an appropriate partition of the n-dimensional state space, when nonlinear activation functions of an n-neuron neural network are concave or convex in 2k+2m-1 intervals, this neural network can have (2k+2m-1)n equilibrium points. This result can be applied to the multiobjective optimal control and associative memory. In particular, several succinct criteria are given to ascertain multistability of cellular neural networks. These stability conditions are the improvement and extension of the existing stability results in the literature. A numerical example is given to illustrate the theoretical findings via computer simulations.

Complete Synchronization of Boolean Networks

Abstract: We examine complete synchronization of two deterministic Boolean networks (BNs) coupled unidirectionally in the drive-response configuration. A necessary and sufficient criterion is presented in terms of algebraic representations of BNs. As a consequence, we show that complete synchronization can occur only between two conditionally identical BNs when the transition matrix of the drive network is nonsingular. Two examples are worked out to illustrate the obtained results.

Feature Extraction With Deep Neural Networks by a Generalized Discriminant Analysis

Abstract: We present an approach to feature extraction that is a generalization of the classical linear discriminant analysis (LDA) on the basis of deep neural networks (DNNs). As for LDA, discriminative features generated from independent Gaussian class conditionals are assumed. This modeling has the advantages that the intrinsic dimensionality of the feature space is bounded by the number of classes and that the optimal discriminant function is linear. Unfortunately, linear transformations are insufficient to extract optimal discriminative features from arbitrarily distributed raw measurements. The generalized discriminant analysis (GerDA) proposed in this paper uses nonlinear transformations that are learnt by DNNs in a semisupervised fashion. We show that the feature extraction based on our approach displays excellent performance on real-world recognition and detection tasks, such as handwritten digit recognition and face detection. In a series of experiments, we evaluate GerDA features with respect to dimensionality reduction, visualization, classification, and detection. Moreover, we show that GerDA DNNs can preprocess truly high-dimensional input data to low-dimensional representations that facilitate accurate predictions even if simple linear predictors or measures of similarity are used.

Adaptive Dynamic Output Feedback Neural Network Control of Uncertain MIMO Nonlinear Systems With Prescribed Performance

Abstract: An adaptive dynamic output feedback neural network controller for a class of multi-input/multi-output affine in the control uncertain nonlinear systems is designed, capable of guaranteeing prescribed performance bounds on the system's output as well as boundedness of all other closed loop signals. It is proved that simply guaranteeing a boundedness property for the states of a specifically defined augmented closed loop system is necessary and sufficient to solve the problem under consideration. The proposed dynamic controller is of switching type. However, its continuity is guaranteed, thus alleviating any issues related to the existence and uniqueness of solutions. Simulations on a planar two-link articulated manipulator illustrate the approach.

Toward Automatic Time-Series Forecasting Using Neural Networks

Abstract: Over the past few decades, application of artificial neural networks (ANN) to time-series forecasting (TSF) has been growing rapidly due to several unique features of ANN models. However, to date, a consistent ANN performance over different studies has not been achieved. Many factors contribute to the inconsistency in the performance of neural network models. One such factor is that ANN modeling involves determining a large number of design parameters, and the current design practice is essentially heuristic and ad hoc, this does not exploit the full potential of neural networks. Systematic ANN modeling processes and strategies for TSF are, therefore, greatly needed. Motivated by this need, this paper attempts to develop an automatic ANN modeling scheme. It is based on the generalized regression neural network (GRNN), a special type of neural network. By taking advantage of several GRNN properties (i.e., a single design parameter and fast learning) and by incorporating several design strategies (e.g., fusing multiple GRNNs), we have been able to make the proposed modeling scheme to be effective for modeling large-scale business time series. The initial model was entered into the NN3 time-series competition. It was awarded the best prediction on the reduced dataset among approximately 60 different models submitted by scholars worldwide.