Showing papers on "Function approximation published in 2005"

A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

Abstract: This work presents a new sequential learning algorithm for radial basis function (RBF) networks referred to as generalized growing and pruning algorithm for RBF (GGAP-RBF). The paper first introduces the concept of significance for the hidden neurons and then uses it in the learning algorithm to realize parsimonious networks. The growing and pruning strategy of GGAP-RBF is based on linking the required learning accuracy with the significance of the nearest or intentionally added new neuron. Significance of a neuron is a measure of the average information content of that neuron. The GGAP-RBF algorithm can be used for any arbitrary sampling density for training samples and is derived from a rigorous statistical point of view. Simulation results for bench mark problems in the function approximation area show that the GGAP-RBF outperforms several other sequential learning algorithms in terms of learning speed, network size and generalization performance regardless of the sampling density function of the training data.

Incremental Online Learning in High Dimensions

Abstract: Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of—possibly redundant—inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.

Natural actor-critic

Abstract: This paper investigates a novel model-free reinforcement learning architecture, the Natural Actor-Critic. The actor updates are based on stochastic policy gradients employing Amari's natural gradient approach, while the critic obtains both the natural policy gradient and additional parameters of a value function simultaneously by linear regression. We show that actor improvements with natural policy gradients are particularly appealing as these are independent of coordinate frame of the chosen policy representation, and can be estimated more efficiently than regular policy gradients. The critic makes use of a special basis function parameterization motivated by the policy-gradient compatible function approximation. We show that several well-known reinforcement learning methods such as the original Actor-Critic and Bradtke's Linear Quadratic Q-Learning are in fact Natural Actor-Critic algorithms. Empirical evaluations illustrate the effectiveness of our techniques in comparison to previous methods, and also demonstrate their applicability for learning control on an anthropomorphic robot arm.

Backstepping-Based Flight Control with Adaptive Function Approximation

Abstract: A command filtered backstepping approach is presented that uses adaptive function approximation to control unmanned air vehicles. The controller is designed using three feedback loops. The command inputs to the airspeed and flight-path angle controller are x c , γ c , V c and the bounded first derivatives of these signals. That loop generates comand inputs μ c , α c for a wind-axis angle loop. The sideslip angle command β c is always zero. The wind-axis angle loop generates rate commands P c , Q c , R c for an inner loop that generates surface position commands. The control approach includes adaptive approximation of the aerodynamic force and moment coefficient functions. The approach maintains the stability (in the sense of Lyapunov) of the adaptive function approximation process in the presence of magnitude, rate, and bandwidth limitations on the intermediate states and the surfaces.

Smooth function approximation using neural networks

Abstract: An algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented. In this paper, the approach is implemented for the approximation of smooth batch data containing the function's input, output, and possibly, gradient information. The training set is associated to the network adjustable parameters by nonlinear weight equations. The cascade structure of these equations reveals that they can be treated as sets of linear systems. Hence, the training process and the network approximation properties can be investigated via linear algebra. Four algorithms are developed to achieve exact or approximate matching of input-output and/or gradient-based training sets. Their application to the design of forward and feedback neurocontrollers shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques.

A study of slope stability prediction using neural networks

Abstract: The determination of the non-linear behaviour of multivariate dynamic systems often presents a challenging and demanding problem. Slope stability estimation is an engineering problem that involves several parameters. The impact of these parameters on the stability of slopes is investigated through the use of computational tools called neural networks. A number of networks of threshold logic unit were tested, with adjustable weights. The computational method for the training process was a back-propagation learning algorithm. In this paper, the input data for slope stability estimation consist of values of geotechnical and geometrical input parameters. As an output, the network estimates the factor of safety (FS) that can be modelled as a function approximation problem, or the stability status (S) that can be modelled either as a function approximation problem or as a classification model. The performance of the network is measured and the results are compared to those obtained by means of standard analytical methods. Furthermore, the relative importance of the parameters is studied using the method of the partitioning of weights and compared to the results obtained through the use of Index Information Theory.

Basis Function Adaptation in Temporal Difference Reinforcement Learning

Abstract: Reinforcement Learning (RL) is an approach for solving complex multi-stage decision problems that fall under the general framework of Markov Decision Problems (MDPs), with possibly unknown parameters. Function approximation is essential for problems with a large state space, as it facilitates compact representation and enables generalization. Linear approximation architectures (where the adjustable parameters are the weights of pre-fixed basis functions) have recently gained prominence due to efficient algorithms and convergence guarantees. Nonetheless, an appropriate choice of basis function is important for the success of the algorithm. In the present paper we examine methods for adapting the basis function during the learning process in the context of evaluating the value function under a fixed control policy. Using the Bellman approximation error as an optimization criterion, we optimize the weights of the basis function while simultaneously adapting the (non-linear) basis function parameters. We present two algorithms for this problem. The first uses a gradient-based approach and the second applies the Cross Entropy method. The performance of the proposed algorithms is evaluated and compared in simulations.

A Generalization Error for Q-Learning

Abstract: Planning problems that involve learning a policy from a single training set of finite horizon trajectories arise in both social science and medical fields. We consider Q-learning with function approximation for this setting and derive an upper bound on the generalization error. This upper bound is in terms of quantities minimized by a Q-learning algorithm, the complexity of the approximation space and an approximation term due to the mismatch between Q-learning and the goal of learning a policy that maximizes the value function.

High-speed function approximation using a minimax quadratic interpolator

Abstract: A table-based method for high-speed function approximation in single-precision floating-point format is presented in this paper. Our focus is the approximation of reciprocal, square root, square root reciprocal, exponentials, logarithms, trigonometric functions, powering (with a fixed exponent p), or special functions. The algorithm presented here combines table look-up, an enhanced minimax quadratic approximation, and an efficient evaluation of the second-degree polynomial (using a specialized squaring unit, redundant arithmetic, and multioperand addition). The execution times and area costs of an architecture implementing our method are estimated, showing the achievement of the fast execution times of linear approximation methods and the reduced area requirements of other second-degree interpolation algorithms. Moreover, the use of an enhanced minimax approximation which, through an iterative process, takes into account the effect of rounding the polynomial coefficients to a finite size allows for a further reduction in the size of the look-up tables to be used, making our method very suitable for the implementation of an elementary function generator in state-of-the-art DSPs or graphics processing units (GPUs).

Function approximation via tile coding: automating parameter choice

Abstract: Reinforcement learning (RL) is a powerful abstraction of sequential decision making that has an established theoretical foundation and has proven effective in a variety of small, simulated domains. The success of RL on real-world problems with large, often continuous state and action spaces hinges on effective function approximation. Of the many function approximation schemes proposed, tile coding strikes an empirically successful balance among representational power, computational cost, and ease of use and has been widely adopted in recent RL work. This paper demonstrates that the performance of tile coding is quite sensitive to parameterization. We present detailed experiments that isolate the effects of parameter choices and provide guidance to their setting. We further illustrate that no single parameterization achieves the best performance throughout the learning curve, and contribute an automated technique for adjusting tile-coding parameters online. Our experimental findings confirm the superiority of adaptive parameterization to fixed settings. This work aims to automate the choice of approximation scheme not only on a problem basis but also throughout the learning process, eliminating the need for a substantial tuning effort.

Finite time bounds for sampling based fitted value iteration

Abstract: In this paper we consider sampling based fitted value iteration for discounted, large (possibly infinite) state space, finite action Markovian Decision Problems where only a generative model of the transition probabilities and rewards is available. At each step the image of the current estimate of the optimal value function under a Monte-Carlo approximation to the Bellman-operator is projected onto some function space. PAC-style bounds on the weighted Lp-norm approximation error are obtained as a function of the covering number and the approximation power of the function space, the iteration number and the sample size.

Approximation spaces and information granulation

Abstract: In this paper, we discuss approximation spaces in a granular computing framework. Such approximation spaces generalise the approaches to concept approximation existing in rough set theory. Approximation spaces are constructed as higher level information granules and are obtained as the result of complex modelling. We present illustrative examples of modelling approximation spaces that include approximation spaces for function approximation, inducing concept approximation, and some other information granule approximations. In modelling of such approximation spaces we use an important assumption that not only objects but also more complex information granules involved in approximations are perceived using only partial information about them.

Comparison of function approximation, heuristic, and derivative-based methods for automatic calibration of computationally expensive groundwater bioremediation models

Abstract: [1] The performance of function approximation (FA) methods is compared to heuristic and derivative-based nonlinear optimization methods for automatic calibration of biokinetic parameters of a groundwater bioremediation model of chlorinated ethenes on a hypothetical and a real field case. For the hypothetical case, on the basis of 10 trials on two different objective functions, the FA methods had the lowest mean and smaller deviation of the objective function among all algorithms for a combined Nash-Sutcliffe objective and among all but the derivative-based algorithm for a total squared error objective. The best algorithms in the hypothetical case were applied to calibrate eight parameters to data obtained from a site in California. In three trials the FA methods outperformed heuristic and derivative-based methods for both objective functions. This study indicates that function approximation methods could be a more efficient alternative to heuristic and derivative-based methods for automatic calibration of computationally expensive bioremediation models.

Table-based polynomials for fast hardware function evaluation

Abstract: Many general table-based methods for the evaluation in hardware of elementary functions have been published. The bipartite and multipartite methods implement a first-order approximation of the function using only table lookups and additions. Recently, a single multiplier second order method of similar inspiration has also been published. This paper extends such methods to approximations of arbitrary order, using adders, small multipliers, and very small ad hoc powering units. We obtain implementations that are both smaller and faster than previously published approaches. This paper also deals with the FPGA implementation of such methods. Previous work have consistently shown that increasing the approximation degree lead to not only smaller but also faster designs, as the reduction of the table size meant a reduction of its lookup time, which compensated for the addition and multiplication time. The experiments in this paper suggest that this still holds when going from order 2 to order 3, but no longer when using higher order approximations, where a tradeoff appears.

Fast rates for support vector machines

Abstract: We establish learning rates to the Bayes risk for support vector machines (SVMs) using a regularization sequence ${\it \lambda}_{n}={\it n}^{-\rm \alpha}$, where ${\it \alpha}\in$(0,1) is arbitrary. Under a noise condition recently proposed by Tsybakov these rates can become faster than n−1/2. In order to deal with the approximation error we present a general concept called the approximation error function which describes how well the infinite sample versions of the considered SVMs approximate the data-generating distribution. In addition we discuss in some detail the relation between the “classical” approximation error and the approximation error function. Finally, for distributions satisfying a geometric noise assumption we establish some learning rates when the used RKHS is a Sobolev space.

Nonlinear speech analysis using models for chaotic systems

Abstract: In this paper, we use concepts and methods from chaotic systems to model and analyze nonlinear dynamics in speech signals. The modeling is done not on the scalar speech signal, but on its reconstructed multidimensional attractor by embedding the scalar signal into a phase space. We have analyzed and compared a variety of nonlinear models for approximating the dynamics of complex systems using a small record of their observed output. These models include approximations based on global or local polynomials as well as approximations inspired from machine learning such as radial basis function networks, fuzzy-logic systems and support vector machines. Our focus has been on facilitating the application of the methods of chaotic signal analysis even when only a short time series is available, like phonemes in speech utterances. This introduced an increased degree of difficulty that was dealt with by resorting to sophisticated function approximation models that are appropriate for short data sets. Using these models enabled us to compute for short time series of speech sounds useful features like Lyapunov exponents that are used to assist in the characterization of chaotic systems. Several experimental insights are reported on the possible applications of such nonlinear models and features.

An input-output clustering approach to the synthesis of ANFIS networks

Abstract: A useful neural network paradigm for the solution of function approximation problems is represented by adaptive neuro-fuzzy inference systems (ANFIS). Data driven procedures for the synthesis of ANFIS networks are typically based on clustering a training set of numerical samples of the unknown function to be approximated. Some serious drawbacks often affect the clustering algorithms adopted in this context, according to the particular data space where they are applied. To overcome such problems, we propose a new ANFIS synthesis procedure where clustering is applied in the joint input-output data space. Using this approach, it is possible to determine the consequent part of Sugeno first-order rules and therefore the hyperplanes characterizing the local structure of the function to be approximated. Successively, the fuzzy antecedent part of each rule is determined using a particular fuzzy min-max classifier, which is based on the adaptive resolution mechanism. The generalization capability of the resulting ANFIS architecture is optimized using a constructive procedure for the automatic determination of the optimal number of rules. Simulation tests and comparisons with respect to other neuro-fuzzy techniques are discussed in the paper, in order to assess the efficiency of the proposed approach.

Efficient function approximation using truncated multipliers and squarers

Abstract: This paper presents a technique for designing linear and quadratic interpolators for function approximation using truncated multipliers and squarers. Initial coefficient values are found using a Chebyshev series approximation, and then adjusted through exhaustive simulation to minimize the maximum absolute error of the interpolator output. This technique is suitable for any function and any precision up to 24-bits (IEEE single precision). Designs for linear and quadratic interpolators that implement the reciprocal function, f(x)=1/x, are presented and analyzed as an example. We show that a 24-bit truncated reciprocal quadratic interpolator with a design specification /spl plusmn/1 ulp error requires 24.1% fewer partial products to implement than a comparable standard interpolator with the same error specification.

Optimal control of a fed-batch bioreactor using simulation-based approximate dynamic programming

Abstract: In this brief, we extend the simulation-based approximate dynamic programming (ADP) method to optimal feedback control of fed-batch reactors. We consider a free-end problem, wherein the batch time is considered in finding the optimal feeding strategy in addition to the final time productivity. In ADP, the optimal solution is parameterized in the form of profit-to-go function. The original definition of profit-to-go is modified to include the decision of batch termination. Simulations from heuristic feeding policies generate the initial profit-to-go versus state data. An artificial neural network then approximates profit-to-go as a function of process state. Iterations of the Bellman equation are used to improve the profit-to-go function approximator. The profit-to-go function approximator thus obtained, is then implemented in an online controller. This method is applied to cloned invertase expression in Saccharomyces cerevisiae in a fed-batch bioreactor.

Error estimates of local multiquadric‐based differential quadrature (LMQDQ) method through numerical experiments

Abstract: SUMMARY In this article, we present an error estimate of the derivative approximation by the local multiquadricbased differential quadrature (LMQDQ) method. Radial basis function is different from the polynomial approximation, in which Taylor series expansion is not applicable. So, the present analysis is performed through the numerical solution of Poisson equation. It is known that the approximation error of LMQDQ method depends on three factors, i.e. local density of knots h, free shape parameter c and number of supporting knots ns. By numerical experiments, their contribution to the approximation error and correlation were studied and analysed in this paper. An error estimate ∼ O((h/c) n ) is thereafter proposed, in which n is a positive constant and determined by the number of supporting knots ns. Copyright 2005 John Wiley & Sons, Ltd.

Function approximation processing method and image processing method

Abstract: A function approximation processing method comprises inputting and binarizing image data, extracting contour from the binarized image data, estimating tangent points in horizontal and vertical directions from the contour, and approximating contour between adjacent tangent points among the estimated tangent points with a predetermined function, thereby an input object such as a character or a figure can be processed at high speed, and outline data with high image quality can be generated with a reduced number of points.

CBR for state value function approximation in reinforcement learning

Abstract: CBR is one of the techniques that can be applied to the task of approximating a function over high-dimensional, continuous spaces In Reinforcement Learning systems a learning agent is faced with the problem of assessing the desirability of the state it finds itself in If the state space is very large and/or continuous the availability of a suitable mechanism to approximate a value function – which estimates the value of single states – is of crucial importance In this paper, we investigate the use of case-based methods to realise that task The approach we take is evaluated in a case study in robotic soccer simulation

Solving high-order partial differential equations with indirect radial basis function networks

Abstract: This paper reports a new numerical method based on radial basis function networks (RBFNs) for solving high-order partial differential equations (PDEs). The variables and their derivatives in the governing equations are represented by integrated RBFNs. The use of integration in constructing neural networks allows the straightforward implementation of multiple boundary conditions and the accurate approximation of high-order derivatives. The proposed RBFN method is verified successfully through the solution of thin-plate bending and viscous flow problems which are governed by biharmonic equations. For thermally driven cavity flows, the solutions are obtained up to a high Rayleigh number.

XCS with computed prediction for the learning of Boolean functions

Abstract: Computed prediction represents a major shift in learning classifier system research. XCS with computed prediction, based on linear approximates, has been applied so far to function approximation, to single step problems involving continuous payoff functions, and to multi step problems. In this paper we take this new approach in a different direction and apply it to the learning of Boolean functions - a domain characterized by highly discontinuous 0/1000 payoff functions. We also extend it to the case of computed prediction based on functions, borrowed from neural networks, that may be more suitable for 0/1000 payoff problems: the perceptron and the sigmoid. The results we present show that XCSF with linear prediction performs optimally in typical Boolean domains and it allows more compact solutions evolving classifiers that are more general compared with XCS. In addition, perceptron based and sigmoid based prediction can converge slightly faster than linear prediction while producing slightly more compact solutions