Linear approximation

About: Linear approximation is a research topic. Over the lifetime, 3901 publications have been published within this topic receiving 74764 citations.

Stochastic dynamics of supervised learning

Abstract: The stochastic evolution of adiabatic (slow) backpropagation training of a neural network is discussed and a Fokker-Planck equation for the post-training distribution function in the network space is derived. The distribution obtained differs from the one given by Radons et al. (1990). Studying the character of the post-training distribution, the authors find that, except under very special circumstances, the distribution will be non-Gibbsian. The validity of the present approach is tested on a simple backpropagation learning system in one dimension, which can be solved analytically as well. Implications of the Fokker-Planck approach for general situations are examined in the local linear approximation. Surprisingly they find that the post-training distribution is isotropic close to its peak, hence simpler than the corresponding Gibbs distribution.

A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid Function

Abstract: The sigmoid function is a widely used nonlinear activation function in neural networks. In this article, we present a modular approximation methodology for efficient fixed-point hardware implementation of the sigmoid function. Our design consists of three modules: piecewise linear (PWL) approximation as the initial solution, Taylor series approximation of the exponential function, and Newton–Raphson method-based approximation as the final solution. Its modularity enables the designer to flexibly choose the most appropriate approximation method for each module separately. Performance evaluation results indicate that our work strikes an appropriate balance among the objectives of approximation accuracy, hardware resource utilization, and performance.

MAP decoder with correction function in LOG-MAX approximation

Abstract: A decoder and a decoding method can perform log-sum corrections by means of linear approximation. The decoder and decoding method provide improved decoding speed, with a reduced circuit dimension without adversely affecting the decoding performance of the circuit. The decoder comprises a linear approximation circuit 68 added to obtain log likelihoods and adapted to compute the correction term expressed by a one-dimensional function of a variable by linear approximation. The linear approximation circuit 68 computes the correction term by log-sum corrections by means of linear approximation using function F = -a P - Q + b, where the coefficient -a representing the gradient of the function and the coefficient b representing the intercept are expressed by a power exponent of 2. More specifically, when the coefficients a and b are expressed respectively by -2-k and 2m -1, the linear approximation circuit 68 discards from the lowest bit the k-th lowest bits, bit-shifts the absolute value data P - Q and then inverts the m bits from the k+1-th lowest bit to the m+k-th lowest bit by means of inverter 91.

Finite time estimation through a continuous-discrete observer

Abstract: We study two broad classes of nonlinear time-varying continuous time systems with outputs. For the first class, we build an observer in the case where a state dependent disturbance a↵ects the linear approximation. When the disturbances are the zero functions, our observer provides exact values of the state at all times larger than a suitable finite time, and it provides an approximate estimate when there are nonzero disturbances, so our observers are called finite time observers. We use this construction, which is of interest for its own sake, to design a globally exponentially stabilizing dynamic output feedback for a family of nonlinear systems whose outputs are only available on some finite time intervals. Our simulations illustrate the ecacy of our methods.

Comparison of four gravity models

Abstract: Inertial navigation systems must utilize a valid gravity model in order to accurately mechanize the navigation equations. Four commonly used gravity models are compared and their fundamental differences are summarized in this paper. The 4 gravity models (listed with increasing accuracy and/or validity) are as follows. 1. Low altitude gravity model is an approximation to the normal gravity vector. 2. J/sub 2/ gravity model uses an approximation to the normal graviational potential (an infinite series of spherical harmonics) and generates the gravity vector in the Earth-Centered-Earth-Fixed (ECEF) frame. These components are transformed to the local level navigation. 3. Normal gravity model generates a gravity vector normal to the reference ellipsoid on the ellipsoidal surface by definition; above the ellipsoid, the gravity vector has a nonzero north component. 4. General gravity approximation model uses a multiple-term approximation to the true gravitational potential (a double sum of infinite terms). All three components are nonzero at or above the ellipsoidal surface. The components depend upon longitude. This paper gives an introduction to the concept of gravity potential and normal gravity potential. Gravity vector direction comparison for these four gravity models are made at all latitudes, longitudes and altitudes. Coordinate dependency of the gravity components is also examined.