Showing papers on "Piecewise linear function published in 2016"

Two Fully Discrete Schemes for Fractional Diffusion and Diffusion-Wave Equations with Nonsmooth Data

Abstract: We consider initial/boundary value problems for the subdiffusion and diffusion-wave equations involving a Caputo fractional derivative in time. We develop two fully discrete schemes based on the piecewise linear Galerkin finite element method in space and convolution quadrature in time with the generating function given by the backward Euler method/second-order backward difference method, and establish error estimates optimal with respect to the regularity of problem data. These two schemes are first- and second-order accurate in time for both smooth and nonsmooth data. Extensive numerical experiments for two-dimensional problems confirm the convergence analysis and robustness of the schemes with respect to data regularity.

Deep learning with S-shaped rectified linear activation units

Abstract: Rectified linear activation units are important components for state-of-the-art deep convolutional networks. In this paper, we propose a novel S-shaped rectified linear activation unit (SReLU) to learn both convex and non-convex functions, imitating the multiple function forms given by the two fundamental laws, namely the Webner-Fechner law and the Stevens law, in psychophysics and neural sciences. Specifically, SReLU consists of three piecewise linear functions, which are formulated by four learnable parameters. The SReLU is learned jointly with the training of the whole deep network through back propagation. During the training phase, to initialize SReLU in different layers, we propose a "freezing" method to degenerate SReLU into a predefined leaky rectified linear unit in the initial several training epochs and then adaptively learn the good initial values. SReLU can be universally used in the existing deep networks with negligible additional parameters and computation cost. Experiments with two popular CNN architectures, Network in Network and GoogLeNet on scale-various benchmarks including CI-FAR10, CIFAR100, MNIST and ImageNet demonstrate that SReLU achieves remarkable improvement compared to other activation functions.

Optimal Local Reactive Power Control by PV Inverters

Abstract: The high penetration of photovoltaic (PV) generators leads to a voltage rise in the distribution network. To comply with grid standards, distribution system operators need to limit this voltage rise. Reactive power control is one of the most proposed remedies. A popular form of reactive power control is an active power dependent characteristic to define the reactive power control of a PV generator. This standard Q ( P ) characteristic is a simple curve, which is not adapted to the specific situation in the grid. Therefore, this work proposes a method to define the optimal Q ( P ) curve. The optimal Q ( P ) curve is represented as a piecewise constant or a piecewise linear function. The parameters are optimized based on historical smart meter information, to obtain a Q ( P ) curve that keeps the voltage within limits throughout the whole year, with a minimal amount of reactive power. An easy to solve convex optimization problem defines the parameters. The method is applied to unbalanced three-phase four-wire systems. Several simulations with realistic data are performed on an existing distribution network to compare the optimal Q ( P ) curve with standard Q ( P ) and Q ( V ) curves.

Mathematical programming for piecewise linear regression analysis

Abstract: A novel piece-wise linear regression method has been proposed in this work.The method partitions samples into multiple regions from a single attribute.Each region is fitted with a linear regression function.An optimisation model is proposed to decide break-points and regression functions.Benchmark examples have been used to demonstrate its efficiency. In data mining, regression analysis is a computational tool that predicts continuous output variables from a number of independent input variables, by approximating their complex inner relationship. A large number of methods have been successfully proposed, based on various methodologies, including linear regression, support vector regression, neural network, piece-wise regression, etc. In terms of piece-wise regression, the existing methods in literature are usually restricted to problems of very small scale, due to their inherent non-linear nature. In this work, a more efficient piece-wise linear regression method is introduced based on a novel integer linear programming formulation. The proposed method partitions one input variable into multiple mutually exclusive segments, and fits one multivariate linear regression function per segment to minimise the total absolute error. Assuming both the single partition feature and the number of regions are known, the mixed integer linear model is proposed to simultaneously determine the locations of multiple break-points and regression coefficients for each segment. Furthermore, an efficient heuristic procedure is presented to identify the key partition feature and final number of break-points. 7 real world problems covering several application domains have been used to demonstrate the efficiency of our proposed method. It is shown that our proposed piece-wise regression method can be solved to global optimality for datasets of thousands samples, which also consistently achieves higher prediction accuracy than a number of state-of-the-art regression methods. Another advantage of the proposed method is that the learned model can be conveniently expressed as a small number of if-then rules that are easily interpretable. Overall, this work proposes an efficient rule-based multivariate regression method based on piece-wise functions and achieves better prediction performance than state-of-the-arts approaches. This novel method can benefit expert systems in various applications by automatically acquiring knowledge from databases to improve the quality of knowledge base.

Finite element approximation of optimal control problems governed by time fractional diffusion equation

Abstract: In this paper Galerkin finite element approximation of optimal control problems governed by time fractional diffusion equations is investigated. Piecewise linear polynomials are used to approximate the state and adjoint state, while the control is discretized by variational discretization method. A priori error estimates for the semi-discrete approximations of the state, adjoint state and control are derived. Furthermore, we also discuss the fully discrete scheme for the control problems. A finite difference method developed in Lin and Xu (2007) is used to discretize the time fractional derivative. Fully discrete first order optimality condition is developed based on 'first discretize, then optimize' approach. The stability and truncation error of the fully discrete scheme are analyzed. Numerical example is given to illustrate the theoretical findings.

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Abstract: The numerical success of the alternating direction method of multipliers (ADMM) inspires much attention in analyzing its theoretical convergence rate. While there are several results on the iterative complexity results implying sublinear convergence rate for the general case, there are only a few results for the special cases such as linear programming, quadratic programming, and nonlinear programming with strongly convex functions. In this paper, we consider the convergence rate of ADMM when applying to the convex optimization problems that the subdifferentials of the underlying functions are piecewise linear multifunctions, including LASSO, a well-known regression model in statistics, as a special case. We prove that due to its inherent polyhedral structure, a recent global error bound holds for this class of problems. Based on this error bound, we derive the linear rate of convergence for ADMM. We also consider the proximal based ADMM and derive its linear convergence rate.

L2-gain analysis for a class of hybrid systems with applications to reset and event-triggered control: A lifting approach

Abstract: In this paper we study the stability and L 2 -gain properties of a class of hybrid systems that exhibit linear flow dynamics, periodic time-triggered jumps and arbitrary nonlinear jump maps. This class of hybrid systems is relevant for a broad range of applications including periodic event-triggered control, sampled-data reset control, sampled-data saturated control, and certain networked control systems with scheduling protocols. For this class of continuous-time hybrid systems we provide new stability and L 2 -gain analysis methods. Inspired by ideas from lifting we show that the stability and the contractivity in L 2 -sense (meaning that the L 2 -gain is smaller than 1) of the continuous-time hybrid system is equivalent to the stability and the contractivity in L 2 -sense (meaning that the l 2 -gain is smaller than 1) of an appropriate discrete-time nonlinear system. These new characterizations generalize earlier (more conservative) conditions provided in the literature. We show via a reset control example and an event-triggered control application, for which stability and contractivity in L 2 -sense is the same as stability and contractivity in l 2 -sense of a discrete-time piecewise linear system, that the new conditions are significantly less conservative than the existing ones in the literature. Moreover, we show that the existing conditions can be reinterpreted as a conservative l 2 -gain analysis of a discretetime piecewise linear system based on common quadratic storage/Lyapunov functions. These new insights are obtained by the adopted lifting-based perspective on this problem, which leads to computable L 2 -gain (and thus L 2 -gain) conditions, despite the fact that the linearity assumption, which is usually needed in the lifting literature, is not satisfied.

Robust Optimization of Sums of Piecewise Linear Functions with Application to Inventory Problems

Abstract: Robust optimization is a methodology that has gained a lot of attention in the recent years. This is mainly due to the simplicity of the modeling process and ease of resolution even for large scale...

Dynamical Behaviors of Multiple Equilibria in Competitive Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions

Abstract: This paper addresses the problem of coexistence and dynamical behaviors of multiple equilibria for competitive neural networks. First, a general class of discontinuous nonmonotonic piecewise linear activation functions is introduced for competitive neural networks. Then based on the fixed point theorem and theory of strict diagonal dominance matrix, it is shown that under some conditions, such $\boldsymbol {n}$ -neuron competitive neural networks can have $5^{\boldsymbol n}$ equilibria, among which $3^{\boldsymbol n}$ equilibria are locally stable and the others are unstable. More importantly, it is revealed that the neural networks with the discontinuous activation functions introduced in this paper can have both more total equilibria and locally stable equilibria than the ones with other activation functions, such as the continuous Mexican-hat-type activation function and discontinuous two-level activation function. Furthermore, the $3^{\boldsymbol n}$ locally stable equilibria given in this paper are located in not only saturated regions, but also unsaturated regions, which is different from the existing results on multistability of neural networks with multiple level activation functions. A simulation example is provided to illustrate and validate the theoretical findings.

On the importance of normalisation layers in deep learning with piecewise linear activation units

Abstract: Deep feedforward neural networks with piecewise linear activations are currently producing the state-of-the-art results in several public datasets (e.g., CIFAR-10, CIFAR-100, MNIST, and SVHN). The combination of deep learning models and piecewise linear activation functions allows for the estimation of exponentially complex functions with the use of a large number of subnetworks specialized in the classification of similar input examples. During the training process, these subnetworks avoid overfitting with an implicit regularization scheme based on the fact that they must share their parameters with other subnetworks. Using this framework, we have made an empirical observation that can improve even more the performance of such models. We notice that these models assume a balanced initial distribution of data points with respect to the domain of the piecewise linear activation function. If that assumption is violated, then the piecewise linear activation units can degenerate into purely linear activation units, which can result in a significant reduction of their capacity to learn complex functions. Furthermore, as the number of model layers increases, this unbalanced initial distribution makes the model ill-conditioned. Therefore, we propose the introduction of batch normalisation units into deep feedforward neural networks with piecewise linear activations, which drives a more balanced use of these activation units, where each region of the activation function is trained with a relatively large proportion of training samples. Also, this batch normalisation promotes the pre-conditioning of very deep learning models. We show that by introducing maxout and batch normalisation units to the network in network model results in a model that produces classification results that are better than or comparable to the current state of the art in CIFAR-10, CIFAR-100, MNIST, and SVHN datasets.

Bursting phenomenon in a piecewise mechanical system with parameter perturbation in stiffness

Abstract: In this paper the existence condition and generation mechanism of the possible bursting phenomenon in a piecewise mechanical system with different time scales are studied. As an example of mechanical systems, a piecewise linear oscillator with parameter perturbation in stiffness and subject to external excitation is examined. The order gaps between the time scales are considered in the model, which are related to the periodic excitation and the changing rates of the variables. The focus-type periodic bursting oscillation with two time scales is presented, and the corresponding generation mechanism is revealed by using slow–fast analysis method. Furthermore, the analytical solution of piecewise linear subsystem as well as the stability condition of the fast subsystem are explored to explain the transition of bursting behaviors coming from the variation of intrinsic parameter and external excitation. The results about bursting phenomenon and its generation mechanism would provide important theoretical basis on the mechanical manufacturing and engineering practice.

A Petrov--Galerkin Finite Element Method for Fractional Convection-Diffusion Equations

Abstract: In this work, we develop variational formulations of Petrov--Galerkin type for one-dimensional fractional boundary value problems involving either a Riemann--Liouville or Caputo derivative of order $\alpha\in(3/2, 2)$ in the leading term and both convection and potential terms. They arise in the mathematical modeling of asymmetric superdiffusion processes in heterogeneous media. The well-posedness of the formulations and sharp regularity pickup of the variational solutions are established. A novel finite element method (FEM) is developed, which employs continuous piecewise linear finite elements and “shifted” fractional powers for the trial and test space, respectively. The new approach has a number of distinct features: it allows the derivation of optimal error estimates in both the $L^2(D)$ and $H^1(D)$ norms; and on a uniform mesh, the stiffness matrix of the leading term is diagonal and the resulting linear system is well conditioned. Further, in the Riemann--Liouville case, an enriched FEM is propose...

Finite Element Pointwise Results on Convex Polyhedral Domains

Abstract: The main goal of the paper is to establish that the $L^1$ norm of jumps of the normal derivative across element boundaries and the $L^1$ norm of the Laplacian of a piecewise polynomial finite element function can be controlled by corresponding weighted discrete $H^2$ norm on convex polyhedral domains. In the finite element literature such results are only available for piecewise linear elements in two dimensions and the extension to convex polyhedral domains is rather technical. As a consequence of this result, we establish almost pointwise stability of the Ritz projection and the discrete resolvent estimate in $L^\infty$ norm.

Piecewise Linear Approximations for Cure Rate Models and Associated Inferential Issues

Abstract: Cure rate models offer a convenient way to model time-to-event data by allowing a proportion of individuals in the population to be completely cured so that they never face the event of interest (say, death). The most studied cure rate models can be defined through a competing cause scenario in which the random variables corresponding to the time-to-event for each competing causes are conditionally independent and identically distributed while the actual number of competing causes is a latent discrete random variable. The main interest is then in the estimation of the cured proportion as well as in developing inference about failure times of the susceptibles. The existing literature consists of parametric and non/semi-parametric approaches, while the expectation maximization (EM) algorithm offers an efficient tool for the estimation of the model parameters due to the presence of right censoring in the data. In this paper, we study the cases wherein the number of competing causes is either a binary or Poisson random variable and a piecewise linear function is used for modeling the hazard function of the time-to-event. Exact likelihood inference is then developed based on the EM algorithm and the inverse of the observed information matrix is used for developing asymptotic confidence intervals. The Monte Carlo simulation study demonstrates the accuracy of the proposed non-parametric approach compared to the results attained from the true correct parametric model. The proposed model and the inferential method is finally illustrated with a data set on cutaneous melanoma.

Finite Element Method and A Priori Error Estimates for Dirichlet Boundary Control Problems Governed by Parabolic PDEs

Abstract: Finite element approximations of Dirichlet boundary control problems governed by parabolic PDEs on convex polygonal domains are studied in this paper. The existence of a unique solution to optimal control problems is guaranteed based on very weak solution of the state equation and $$L^2(0,T;L^2(\varGamma ))$$L2(0,T?L2(Γ)) as control space. For the numerical discretization of the state equation we use standard piecewise linear and continuous finite elements for the space discretization of the state, while a dG(0) scheme is used for time discretization. The Dirichlet boundary control is realized through a space---time $$L^2$$L2-projection. We consider both piecewise linear, continuous finite element approximation and variational discretization for the controls and derive a priori $$L^2$$L2-error bounds for controls and states. We finally present numerical examples to support our theoretical findings.

Finite volume element method for two-dimensional fractional subdiffusion problems

Abstract: In this paper, a semi-discrete spatial finite volume (FV) method is proposed and analyzed for approximating solutions of anomalous subdiffusion equations involving a temporal fractional derivative of order $\alpha \in (0,1)$ in a two-dimensional convex polygonal domain. Optimal error estimates in $L^\infty(L^2)$- norm is shown to hold. Superconvergence result is proved and as a consequence, it is established that quasi-optimal order of convergence in $L^{\infty}(L^{\infty})$ holds. We also consider a fully discrete scheme that employs FV method in space, and a piecewise linear discontinuous Galerkin method to discretize in temporal direction. It is, further, shown that convergence rate is of order $O(h^2+k^{1+\alpha}),$ where $h$ denotes the space discretizing parameter and $k$ represents the temporal discretizing parameter. Numerical experiments indicate optimal convergence rates in both time and space, and also illustrate that the imposed regularity assumptions are pessimistic.

On Zhang's semipositive metrics

Abstract: Zhang introduced semipositive metrics on a line bundle of a proper variety. In this paper, we generalize such metrics for a line bundle $L$ of a paracompact strictly $K$-analytic space $X$ over any non-archimedean field $K$. We prove various properties in this setting such as density of piecewise $\mathbb{Q}$-linear metrics in the space of continuous metrics on $L$. If $X$ is proper scheme, then we show that algebraic, formal and piecewise linear metrics are the same. Our main result is that on a proper scheme $X$ over an arbitrary non-archimedean field $K$, the set of semipositive model metrics is closed with respect to pointwise convergence generalizing a result from Boucksom, Favre and Jonsson where $K$ was assumed to be discretely valued with residue characteristic $0$.

Capacitated dynamic production and remanufacturing planning under demand and return uncertainty

Abstract: This paper considers a stochastic dynamic multi-product capacitated lot sizing problem with remanufacturing. Finished goods come from two sources: a standard production resource using virgin material and a remanufacturing resource that processes recoverable returns. Both the period demands and the inflow of returns are random. For this integrated stochastic production and remanufacturing problem, we propose a nonlinear model formulation that is approximated by sample averages and a piecewise linear approximation model. In the first approach, the expected values of random variables are replaced by sample averages. The idea of the piecewise linear approximation model is to replace the nonlinear functions with piecewise linear functions. The resulting mixed-integer linear programs are solved to create robust (re)manufacturing plans.

TiO 2 based nanostructured memristor for RRAM and neuromorphic applications: a simulation approach

Abstract: We report simulation of nanostructured memristor device using piecewise linear and nonlinear window functions for RRAM and neuromorphic applications. The linear drift model of memristor has been exploited for the simulation purpose with the linear and non-linear window function as the mathematical and scripting basis. The results evidences that the piecewise linear window function can aptly simulate the memristor characteristics pertaining to RRAM application. However, the nonlinear window function could exhibit the nonlinear phenomenon in simulation only at the lower magnitude of control parameter. This has motivated us to propose a new nonlinear window function for emulating the simulation model of the memristor. Interestingly, the proposed window function is scalable up to f(x) = 1 and exhibits the nonlinear behavior at higher magnitude of control parameter. Moreover, the simulation results of proposed nonlinear window function are encouraging and reveals the smooth nonlinear change from LRS to HRS and vice versa and therefore useful for the neuromorphic applications.

A Band-Limited Canonical Piecewise-Linear Function-Based Behavioral Model for Wideband Power Amplifiers

Abstract: A band-limited canonical piecewise-linear function (CPWL)-based model is proposed for wideband power amplifiers (PAs). The model has a similar structure of the band-limited dynamic deviation reduction (DDR) Volterra series model but without high-order terms and long finite impulse response filters, which are replaced by CPWL. The model has lower complexity and more flexibility than the band-limited DDR Volterra series model and the model parameters can be estimated by the least-square method. Experimental results show that the proposed model nearly gives the same normalized mean square error and digital predistortion performance as the band-limited Volterra series when a wideband PA is excited by a 5-carrier long-term evolution advanced signal of 100-MHz bandwidth.