Showing papers on "Function (mathematics) published in 2020"

Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities

Abstract: In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fake samples by designated margins, while learning a generator alternately to produce realistic samples by minimizing their losses. The LS-GAN further regularizes its loss function with a Lipschitz regularity condition on the density of real data, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN. We will further present a Generalized LS-GAN (GLS-GAN) and show it contains a large family of regularized GAN models, including both LS-GAN and Wasserstein GAN, as its special cases. Compared with the other GAN models, we will conduct experiments to show both LS-GAN and GLS-GAN exhibit competitive ability in generating new images in terms of the Minimum Reconstruction Error (MRE) assessed on a separate test set. We further extend the LS-GAN to a conditional form for supervised and semi-supervised learning problems, and demonstrate its outstanding performance on image classification tasks.

Diachronic Embedding for Temporal Knowledge Graph Completion

Abstract: Knowledge graphs (KGs) typically contain temporal facts indicating relationships among entities at different times. Due to their incompleteness, several approaches have been proposed to infer new facts for a KG based on the existing ones–a problem known as KG completion. KG embedding approaches have proved effective for KG completion, however, they have been developed mostly for static KGs. Developing temporal KG embedding models is an increasingly important problem. In this paper, we build novel models for temporal KG completion through equipping static models with a diachronic entity embedding function which provides the characteristics of entities at any point in time. This is in contrast to the existing temporal KG embedding approaches where only static entity features are provided. The proposed embedding function is model-agnostic and can be potentially combined with any static model. We prove that combining it with SimplE, a recent model for static KG embedding, results in a fully expressive model for temporal KG completion. Our experiments indicate the superiority of our proposal compared to existing baselines.

Obstacles to Variational Quantum Optimization from Symmetry Protection

Abstract: The quantum approximate optimization algorithm (QAOA) employs variational states generated by a parameterized quantum circuit to maximize the expected value of a Hamiltonian encoding a classical cost function. Whether or not the QAOA can outperform classical algorithms in some tasks is an actively debated question. Our work exposes fundamental limitations of the QAOA resulting from the symmetry and the locality of variational states. A surprising consequence of our results is that the classical Goemans-Williamson algorithm outperforms the QAOA for certain instances of MaxCut, at any constant level. To overcome these limitations, we propose a nonlocal version of the QAOA and give numerical evidence that it significantly outperforms the standard QAOA for frustrated Ising models.

On a Fractional Operator Combining Proportional and Classical Differintegrals

Abstract: The Caputo fractional derivative has been one of the most useful operators for modelling non-local behaviours by fractional differential equations. It is defined, for a differentiable function f ( t ) , by a fractional integral operator applied to the derivative f ′ ( t ) . We define a new fractional operator by substituting for this f ′ ( t ) a more general proportional derivative. This new operator can also be written as a Riemann–Liouville integral of a proportional derivative, or in some important special cases as a linear combination of a Riemann–Liouville integral and a Caputo derivative. We then conduct some analysis of the new definition: constructing its inverse operator and Laplace transform, solving some fractional differential equations using it, and linking it with a recently described bivariate Mittag-Leffler function.

Event-Triggered Adaptive Control of Saturated Nonlinear Systems With Time-Varying Partial State Constraints

Abstract: This paper investigates the problem of event-triggered adaptive control for a class of nonlinear systems subject to asymmetric input saturation and time-varying partial state constraints. To facilitate analyzing the influence of asymmetric input saturation, the saturation function is converted into a linear form with respect to control input. To achieve the objective that partial states do not exceed the constraints, a more general form of Lyapunov function is offered. Different from some existing results about output/full state constraints, the proposed scheme only requires that the partial states satisfy the time-varying constraints. Moreover, an event-triggered scheme with a varying threshold is designed to reduce the communication burden. With the time-varying asymmetric barrier Lyapunov functions, a novel event-triggered control scheme is developed, which ensures that partial states are without violation of required constraints and the tracking error converges to a small neighborhood of the origin despite appearing as saturated phenomenon. Eventually, the theoretic results are confirmed by two examples.

Sequential minimal optimization for quantum-classical hybrid algorithms

Abstract: We propose a sequential minimal optimization method for quantum-classical hybrid algorithms, which converges faster, is robust against statistical error, and is hyperparameter-free. Specifically, the optimization problem of the parameterized quantum circuits is divided into solvable subproblems by considering only a subset of the parameters. In fact, if we choose a single parameter, the cost function becomes a simple sine curve with period $2\pi$, and hence we can exactly minimize with respect to the chosen parameter. Furthermore, even in general cases, the cost function is given by a simple sum of trigonometric functions with certain periods and hence can be minimized by using a classical computer. By repeatedly performing this procedure, we can optimize the parameterized quantum circuits so that the cost function becomes as small as possible. We perform numerical simulations and compare the proposed method with existing gradient-free and gradient-based optimization algorithms. We find that the proposed method substantially outperforms the existing optimization algorithms and converges to a solution almost independent of the initial choice of the parameters. This accelerates almost all quantum-classical hybrid algorithms readily and would be a key tool for harnessing near-term quantum devices.

Cooperative Training of Descriptor and Generator Networks

Abstract: This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.

Information Volume of Mass Function

Abstract: Given a probability distribution, its corresponding information volume is Shannon entropy. However, how to determine the information volume of a given mass function is still an open issue. Based on Deng entropy, the information volume of mass function is presented in this paper. Given a mass function, the corresponding information volume is larger than its uncertainty measured by Deng entropy. In addition, when the cardinal of the frame of discernment is identical, both the total uncertainty case and the BPA distribution of the maximum Deng entropy have the same information volume. Some numerical examples are illustrated to show the efficiency of the proposed information volume of mass function.

A Note on a New Type of Degenerate Bernoulli Numbers

Abstract: Studying degenerate versions of various special polynomials has became an active area of research and has yielded many interesting arithmetic and combinatorial results. Here we introduce a degenerate version of the polylogarithm function, the so-called degenerate polylogarithm function. Then we construct a new type of degenerate Bernoulli polynomial and number, the so-called degenerate poly-Bernoulli polynomial and number, by using the degenerate polylogarithm function, and derive several properties concerning the degenerate poly-Bernoulli numbers.

Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning

Abstract: We present an algorithm, HOMER, for exploration and reinforcement learning in rich observation environments that are summarizable by an unknown latent state space. The algorithm interleaves representation learning to identify a new notion of kinematic state abstraction with strategic exploration to reach new states using the learned abstraction. The algorithm provably explores the environment with sample complexity scaling polynomially in the number of latent states and the time horizon, and, crucially, with no dependence on the size of the observation space, which could be infinitely large. This exploration guarantee further enables sample-efficient global policy optimization for any reward function. On the computational side, we show that the algorithm can be implemented efficiently whenever certain supervised learning problems are tractable. Empirically, we evaluate HOMER on a challenging exploration problem, where we show that the algorithm is exponentially more sample efficient than standard reinforcement learning baselines.

The Discrete Gaussian for Differential Privacy

Abstract: A key tool for building differentially private systems is adding Gaussian noise to the output of a function evaluated on a sensitive dataset. Unfortunately, using a continuous distribution presents several practical challenges. First and foremost, finite computers cannot exactly represent samples from continuous distributions, and previous work has demonstrated that seemingly innocuous numerical errors can entirely destroy privacy. Moreover, when the underlying data is itself discrete (e.g., population counts), adding continuous noise makes the result less interpretable. With these shortcomings in mind, we introduce and analyze the discrete Gaussian in the context of differential privacy. Specifically, we theoretically and experimentally show that adding discrete Gaussian noise provides essentially the same privacy and accuracy guarantees as the addition of continuous Gaussian noise. We also present an simple and efficient algorithm for exact sampling from this distribution. This demonstrates its applicability for privately answering counting queries, or more generally, low-sensitivity integer-valued queries.

A Lyapunov-Like Characterization of Predefined-Time Stability

Abstract: This article studies Lyapunov-like conditions to ensure a class of dynamical systems to exhibit predefined-time stability. The origin of a dynamical system is predefined-time stable if it is fixed-time stable, and an upper bound of the settling-time function can be arbitrarily chosen a priori through a suitable selection of the system parameters. We show that the studied Lyapunov-like conditions allow us to demonstrate the equivalence between previous Lyapunov theorems for predefined-time stability for autonomous systems. Moreover, the obtained Lyapunov-like theorem is extended for analyzing the property of predefined-time ultimate boundedness with predefined bound, which is useful when analyzing uncertain dynamical systems. Therefore, the proposed results constitute a general framework for analyzing the predefined-time stability, and they also unify a broad class of systems that present the predefined-time stability property. On the other hand, the proposed framework is used to design robust controllers for affine control systems, which induce predefined-time stability (predefined-time ultimate boundedness of the solutions) w.r.t. to some desired manifold. A simulation example is presented to show the behavior of a developed controller, especially regarding the settling time estimation.

JT gravity, KdV equations and macroscopic loop operators

Abstract: We study the thermal partition function of Jackiw-Teitelboim (JT) gravity in asymptotically Euclidean AdS2 background using the matrix model description recently found by Saad, Shenker and Stanford [ arXiv:1903.11115 ]. We show that the partition function of JT gravity is written as the expectation value of a macroscopic loop operator in the old matrix model of 2d gravity in the background where infinitely many couplings are turned on in a specific way. Based on this expression we develop a very efficient method of computing the partition function in the genus expansion as well as in the low temperature expansion by making use of the Korteweg-de Vries constraints obeyed by the partition function. We have computed both these expansions up to very high orders using this method. It turns out that we can take a low temperature limit with the ratio of the temperature and the genus counting parameter held fixed. We find the first few orders of the expansion of the free energy in a closed form in this scaling limit. We also study numerically the behavior of the eigenvalue density and the Baker-Akhiezer function using the results in the scaling limit.

Can transfer function and Bode diagram be obtained from Sumudu transform

Abstract: In the last past year researchers have relied on the ability of Laplace transform to solve partial, ordinary linear equations with great success. Important analysis in signal analysis including the transfer function, Bode diagram, Nyquist plot and Nichols plot are obtained based on the Laplace transform. The output of the analysis depends only on the results obtained from Laplace transform. However, one weakness of Laplace transform is that the Laplace transform of even function is odd while the Laplace transform of an old function is even which is lack of conservation of properties. On the other hand there exist a similar integral transform known as Sumudu transform has the ability to conserve the properties of the function from real space to complex space. The question that arises in the work, is the following: Can we apply the Sumudu transform to construct new transfer functions that will lead to new Bode, Nichols and Nyquist plots? this question is answered in this work.

Electric Load Forecasting by Hybrid Self-Recurrent Support Vector Regression Model With Variational Mode Decomposition and Improved Cuckoo Search Algorithm

Abstract: Accurate electric load forecasting is critical not only in preventing wasting electricity production but also in facilitating the reasonable integration of clean energy resources. Hybridizing the variational mode decomposition (VMD) method, the chaotic mapping mechanism, and improved meta-heuristic algorithm with the support vector regression (SVR) model is crucial to preventing the premature problem and providing satisfactory forecasting accuracy. To solve the boundary handling problem of the cuckoo search (CS) algorithm in the cuckoo birds' searching processes, this investigation proposes a simple method, called the out-bound-back mechanism, to help those out-bounded cuckoo birds return to their previous (the most recent iteration) optimal location. The proposed self-recurrent (SR) mechanism, inspired from the combination of Jordan's and Elman's recurrent neural networks, is used to collect comprehensive and useful information from the training and testing data. Therefore, the self-recurrent mechanism is hybridized with the SVR-based model. Ultimately, this investigation presents the VMD-SR-SVRCBCS model, by hybridizing the VMD method, the SVR model with the self-recurrent mechanism, the Tent chaotic mapping function, the out-bound-back mechanism, and the cuckoo search algorithm. Two real-world datasets are used to demonstrate that the proposed model has greater forecasting accuracy than other models.

GE-GAN: A novel deep learning framework for road traffic state estimation

Abstract: Traffic state estimation is a crucial elemental function in Intelligent Transportation Systems (ITS) However, the collected traffic state data are often incomplete in the real world In this paper, a novel deep learning framework is proposed to use information from adjacent links to estimate road traffic states First, the representation of the road network is realized based on graph embedding (GE) Second, with this representation information, the generative adversarial network (GAN) is applied to generate the road traffic state information in real-time Finally, two typical road networks in Caltrans District 7 and Seattle area are adopted as cases study Experimental results indicate that the estimated road traffic state data of the detectors have higher accuracy than the data estimated by other models

Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold

Abstract: We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of proble...

Neural network approximation

Abstract: Neural networks (NNs) are the method of choice for building learning algorithms. They are now being investigated for other numerical tasks such as solving high-dimensional partial differential equations. Their popularity stems from their empirical success on several challenging learning problems (computer chess/Go, autonomous navigation, face recognition). However, most scholars agree that a convincing theoretical explanation for this success is still lacking. Since these applications revolve around approximating an unknown function from data observations, part of the answer must involve the ability of NNs to produce accurate approximations. This article surveys the known approximation properties of the outputs of NNs with the aim of uncovering the properties that are not present in the more traditional methods of approximation used in numerical analysis, such as approximations using polynomials, wavelets, rational functions and splines. Comparisons are made with traditional approximation methods from the viewpoint of rate distortion, i.e. error versus the number of parameters used to create the approximant. Another major component in the analysis of numerical approximation is the computational time needed to construct the approximation, and this in turn is intimately connected with the stability of the approximation algorithm. So the stability of numerical approximation using NNs is a large part of the analysis put forward. The survey, for the most part, is concerned with NNs using the popular ReLU activation function. In this case the outputs of the NNs are piecewise linear functions on rather complicated partitions of the domain of f into cells that are convex polytopes. When the architecture of the NN is fixed and the parameters are allowed to vary, the set of output functions of the NN is a parametrized nonlinear manifold. It is shown that this manifold has certain space-filling properties leading to an increased ability to approximate (better rate distortion) but at the expense of numerical stability. The space filling creates the challenge to the numerical method of finding best or good parameter choices when trying to approximate.

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

Abstract: The classical bias-variance trade-off predicts that bias decreases and variance increase with model complexity, leading to a U-shaped risk curve. Recent work calls this into question for neural networks and other over-parameterized models, for which it is often observed that larger models generalize better. We provide a simple explanation for this by measuring the bias and variance of neural networks: while the bias is monotonically decreasing as in the classical theory, the variance is unimodal or bell-shaped: it increases then decreases with the width of the network. We vary the network architecture, loss function, and choice of dataset and confirm that variance unimodality occurs robustly for all models we considered. The risk curve is the sum of the bias and variance curves and displays different qualitative shapes depending on the relative scale of bias and variance, with the double descent curve observed in recent literature as a special case. We corroborate these empirical results with a theoretical analysis of two-layer linear networks with random first layer. Finally, evaluation on out-of-distribution data shows that most of the drop in accuracy comes from increased bias while variance increases by a relatively small amount. Moreover, we find that deeper models decrease bias and increase variance for both in-distribution and out-of-distribution data.

Extended Feature Pyramid Network for Small Object Detection

Abstract: Small object detection remains an unsolved challenge because it is hard to extract information of small objects with only a few pixels. While scale-level corresponding detection in feature pyramid network alleviates this problem, we find feature coupling of various scales still impairs the performance of small objects. In this paper, we propose extended feature pyramid network (EFPN) with an extra high-resolution pyramid level specialized for small object detection. Specifically, we design a novel module, named feature texture transfer (FTT), which is used to super-resolve features and extract credible regional details simultaneously. Moreover, we design a foreground-background-balanced loss function to alleviate area imbalance of foreground and background. In our experiments, the proposed EFPN is efficient on both computation and memory, and yields state-of-the-art results on small traffic-sign dataset Tsinghua-Tencent 100K and small category of general object detection dataset MS COCO.

Joint Transmission Map Estimation and Dehazing Using Deep Networks

Abstract: Single image haze removal is an extremely challenging problem due to its inherent ill-posed nature. Several prior-based and learning-based methods have been proposed in the literature to solve this problem and they have achieved visually appealing results. However, most of the existing methods assume constant atmospheric light model and tend to follow a two-step procedure involving prior-based methods for estimating transmission map followed by calculation of dehazed image using the closed form solution. In this paper, we relax the constant atmospheric light assumption and propose a novel unified single image dehazing network that jointly estimates the transmission map and performs dehazing. In other words, our new approach provides an end-to-end learning framework, where the inherent transmission map and dehazed result are learned jointly from the loss function. The extensive experiments evaluated on synthetic and real datasets with challenging hazy images demonstrate that the proposed method achieves significant improvements over the state-of-the-art methods.

A method for evaluating definite integrals in terms of special functions with examples

Abstract: We present a method using contour integration to derive definite integrals and their associated infinite sums which can be expressed as a special function. We give a proof of the basic equation and some examples of the method. The advantage of using special functions is their analytic continuation which widens the range of the parameters of the definite integral over which the formula is valid. We give as examples definite integrals of logarithmic functions times a trigonometric function. In various cases these generalizations evaluate to known mathematical constants such as Catalan's constant and $\pi$.

Variational Quantum Linear Solver: A Hybrid Algorithm for Linear Systems

Abstract: Solving linear systems of equations is central to many engineering and scientific fields. Several quantum algorithms have been proposed for linear systems, where the goal is to prepare $|x\rangle$ such that $A|x\rangle \propto |b\rangle$. While these algorithms are promising, the time horizon for their implementation is long due to the required quantum circuit depth. In this work, we propose a variational hybrid quantum-classical algorithm for solving linear systems, with the aim of reducing the circuit depth and doing much of the computation classically. We propose a cost function based on the overlap between $|b\rangle$ and $A|x\rangle$, and we derive an operational meaning for this cost in terms of the solution precision $\epsilon$. We also introduce a quantum circuit to estimate this cost, while showing that this cost cannot be efficiently estimated classically. Using Rigetti's quantum computer, we successfully implement our algorithm up to a problem size of $32 \times 32$. Furthermore, we numerically find that the complexity of our algorithm scales efficiently in both $1/\epsilon$ and $\kappa$, with $\kappa$ the condition number of $A$. Our algorithm provides a heuristic for quantum linear systems that could make this application more near term.

Answers to three conjectures on convexity of three functions involving complete elliptic integrals of the first kind

Abstract: In the article, we prove that the function x → (1-x)pK(√x) is logarithmically concave on (0,1) if and only if p ≥ 7/32, the function x → K(√x)/log(1+4/√1-x) is convex on (0,1) and the function x → d2/dx2 [K(√x)- log (1+4/√1-x) is absolutely monotonic on (0,1), where K(x) = ∫π/20 (1-x2 sin2t)-1/2 dt (0 < x < 1) is the complete elliptic integral of the first kind.

Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimension

Abstract: Value function approximation has demonstrated phenomenal empirical success in reinforcement learning (RL). Nevertheless, despite a handful of recent progress on developing theory for RL with linear function approximation, the understanding of general function approximation schemes largely remains missing. In this paper, we establish a provably efficient RL algorithm with general value function approximation. We show that if the value functions admit an approximation with a function class $\mathcal{F}$, our algorithm achieves a regret bound of $\widetilde{O}(\mathrm{poly}(dH)\sqrt{T})$ where $d$ is a complexity measure of $\mathcal{F}$ that depends on the eluder dimension [Russo and Van Roy, 2013] and log-covering numbers, $H$ is the planning horizon, and $T$ is the number interactions with the environment. Our theory generalizes recent progress on RL with linear value function approximation and does not make explicit assumptions on the model of the environment. Moreover, our algorithm is model-free and provides a framework to justify the effectiveness of algorithms used in practice.