Open AccessPosted Content
Lipschitz regularized Deep Neural Networks converge and generalize
Adam M. Oberman,Jeff Calder +1 more
Reads0
Chats0
TLDR
This paper shows that if the usual fidelity term used in training DNNs is augmented by a Lipschitz regularization term, then the networks converge and generalize.Abstract:
Generalization of deep neural networks (DNNs) is an open problem which, if solved, could impact the reliability and verification of deep neural network architectures. In this paper, we show that if the usual fidelity term used in training DNNs is augmented by a Lipschitz regularization term, then the networks converge and generalize. The convergence is in the limit as the number of data points, n → ∞, while also allowing the network to grow as needed to fit the data. Two regimes are identified: in the case of clean labels, we prove convergence to the label function which corresponds to zero loss, in the case of corrupted labels which we prove convergence to a regularized label function which is the solution of a limiting variational problem. In both cases, a convergence rate is also provided.read more
Citations
More filters
Proceedings Article
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
TL;DR: In this article, the authors theoretically established convergence of PnP-FBS and DNN-ADMM without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers.
Proceedings ArticleDOI
A Deep Value-network Based Approach for Multi-Driver Order Dispatching
TL;DR: This work proposes a deep reinforcement learning based solution for order dispatching and conducts large scale online A/B tests on DiDi's ride-dispatching platform to show that the proposed method achieves significant improvement on both total driver income and user experience related metrics.
Posted Content
Metric Learning for Adversarial Robustness
TL;DR: The authors proposed to regularize the representation space under attack with metric learning to produce more robust classifiers by carefully sampling examples for metric learning, which not only increases robustness, but also detects previously unseen adversarial samples.
Posted Content
Deep Limits of Residual Neural Networks
Matthew Thorpe,Yves van Gennip +1 more
TL;DR: The variational analysis provides a discrete-to-continuum $\Gamma$-convergence result for the objective function of the residual neural network training step to a variational problem constrained by a system of ordinary differential equations; this rigorously connects the discrete setting to a continuum problem.
Posted Content
Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality.
Cory M. Resnick,Rui Gao +1 more
TL;DR: A non-asymptotic framework for analyzing the out-of-sample performance for Wasserstein robust learning and the generalization bound for its related Lipschitz and gradient regularization problems without suffering from the curse of dimensionality is developed.
References
More filters
Journal ArticleDOI
Nonlinear total variation based noise removal algorithms
TL;DR: In this article, a constrained optimization type of numerical algorithm for removing noise from images is presented, where the total variation of the image is minimized subject to constraints involving the statistics of the noise.
Proceedings Article
Intriguing properties of neural networks
Christian Szegedy,Wojciech Zaremba,Ilya Sutskever,Joan Bruna,Dumitru Erhan,Ian Goodfellow,Rob Fergus,Rob Fergus +7 more
TL;DR: It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.
Proceedings Article
Explaining and Harnessing Adversarial Examples
TL;DR: It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
Book
Principles of mathematical analysis
TL;DR: The real and complex number system as discussed by the authors is a real number system where the real number is defined by a real function and the complex number is represented by a complex field of functions.