J
Jeffrey Pennington
Researcher at Google
Publications - 84
Citations - 37425
Jeffrey Pennington is an academic researcher from Google. The author has contributed to research in topics: Artificial neural network & Deep learning. The author has an hindex of 32, co-authored 75 publications receiving 28787 citations. Previous affiliations of Jeffrey Pennington include University of Southern California & Princeton University.
Papers
More filters
Proceedings Article
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
TL;DR: In this paper, a mean field theory for signal propagation and the conditions for dynamical isometry, the equilibration of singular values of the input-output Jacobian matrix are derived.
Proceedings Article
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
TL;DR: This work uses powerful tools from free probability theory to compute analytically the entire singular value distribution of a deep network's input-output Jacobian, and reveals that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning.
Journal ArticleDOI
Leading singularities and off-shell conformal integrals
James M. Drummond,James M. Drummond,Claude Duhr,Claude Duhr,Burkhard Eden,Paul Heslop,Jeffrey Pennington,Vladimir A. Smirnov,Vladimir A. Smirnov +8 more
TL;DR: In this paper, the authors used the idea of leading singularities to obtain the rational coefficients, the symbol with an appropriate ansatz for its structure, as a means of characterising multiple polylogarithms, and the technique of asymptotic expansion of Feynman integrals in certain limits.
Posted Content
Sensitivity and Generalization in Neural Networks: an Empirical Study
TL;DR: It is found that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the norm of the input-output Jacobian of the network, and that it correlates well with generalization.
Proceedings Article
Geometry of neural network loss surfaces via random matrix theory
Jeffrey Pennington,Yasaman Bahri +1 more
TL;DR: An analytical framework and a set of tools from random matrix theory that allow us to compute an approximation of the distribution of eigenvalues of the Hessian matrix at critical points of varying energy are introduced.