J
Jeffrey Pennington
Researcher at Google
Publications - 84
Citations - 37425
Jeffrey Pennington is an academic researcher from Google. The author has contributed to research in topics: Artificial neural network & Deep learning. The author has an hindex of 32, co-authored 75 publications receiving 28787 citations. Previous affiliations of Jeffrey Pennington include University of Southern California & Princeton University.
Papers
More filters
Proceedings Article
Nonlinear random matrix theory for deep learning
Jeffrey Pennington,Pratik Worah +1 more
TL;DR: In this paper, the authors show that the pointwise nonlinearities typically applied in neural networks can be incorporated into a standard method of proof in random matrix theory known as the moments method, and apply these results to the computation of the asymptotic performance of single-layer random feature methods on a memorization task.
Proceedings ArticleDOI
Bootstrapping six-gluon scattering in planar N=4 super-Yang-Mills theory
TL;DR: In this paper, the hexagon function bootstrap was used to solve six-gluon scattering amplitudes in the large $N_c$ limit of the super-Yang-Mills theorem.
Proceedings Article
The Emergence of Spectral Universality in Deep Networks
TL;DR: This work uses powerful tools from free probability theory to provide a detailed analytic understanding of how a deep network's Jacobian spectrum depends on various hyperparameters including the nonlinearity, the weight and bias distributions, and the depth.
Proceedings Article
A Mean Field Theory of Batch Normalization
TL;DR: The theory shows that gradient signals grow exponentially in depth and that these exploding gradients cannot be eliminated by tuning the initial weight variances or by adjusting the nonlinear activation function, so vanilla batch-normalized networks without skip connections are not trainable at large depths for common initialization schemes.
Proceedings Article
Spherical Random Features for polynomial kernels
TL;DR: A new approximation paradigm is introduced, Spherical Random Fourier features, which circumvents these issues and delivers a compact approximation to polynomial kernels for data on the unit sphere, and has lower variance and typically yield better classification accuracy.