I
Ilya Sutskever
Researcher at OpenAI
Publications - 137
Citations - 294374
Ilya Sutskever is an academic researcher from OpenAI. The author has contributed to research in topics: Artificial neural network & Reinforcement learning. The author has an hindex of 75, co-authored 131 publications receiving 235539 citations. Previous affiliations of Ilya Sutskever include Google & University of Toronto.
Papers
More filters
Posted Content
Adding Gradient Noise Improves Learning for Very Deep Networks
Arvind Neelakantan,Luke Vilnis,Quoc V. Le,Ilya Sutskever,Lukasz Kaiser,Karol Kurach,James Martens +6 more
TL;DR: This paper explores the low-overhead and easy-to-implement optimization technique of adding annealed Gaussian noise to the gradient, which it is found surprisingly effective when training these very deep architectures.
Posted Content
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford,Jong Wook Kim,Chris Hallacy,Aditya Ramesh,Gabriel Goh,Sandhini Agarwal,Girish Sastry,Amanda Askell,Pamela Mishkin,Jack Clark,Gretchen Krueger,Ilya Sutskever +11 more
TL;DR: In this article, a pre-training task of predicting which caption goes with which image is used to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.
Posted Content
Variational Lossy Autoencoder
Xi Chen,Diederik P. Kingma,Tim Salimans,Yan Duan,Prafulla Dhariwal,John Schulman,Ilya Sutskever,Pieter Abbeel +7 more
TL;DR: Li et al. as mentioned in this paper combine VAE with neural autoregressive models such as RNN, MADE and PixelRNN/CNN to learn a global representation for 2D images that describes only global structure and discards information about detailed texture.
Proceedings Article
Variational Lossy Autoencoder
Xi Chen,Diederik P. Kingma,Tim Salimans,Yan Duan,Prafulla Dhariwal,John Schulman,Ilya Sutskever,Pieter Abbeel +7 more
TL;DR: This paper presents a simple but principled method to learn global representations by combining Variational Autoencoder (VAE) with neural autoregressive models such as RNN, MADE and PixelRNN/CNN with greatly improve generative modeling performance of VAEs.
Dissertation
Training recurrent neural networks
TL;DR: A new probabilistic sequence model that combines Restricted Boltzmann Machines and RNNs is described, more powerful than similar models while being less difficult to train, and a random parameter initialization scheme is described that allows gradient descent with momentum to train Rnns on problems with long-term dependencies.