Open AccessProceedings Article
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke,Sam Gross,Francisco Massa,Adam Lerer,James Bradbury,Gregory Chanan,Trevor Killeen,Zeming Lin,Natalia Gimelshein,Luca Antiga,Alban Desmaison,Andreas Kopf,Edward Z. Yang,Zachary DeVito,Martin Raison,Alykhan Tejani,Sasank Chilamkurthy,Benoit Steiner,Lu Fang,Junjie Bai,Soumith Chintala +20 more
- Vol. 32, pp 8026-8037
Reads0
Chats0
TLDR
This paper details the principles that drove the implementation of PyTorch and how they are reflected in its architecture, and explains how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.Abstract:
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it was designed from first principles to support an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several commonly used benchmarks.read more
Citations
More filters
Journal ArticleDOI
Array programming with NumPy
Charles R. Harris,K. Jarrod Millman,Stefan van der Walt,Stefan van der Walt,Ralf Gommers,Pauli Virtanen,David Cournapeau,Eric Wieser,Julian Taylor,Sebastian Berg,Nathaniel J. Smith,Robert Kern,Matti Picus,Stephan Hoyer,Marten H. van Kerkwijk,Matthew Brett,Matthew Brett,Allan Haldane,Jaime Fernández del Río,Mark Wiebe,Mark Wiebe,Pearu Peterson,Pierre Gérard-Marchant,Kevin Sheppard,Tyler Reddy,Warren Weckesser,Hameer Abbasi,Christoph Gohlke,Travis E. Oliphant +28 more
TL;DR: In this paper, the authors review how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data, and their evolution into a flexible interoperability layer between increasingly specialized computational libraries is discussed.
Journal ArticleDOI
Array Programming with NumPy
Charles R. Harris,K. Jarrod Millman,Stefan van der Walt,Stefan van der Walt,Ralf Gommers,Pauli Virtanen,David Cournapeau,Eric Wieser,Julian Taylor,Sebastian Berg,Nathaniel J. Smith,Robert Kern,Matti Picus,Stephan Hoyer,Marten H. van Kerkwijk,Matthew Brett,Matthew Brett,Allan Haldane,Jaime Fernández del Río,Mark Wiebe,Mark Wiebe,Pearu Peterson,Pierre Gérard-Marchant,Kevin Sheppard,Tyler Reddy,Warren Weckesser,Hameer Abbasi,Christoph Gohlke,Travis E. Oliphant +28 more
TL;DR: How a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data is reviewed.
Posted Content
End-to-End Object Detection with Transformers
Nicolas Carion,Francisco Massa,Gabriel Synnaeve,Nicolas Usunier,Alexander Kirillov,Sergey Zagoruyko +5 more
TL;DR: This work presents a new method that views object detection as a direct set prediction problem, and demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset.
Journal ArticleDOI
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Fabian Isensee,Fabian Isensee,Paul F. Jaeger,Simon A. A. Kohl,Jens Petersen,Jens Petersen,Klaus H. Maier-Hein,Klaus H. Maier-Hein +7 more
TL;DR: nnU-Net as mentioned in this paper is a deep learning-based segmentation method that automatically configures itself, including preprocessing, network architecture, training and post-processing for any new task.
Book ChapterDOI
End-to-End Object Detection with Transformers
Nicolas Carion,Francisco Massa,Gabriel Synnaeve,Nicolas Usunier,Alexander Kirillov,Sergey Zagoruyko +5 more
TL;DR: DetR as mentioned in this paper proposes a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture to directly output the final set of predictions in parallel.
References
More filters
Automatic differentiation in PyTorch
Adam Paszke,Sam Gross,Soumith Chintala,Gregory Chanan,Edward Z. Yang,Zachary DeVito,Zeming Lin,Alban Desmaison,Luca Antiga,Adam Lerer +9 more
TL;DR: An automatic differentiation module of PyTorch is described — a library designed to enable rapid research on machine learning models that focuses on differentiation of purely imperative programs, with a focus on extensibility and low overhead.
Posted Content
Caffe: Convolutional Architecture for Fast Feature Embedding
Yangqing Jia,Evan Shelhamer,Jeff Donahue,Sergey Karayev,Jonathan Long,Ross Girshick,Sergio Guadarrama,Trevor Darrell +7 more
TL;DR: Caffe as discussed by the authors is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures.
Proceedings ArticleDOI
Data Structures for Statistical Computing in Python
TL;DR: P pandas is a new library which aims to facilitate working with data sets common to finance, statistics, and other related fields and to provide a set of fundamental building blocks for implementing statistical models.
Posted Content
Theano: A Python framework for fast computation of mathematical expressions
Rami Al-Rfou,Guillaume Alain,Amjad Almahairi,Christof Angermueller,Dzmitry Bahdanau,Nicolas Ballas,Frédéric Bastien,Justin Bayer,Anatoly Belikov,Alexander Belopolsky,Yoshua Bengio,Arnaud Bergeron,James Bergstra,Valentin Bisson,Josh Bleecher Snyder,Nicolas Bouchard,Nicolas Boulanger-Lewandowski,Xavier Bouthillier,Alexandre de Brébisson,Olivier Breuleux,Pierre Luc Carrier,Kyunghyun Cho,Jan Chorowski,Paul F. Christiano,Tim Cooijmans,Marc-Alexandre Côté,Myriam Côté,Aaron Courville,Yann N. Dauphin,Olivier Delalleau,Julien Demouth,Guillaume Desjardins,Sander Dieleman,Laurent Dinh,Mélanie Ducoffe,Vincent Dumoulin,Samira Ebrahimi Kahou,Dumitru Erhan,Ziye Fan,Orhan Firat,Mathieu Germain,Xavier Glorot,Ian Goodfellow,Matthew M. Graham,Caglar Gulcehre,Philippe Hamel,Iban Harlouchet,Jean-Philippe Heng,Balázs Hidasi,Sina Honari,Arjun Jain,Sébastien Jean,Kai Jia,Mikhail Korobov,Vivek Kulkarni,Alex Lamb,Pascal Lamblin,Eric Larsen,César Laurent,Sean Lee,Simon Lefrancois,Simon Lemieux,Nicholas Léonard,Zhouhan Lin,Jesse A. Livezey,Cory Lorenz,Jeremiah Lowin,Qianli Ma,Pierre-Antoine Manzagol,Olivier Mastropietro,Robert T. McGibbon,Roland Memisevic,Bart van Merriënboer,Vincent Michalski,Mehdi Mirza,Alberto Orlandi,Chris Pal,Razvan Pascanu,Mohammad Pezeshki,Colin Raffel,Daniel Renshaw,Matthew Rocklin,Adriana Romero,Markus Roth,Peter Sadowski,John Salvatier,François Savard,Jan Schlüter,John Schulman,Gabriel Schwartz,Iulian Vlad Serban,Dmitriy Serdyuk,Samira Shabanian,Étienne Simon,Sigurd Spieckermann,S. Ramana Subramanyam,Jakub Sygnowski,Jérémie Tanguay,Gijs van Tulder,Joseph Turian,Sebastian Urban,Pascal Vincent,Francesco Visin,Harm de Vries,David Warde-Farley,Dustin J. Webb,Matthew Willson,Kelvin Xu,Lijun Xue,Li Yao,Saizheng Zhang,Ying Zhang +111 more
TL;DR: The performance of Theano is compared against Torch7 and TensorFlow on several machine learning models and recently-introduced functionalities and improvements are discussed.