Open AccessPosted Content
Neural Architecture Search with Reinforcement Learning
Barret Zoph,Quoc V. Le +1 more
TLDR
This paper uses a recurrent network to generate the model descriptions of neural networks and trains this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set.Abstract:
Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural networks are still hard to design. In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. On the CIFAR-10 dataset, our method, starting from scratch, can design a novel network architecture that rivals the best human-invented architecture in terms of test set accuracy. Our CIFAR-10 model achieves a test error rate of 3.65, which is 0.09 percent better and 1.05x faster than the previous state-of-the-art model that used a similar architectural scheme. On the Penn Treebank dataset, our model can compose a novel recurrent cell that outperforms the widely-used LSTM cell, and other state-of-the-art baselines. Our cell achieves a test set perplexity of 62.4 on the Penn Treebank, which is 3.6 perplexity better than the previous state-of-the-art model. The cell can also be transferred to the character language modeling task on PTB and achieves a state-of-the-art perplexity of 1.214.read more
Citations
More filters
Posted Content
Rearchitecting Classification Frameworks For Increased Robustness
TL;DR: It is found that applying invariants to the classification task makes robustness and accuracy feasible together, and designs a classification paradigm that leverages these invariances to improve the robustness accuracy trade-off.
Proceedings ArticleDOI
DyRep: Bootstrapping Training with Dynamic Re-parameterization
TL;DR: A dynamic re-parameterization (DyRep) method, which encodes Rep technique into the training process that dynamically evolves the network structures, and applies Rep to enhance their representational capacity to suppress the noisy and redundant operations introduced by Rep.
Journal ArticleDOI
On the generalizability of diffusion MRI signal representations across acquisition parameters, sequences and tissue types: Chronicles of the MEMENTO challenge.
Alberto De Luca,Andrada Ianus,Alexander Leemans,Marco Palombo,Noam Shemesh,Hui Zhang,Daniel C. Alexander,Markus Nilsson,Martijn Froeling,Geert Jan Biessels,Mauro Zucchelli,Matteo Frigo,Enes Albay,Sara Sedlar,Abib Alimi,Samuel Deslauriers-Gauthier,Rachid Deriche,Rutger Fick,Maryam Afzali,Tomasz Pieciak,Fabian Bogusz,Santiago Aja-Fernández,Evren Özarslan,Derek K. Jones,Haoze Chen,Mingwu Jin,Zhijie Zhang,Fengxiang Wang,Vishwesh Nath,Prasanna Parvathaneni,Jan Morez,Jan Sijbers,Ben Jeurissen,Shreyas Fadnavis,Stefan Endres,Ariel Rokem,Eleftherios Garyfallidis,Irina Sánchez,Vesna Prchkovska,Paulo Rodrigues,Bennett A. Landman,Kurt G. Schilling +41 more
TL;DR: The MEMENTO community challenge as mentioned in this paper evaluated the generalizability of existing dMRI signal representations to different diffusion encoding parameters and brain tissue types, highlighting the importance of optimizing and reporting such choices.
Posted Content
AntMan: Sparse Low-Rank Compression To Accelerate RNN Inference
TL;DR: This work develops AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired accuracy.
Journal ArticleDOI
Identification of Heart Sounds with an Interpretable Evolving Fuzzy Neural Network
TL;DR: Results obtained prove that in addition to great assertiveness in detecting heart murmurs, the evolving hybrid model could be concomitant with the extraction of knowledge from data submitted to an intelligent approach.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Proceedings ArticleDOI
Histograms of oriented gradients for human detection
Navneet Dalal,Bill Triggs +1 more
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.