scispace - formally typeset
Open AccessProceedings ArticleDOI

Learning Transferable Architectures for Scalable Image Recognition

Reads0
Chats0
TLDR
NASNet as discussed by the authors proposes to search for an architectural building block on a small dataset and then transfer the block to a larger dataset, which enables transferability and achieves state-of-the-art performance.
Abstract
Developing neural network image classification models often requires significant architecture engineering. In this paper, we study a method to learn the model architectures directly on the dataset of interest. As this approach is expensive when the dataset is large, we propose to search for an architectural building block on a small dataset and then transfer the block to a larger dataset. The key contribution of this work is the design of a new search space (which we call the "NASNet search space") which enables transferability. In our experiments, we search for the best convolutional layer (or "cell") on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell, each with their own parameters to design a convolutional architecture, which we name a "NASNet architecture". We also introduce a new regularization technique called ScheduledDropPath that significantly improves generalization in the NASNet models. On CIFAR-10 itself, a NASNet found by our method achieves 2.4% error rate, which is state-of-the-art. Although the cell is not searched for directly on ImageNet, a NASNet constructed from the best cell achieves, among the published works, state-of-the-art accuracy of 82.7% top-1 and 96.2% top-5 on ImageNet. Our model is 1.2% better in top-1 accuracy than the best human-invented architectures while having 9 billion fewer FLOPS - a reduction of 28% in computational demand from the previous state-of-the-art model. When evaluated at different levels of computational cost, accuracies of NASNets exceed those of the state-of-the-art human-designed models. For instance, a small version of NASNet also achieves 74% top-1 accuracy, which is 3.1% better than equivalently-sized, state-of-the-art models for mobile platforms. Finally, the image features learned from image classification are generically useful and can be transferred to other computer vision problems. On the task of object detection, the learned features by NASNet used with the Faster-RCNN framework surpass state-of-the-art by 4.0% achieving 43.1% mAP on the COCO dataset.

read more

Content maybe subject to copyright    Report

Citations
More filters
Posted Content

On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms

TL;DR: A new variant of the MAML algorithm is proposed called Hessian-free MAM l which preserves all theoretical guarantees of MAMl, without requiring access to second-order information.
Posted Content

XNAS: Neural Architecture Search with Expert Advice

TL;DR: In this article, a novel optimization method for differential neural architecture search, based on the theory of prediction with expert advice, is introduced. But this method requires hard pruning of architectures.
Journal ArticleDOI

Evolving Image Classification Architectures With Enhanced Particle Swarm Optimisation

TL;DR: This work proposes the Swarm Optimized Block Architecture, combined with an enhanced adaptive particle swarm optimization (PSO) algorithm for deep CNN model evolution, where the proposed PSO model is utilized to explore a bespoke search space defined by a simplified block-based structure.
Posted Content

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

TL;DR: A comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs is presented in this paper, which highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions.
Proceedings ArticleDOI

Curls & Whey: Boosting Black-Box Adversarial Attacks

TL;DR: The Curls & Whey attack is proposed, which extends the attack to the targeted misclassification, effectively reducing the difficulty of targeted attacks under black-box condition and shows promising transferability against ensemble models as well as adversarially trained models.
References
More filters
Proceedings ArticleDOI

Deep Residual Learning for Image Recognition

TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article

ImageNet Classification with Deep Convolutional Neural Networks

TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Journal ArticleDOI

Long short-term memory

TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Proceedings Article

Very Deep Convolutional Networks for Large-Scale Image Recognition

TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Proceedings ArticleDOI

ImageNet: A large-scale hierarchical image database

TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Related Papers (5)