scispace - formally typeset
Search or ask a question
Posted Content

Neural Architecture Search with Reinforcement Learning

Barret Zoph1, Quoc V. Le1
05 Nov 2016-arXiv: Learning-
TL;DR: This paper uses a recurrent network to generate the model descriptions of neural networks and trains this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set.
Abstract: Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural networks are still hard to design. In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. On the CIFAR-10 dataset, our method, starting from scratch, can design a novel network architecture that rivals the best human-invented architecture in terms of test set accuracy. Our CIFAR-10 model achieves a test error rate of 3.65, which is 0.09 percent better and 1.05x faster than the previous state-of-the-art model that used a similar architectural scheme. On the Penn Treebank dataset, our model can compose a novel recurrent cell that outperforms the widely-used LSTM cell, and other state-of-the-art baselines. Our cell achieves a test set perplexity of 62.4 on the Penn Treebank, which is 3.6 perplexity better than the previous state-of-the-art model. The cell can also be transferred to the character language modeling task on PTB and achieves a state-of-the-art perplexity of 1.214.
Citations
More filters
Posted Content
TL;DR: RNet-DQN is presented, a solution that uses Reinforcement Learning to address the problem of improving the robustness of graphs in the presence of random and targeted removals of nodes, and relies on changes in the estimated robustness as a reward signal and Graph Neural Networks for representing states.
Abstract: Graphs can be used to represent and reason about real world systems and a variety of metrics have been devised to quantify their global characteristics. An important property is robustness to failures and attacks, which is relevant for the infrastructure and communication networks that power modern society. Prior work on making topological modifications to a graph, e.g., adding edges, in order to increase robustness is typically based on local and spectral properties or a shallow search since robustness is expensive to compute directly. However, such strategies are necessarily suboptimal. In this work, we present RNet-DQN, an approach for constructing networks that uses Reinforcement Learning to address improving the robustness of graphs to random and targeted removals of nodes. In particular, the approach relies on changes in the estimated robustness as a reward signal and Graph Neural Networks for representing states. Experiments on synthetic and real-world graphs show that this approach can deliver performance superior to existing methods while being much cheaper to evaluate and generalizing to out-of-sample graphs, as well as to larger out-of-distribution graphs in some cases. The approach is readily applicable to optimizing other global structural properties of graphs.

3 citations


Cites methods from "Neural Architecture Search with Rei..."

  • ...Recently, approaches have emerged that use RL [Zoph and Le, 2017] or evolutionary algorithms [Liu et al....

    [...]

  • ...Recently, approaches have emerged that use RL [Zoph and Le, 2017] or evolutionary algorithms [Liu et al., 2018] to discover architectures that can deliver state-of-the-art performance on several computer vision benchmarks....

    [...]

Book ChapterDOI
07 Apr 2021
TL;DR: In this paper, the authors reproduce a genetic programming approach to design convolutional neural networks called CGP-CNN, which achieves a final accuracy of ∼90.6% \pm 0.005
Abstract: Designing Neural Network Architectures requires expert knowledge and extensive parameter searches. Neural Architecture Search (NAS) aims to change that by automating the design process. It is important that these approaches are reproducible so they can be used in real-life scenarios. In our work, we reproduce a genetic programming approach to designing convolutional neural networks called CGP-CNN. We show that this is difficult and requires many changes to the training scheme, reducing real-life applicability. We achieve a final accuracy of \(90.6\% \pm 0.005\), substantially lower than the reported \(93.7\% \pm 0.005\). This negates some of the benefits of using CGP-CNN for NAS. We establish a random search as a consensus baseline and show that it produces similar results to the evolutionary method of CGP-CNN. To assess the adaptability and generality of the presented algorithm, it is applied to CIFAR-100 and SVHN with a final accuracy of 63.1% and 95.6%, respectively. We extend the investigated NAS by two methods for predicting candidate fitnesses from partial learning curves. This improves CGP-CNN runtime efficiency by a factor of 1.69.

3 citations

Proceedings ArticleDOI
31 May 2022
TL;DR: This work employs several saliency map algorithms to provide explainability and insight into understanding black-box classifier decisions, and collects and implements numerous data augmentation routines and training improvements to further improve model domain adaptation performance from synthetic to measured data.
Abstract: Deep neural networks have recently demonstrated state-of-the-art accuracy on public Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) benchmark datasets. While attaining competitive accuracy on benchmark datasets is a necessary feature, it is important to characterize other facets of new SAR ATR algorithms. We extend this recent work by demonstrating not only improved state-of-the-art accuracy, but that contemporary deep neural networks can achieve several algorithmic traits beyond competitive accuracy which are necessitated by operational deployment scenarios. First, we employ several saliency map algorithms to provide explainability and insight into understanding black-box classifier decisions. Second, we collect and implement numerous data augmentation routines and training improvements both from the computer vision literature and specific to SAR ATR data in order to further improve model domain adaptation performance from synthetic to measured data, achieving a 99.26% accuracy on SAMPLE validation with a simple network architecture. Finally, we survey model reproducibility and performance variability under domain adaptation from synthetic to measured data, demonstrating potential consequences of training on only synthetic data.

3 citations

Proceedings ArticleDOI
23 Aug 2022
TL;DR: Experiments on different datasets show that the adaptive architecture assembled by MANAS outperforms static global architectures and borrows the idea from modularized neural logic reasoning and consider three basic logical operation modules: AND, OR, NOT.
Abstract: Human intelligence is able to first learn some basic skills for solving basic problems and then assemble such basic skills into complex skills for solving complex or new problems. For example, the basic skills "dig hole,'' "put tree,'' "backfill'' and "watering'' compose a complex skill "plant a tree''. Besides, some basic skills can be reused for solving other problems. For example, the basic skill "dig hole'' not only can be used for planting a tree, but also can be used for mining treasures, building a drain, or landfilling. The ability to learn basic skills and reuse them for various tasks is very important for humans because it helps to avoid learning too many skills for solving each individual task, and makes it possible to solve a compositional number of tasks by learning just a few number of basic skills, which saves a considerable amount of memory and computational power in the human brain. We believe that machine intelligence should also capture the ability of learning basic skills and reusing them by composing into complex skills. In computer science language, each basic skill is a "module'', which is a reusable network that has a concrete meaning and performs a concrete basic operation. The modules are assembled into a bigger "model'' for doing a more complex task. The assembling procedure is adaptive to the input or task, i.e., for a given task, the modules should be assembled into the most suitable model for solving the given task. As a result, different inputs/tasks could have different assembled models. In this work, we take recommender system as an example and propose Modularized Adaptive Neural Architecture Search (MANAS) to demonstrate the above idea. Neural Architecture Search (NAS) has shown its power in discovering superior neural architectures. However, existing NAS mostly focus on searching for a global architecture regardless of the specific input, i.e., the architecture is not adaptive to the input. In this work, we borrow the idea from modularized neural logic reasoning and consider three basic logical operation modules: AND, OR, NOT. Meanwhile, making recommendations for each user is considered as a task. MANAS automatically assembles the logical operation modules into a network architecture tailored for the given user. As a result, a personalized neural architecture is assembled for each user to make recommendations for the user, which means that the resulting neural architecture is adaptive to the model's input (i.e., the user's past behaviors). Experiments on different datasets show that the adaptive architecture assembled by MANAS outperforms static global architectures. Further experiments and empirical analysis provide insights to the effectiveness of MANAS. The code is open-source at https://github.com/TalonCB/MANAS.

3 citations

Posted Content
TL;DR: This work introduces the challenging task of estimating a set of tactile physical properties from visual information and develops a cross-modal framework comprised of an adversarial objective and a novel visuo-tactile joint classification loss.
Abstract: The connection between visual input and tactile sensing is critical for object manipulation tasks such as grasping and pushing. In this work, we introduce the challenging task of estimating a set of tactile physical properties from visual information. We aim to build a model that learns the complex mapping between visual information and tactile physical properties. We construct a first of its kind image-tactile dataset with over 400 multiview image sequences and the corresponding tactile properties. A total of fifteen tactile physical properties across categories including friction, compliance, adhesion, texture, and thermal conductance are measured and then estimated by our models. We develop a cross-modal framework comprised of an adversarial objective and a novel visuo-tactile joint classification loss. Additionally, we develop a neural architecture search framework capable of selecting optimal combinations of viewing angles for estimating a given physical property.

3 citations


Cites methods from "Neural Architecture Search with Rei..."

  • ...Inspired by neural architecture search (NAS) approaches [44,35,73,36], we learn to select a combination of viewing angles instead of choosing a handcrafted selection strategy similar to Xue et al....

    [...]

References
More filters
Proceedings ArticleDOI
27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

123,388 citations

Proceedings Article
01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.

111,197 citations

Proceedings Article
04 Sep 2014
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

55,235 citations


"Neural Architecture Search with Rei..." refers methods in this paper

  • ...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....

    [...]

Journal ArticleDOI
01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Abstract: Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

42,067 citations

Proceedings ArticleDOI
20 Jun 2005
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.
Abstract: We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

31,952 citations


"Neural Architecture Search with Rei..." refers methods in this paper

  • ...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....

    [...]