Neural Architecture Search with Reinforcement Learning
Citations
84 citations
Cites background or methods from "Neural Architecture Search with Rei..."
...The earliest of these approaches use reinforcement learning (RL) to learn promising architectures in a discrete space (Zoph and Le, 2016), whereas others have successfully modeled the problem in a continuous manner (Liu et al....
[...]
...Like recent methods in NAS (Xie and Yuille, 2017; Zoph and Le, 2016; Baker et al., 2016), DARTS represents networks as a directed acyclic graph for a given computation cell (see Figure 1(a))....
[...]
...Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3576–3581, Hong Kong, China, November 3–7, 2019. c©2019 Association for Computational Linguistics 3576 In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing....
[...]
...It is the first time that differentiable NAS methods are adopted in NLP tasks other than language modeling....
[...]
...Also, we apply differentiable NAS to named entity recognition (NER)....
[...]
84 citations
Cites background or methods from "Neural Architecture Search with Rei..."
...The GPT vocabulary size is 10k larger compared to the ENAS’s vocabulary....
[...]
...Due to its greedy nature, CAS is simpler and faster than previous architecture search algorithms (Zoph and Le, 2016; Pham et al., 2018; Liu et al., 2018)....
[...]
..., 2018) as well as language modeling (Zoph and Le, 2016; Pham et al., 2018; Liu et al., 2018) in NLP....
[...]
...The reason of the search cost of GPT-CAS on WT-2 is higher than ENAS is three-fold: 1....
[...]
...ENAS by (Pham et al., 2018) also leverages a reinforcement learning search method....
[...]
84 citations
84 citations
84 citations
Cites background or methods from "Neural Architecture Search with Rei..."
...been reported [27, 28] that searching a network on CIFAR takes hundreds of GPUs for a lengthy period....
[...]
...Several methods [27, 1, 28, 26] have been proposed....
[...]
...In recent works [27, 1, 28, 26], automatic search methods have been proposed....
[...]
References
123,388 citations
111,197 citations
55,235 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]
42,067 citations
31,952 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]