Neural Architecture Search with Reinforcement Learning
Citations
38 citations
38 citations
38 citations
Cites background or methods from "Neural Architecture Search with Rei..."
...Typically, Zoph and Le [29] use 800 GPUs and 28 days to discover the convolutional architecture on Cifar-10 dataset by exploring 12,800 individual architectures....
[...]
...e recently proposed neural architecture search (NAS) [29, 30] employs an RNN controller to sample candidate architectures and updating the controller under the guidance of performances of sampled architectures....
[...]
...• GNAS is a non-parametric approach that it refrains from the loop of adopting extra parameters for meta-learning (such as Bayesian optimization (BO) [23] and reinforcement learning (RL) [29, 30])....
[...]
...A variety of approaches including random search [3], Bayesian optimization [11, 16, 23], evolutionary algorithm [21], and reinforcement learning [20, 29] are proposed for neural architecture optimization....
[...]
38 citations
Cites background or methods from "Neural Architecture Search with Rei..."
...Design Space Exploration: In this set of experiment, we compare NANDS framework on CIFAR-10 with the original NAS framework [1], and the state-of-the-art HW-aware NAS [20]....
[...]
...Unlike the implementation in [1] with monocriteria (accuracy), NAS controller in NANDS will take both accuracy and throughput to update RNN....
[...]
...For NAS [1] and HW-aware NAS [20], we report the finally identified architectures....
[...]
...For instance, as shown in Figure 2(a), for the neural network with 15 layers obtained by NAS in [1], we observe that the generated architecture (1) contains up to 8 different types of kernels leading the use of a uniform design to be inefficient; (2) involves a lot of skip connections between layers resulting in a large amount of data movement....
[...]
...Specifically, we use a reinforcement learningbased NAS controller [1, 6] as the backbone to explore NAS space....
[...]
38 citations
References
123,388 citations
111,197 citations
55,235 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]
42,067 citations
31,952 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]