Neural Architecture Search with Reinforcement Learning
Citations
65 citations
Cites methods from "Neural Architecture Search with Rei..."
...Absent insight into optimal design methods, one can treat architectural details as hyperparameters over which to optimize [42]....
[...]
65 citations
Cites methods from "Neural Architecture Search with Rei..."
...There are also the works that introduce algorithms to increase efficiency of NAS, in particular they include application of those algorithms for designing RNN cells from scratch [49, 28, 34]....
[...]
65 citations
65 citations
64 citations
Cites methods or result from "Neural Architecture Search with Rei..."
...Similar to the AutoML problems of auto-sklearn (Feurer et al., 2015) and NAS (Zoph & Le, 2017; Liu et al., 2019; Yao et al., 2020), this is also a bi-level optimization problem (Colson et al., 2007)....
[...]
...The first includes derivative-free optimization methods, such as reinforcement learning (Zoph & Le, 2017; Baker et al., 2017), genetic programming (Xie & Yuille, 2017), and Bayesian optimization (Bergstra et al., 2011; Snoek et al., 2012)....
[...]
...Gradient-based methods (Bengio, 2000; Liu et al., 2019; Yao et al., 2020) have been popularly used in NAS and hyperparameter optimization....
[...]
...We do not compare with reinforcement learning (Zoph & Le, 2017), as our search problem does not involve a sequence of actions....
[...]
...On NAS problems, gradient-based methods are usually more efficient than derivative-free methods (Liu et al., 2019; Akimoto et al., 2019; Yao et al., 2020)....
[...]
References
123,388 citations
111,197 citations
55,235 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]
42,067 citations
31,952 citations
"Neural Architecture Search with Rei..." refers methods in this paper
...Along with this success is a paradigm shift from feature designing to architecture designing, i.e., from SIFT (Lowe, 1999), and HOG (Dalal & Triggs, 2005), to AlexNet (Krizhevsky et al., 2012), VGGNet (Simonyan & Zisserman, 2014), GoogleNet (Szegedy et al., 2015), and ResNet (He et al., 2016a)....
[...]