Dropout as a Bayesian approximation: representing model uncertainty in deep learning
Citations
2,842 citations
Cites background from "Dropout as a Bayesian approximation..."
...An example is the recent work of Gal and Ghahramani (2015) who show that model confidence can be estimated via sampling the dropout mask....
[...]
2,261 citations
1,991 citations
1,883 citations
1,569 citations
Cites background from "Dropout as a Bayesian approximation..."
...…in that the former involves attacks against a legitimate ML system by an adversary (e.g. a criminal tries to fool a face recognition system), while the latter involves attacks by an ML system controlled by an adversary (e.g. a criminal trains a “smart hacker” system to break into a website)....
[...]
References
111,197 citations
"Dropout as a Bayesian approximation..." refers methods in this paper
...Finally, we used mini-batches of size 32 and the Adam optimiser [38]....
[...]
...Finally, we used mini-batches of size 32 and the Adam optimiser (Kingma & Ba, 2014)....
[...]
42,067 citations
33,597 citations
"Dropout as a Bayesian approximation..." refers background or methods in this paper
...Dropout is used in many models in deep learning as a way to avoid over-fitting (Srivastava et al., 2014), and our interpretation suggests that dropout approximately integrates over the models’ weights....
[...]
...Furthermore, our results carry to other variants of dropout as well (such as drop-connect (Wan et al., 2013), multiplicative Gaussian noise (Srivastava et al., 2014), etc.)....
[...]
...In this paper we give a complete theoretical treatment of the link between Gaussian processes and dropout, and develop the tools necessary to represent uncertainty in deep learning....
[...]
23,074 citations
20,769 citations
"Dropout as a Bayesian approximation..." refers background or methods in this paper
...Recent advances in variational inference introduced new techniques into the field such as sampling-based variational inference and stochastic variational inference (Blei et al., 2012; Kingma & Welling, 2013; Rezende et al., 2014; Titsias & LázaroGredilla, 2014; Hoffman et al., 2013)....
[...]
...Recent advances in variational inference introduced new techniques such as sampling-based variational inference and stochastic variational inference [21, 22, 23, 24, 25]....
[...]