Powering One-shot Topological NAS with Stabilized Share-parameter Proxy
Ronghao Guo,Chen Lin,Chuming Li,Keyu Tian,Ming Sun,Lu Sheng,Junjie Yan +6 more
- pp 625-641
TLDR
The difficulties for architecture searching in such a complex space has been eliminated by the proposed stabilized share-parameter proxy, which employs Stochastic Gradient Langevin Dynamics to enable fast shared parameter sampling, so as to achieve stabilized measurement of architecture performance even in search space with complex topological structures.Citations
More filters
Posted Content
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
Lingxi Xie,Xin Chen,Kaifeng Bi,Longhui Wei,Yuhui Xu,Zhengsu Chen,Lanfei Wang,An Xiao,Jianlong Chang,Xiaopeng Zhang,Qi Tian +10 more
TL;DR: A literature review on the application of NAS to computer vision problems is provided and existing approaches are summarized into several categories according to their efforts in bridging the gap.
Proceedings ArticleDOI
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
TL;DR: HR-NAS as mentioned in this paper adopts a multi-branch architecture that provides convolutional encoding of multiple feature resolutions and proposes an efficient fine-grained search strategy to train HR-NAS, which effectively explores the search space, and finds optimal architectures given various tasks and computation resources.
Journal ArticleDOI
Can GPT-4 Perform Neural Architecture Search?
TL;DR: Zheng et al. as discussed by the authors investigated the potential of GPT-4 to perform Neural Architecture Search (NAS) and proposed GENIUS, a black-box optimiser that leverages the generative capabilities as a black box optimiser to quickly navigate the architecture search space, pinpoint promising candidates, and iteratively refine these candidates.
Posted Content
Evaluating Efficient Performance Estimators of Neural Architectures.
TL;DR: In this article, the authors conduct an extensive and organized assessment of OSEs and ZSEs on three NAS benchmarks: NAS-Bench-101/201/301, and reveal that they have certain biases and variances.
Journal ArticleDOI
Efficient Evaluation Methods for Neural Architecture Search: A Survey
TL;DR: In this paper , the authors comprehensively survey the evaluation methods of Deep Neural Networks (DNNs) and provide a detailed analysis to motivate the further development of this research direction, and divide the existing evaluation methods into four categories based on the number of DNNs trained for constructing these evaluation methods.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings ArticleDOI
Going deeper with convolutions
Christian Szegedy,Wei Liu,Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich +8 more
TL;DR: Inception as mentioned in this paper is a deep convolutional neural network architecture that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
Journal ArticleDOI
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Proceedings ArticleDOI
Densely Connected Convolutional Networks
TL;DR: DenseNet as mentioned in this paper proposes to connect each layer to every other layer in a feed-forward fashion, which can alleviate the vanishing gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.
Proceedings ArticleDOI
Rethinking the Inception Architecture for Computer Vision
TL;DR: In this article, the authors explore ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization.