Open AccessPosted Content
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
TLDR
XNOR-Nets as discussed by the authors approximate convolutions using primarily binary operations, which results in 58x faster convolutional operations and 32x memory savings, and outperforms BinaryConnect and BinaryNets by large margins on ImageNet.Abstract:
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32x memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58x faster convolutional operations and 32x memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is only 2.9% less than the full-precision AlexNet (in top-1 measure). We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than 16% in top-1 accuracy.read more
Citations
More filters
Posted Content
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard,Menglong Zhu,Bo Chen,Dmitry Kalenichenko,Weijun Wang,Tobias Weyand,M. Andreetto,Hartwig Adam +7 more
TL;DR: This work introduces two simple global hyper-parameters that efficiently trade off between latency and accuracy and demonstrates the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.
Proceedings ArticleDOI
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
TL;DR: ShuffleNet as discussed by the authors utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy, and achieves an actual speedup over AlexNet while maintaining comparable accuracy.
Journal ArticleDOI
Recent advances in convolutional neural networks
Jiuxiang Gu,Zhenhua Wang,Jason Kuen,Lianyang Ma,Amir Shahroudy,Bing Shuai,Ting Liu,Xingxing Wang,Gang Wang,Jianfei Cai,Tsuhan Chen +10 more
TL;DR: A broad survey of the recent advances in convolutional neural networks can be found in this article, where the authors discuss the improvements of CNN on different aspects, namely, layer design, activation function, loss function, regularization, optimization and fast computation.
Posted Content
Searching for MobileNetV3.
Andrew Howard,Mark Sandler,Grace Chu,Liang-Chieh Chen,Bo Chen,Mingxing Tan,Weijun Wang,Yukun Zhu,Ruoming Pang,Vijay K. Vasudevan,Quoc V. Le,Hartwig Adam +11 more
TL;DR: This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art of MobileNets.
Journal ArticleDOI
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
TL;DR: In this paper, the authors provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs, and discuss various hardware platforms and architectures that support DNN, and highlight key trends in reducing the computation cost of deep neural networks either solely via hardware design changes or via joint hardware and DNN algorithm changes.
References
More filters
Posted Content
Bitwise Neural Networks
Minje Kim,Paris Smaragdis +1 more
TL;DR: The proposed Bitwise Neural Network (BNN) is especially suitable for resource-constrained environments, since it replaces either floating or fixed-point arithmetic with significantly more efficient bitwise operations.
Posted Content
Low precision storage for deep learning
TL;DR: It is found that very low precision storage is sufficient not just for running trained networks but also for training them.
Journal ArticleDOI
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
Carlo Baldassi,Alessandro Ingrosso,Carlo Lucibello,Luca Saglietti,Riccardo Zecchina,Riccardo Zecchina +5 more
TL;DR: It is shown that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance, and that these synaptic configurations are robust to perturbations and generalize better than typical solutions.
Book ChapterDOI
Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network
TL;DR: A novel architecture called location-sensitive deep network LSDN is proposed for synthesizing images across domains that integrates intensity feature from image voxels and spatial information in a principled manner and is computationally efficient, e.g. 26× faster than other sparse representation based methods.
Posted Content
Big Neural Networks Waste Capacity
Yann N. Dauphin,Yoshua Bengio +1 more
TL;DR: In this paper, the authors show that the first-order gradient descent method fails at this regime, leading to underfitting in ImageNet LSVRC-2010 and show that this may be due to the fact there are highly diminishing returns for capacity in terms of training error.