Open AccessPosted Content
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Reads0
Chats0
TLDR
This work proposes a small DNN architecture called SqueezeNet, which achieves AlexNet-level accuracy on ImageNet with 50x fewer parameters and is able to compress to less than 0.5MB (510x smaller than AlexNet).Abstract:
Recent research on deep neural networks has focused primarily on improving accuracy. For a given accuracy level, it is typically possible to identify multiple DNN architectures that achieve that accuracy level. With equivalent accuracy, smaller DNN architectures offer at least three advantages: (1) Smaller DNNs require less communication across servers during distributed training. (2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car. (3) Smaller DNNs are more feasible to deploy on FPGAs and other hardware with limited memory. To provide all of these advantages, we propose a small DNN architecture called SqueezeNet. SqueezeNet achieves AlexNet-level accuracy on ImageNet with 50x fewer parameters. Additionally, with model compression techniques we are able to compress SqueezeNet to less than 0.5MB (510x smaller than AlexNet).
The SqueezeNet architecture is available for download here: this https URLread more
Citations
More filters
Posted Content
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard,Menglong Zhu,Bo Chen,Dmitry Kalenichenko,Weijun Wang,Tobias Weyand,M. Andreetto,Hartwig Adam +7 more
TL;DR: This work introduces two simple global hyper-parameters that efficiently trade off between latency and accuracy and demonstrates the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.
Proceedings Article
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
TL;DR: Deep Compression as mentioned in this paper proposes a three-stage pipeline: pruning, quantization, and Huffman coding to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy.
Posted Content
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Mingxing Tan,Quoc V. Le +1 more
TL;DR: A new scaling method is proposed that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient and is demonstrated the effectiveness of this method on scaling up MobileNets and ResNet.
Posted Content
YOLOv4: Optimal Speed and Accuracy of Object Detection
TL;DR: This work uses new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, C mBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100.
Proceedings ArticleDOI
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
TL;DR: ShuffleNet as discussed by the authors utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy, and achieves an actual speedup over AlexNet while maintaining comparable accuracy.
References
More filters
Journal ArticleDOI
EIE: efficient inference engine on compressed deep neural network
TL;DR: In this paper, the authors proposed an energy efficient inference engine (EIE) that performs inference on a compressed network model and accelerates the resulting sparse matrix-vector multiplication with weight sharing.
Posted Content
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Tianqi Chen,Mu Li,Yutian Li,Min Lin,Naiyan Wang,Minjie Wang,Tianjun Xiao,Bing Xu,Chiyuan Zhang,Zheng Zhang +9 more
TL;DR: The API design and the system implementation of MXNet are described, and it is explained how embedding of both symbolic expression and tensor operation is handled in a unified fashion.
Posted Content
cuDNN: Efficient Primitives for Deep Learning
Sharan Chetlur,Cliff Woolley,Philippe Vandermersch,Jonathan Cohen,John Tran,Bryan Catanzaro,Evan Shelhamer +6 more
TL;DR: A library similar in intent to BLAS, with optimized routines for deep learning workloads, that contains routines for GPUs, and similarly to the BLAS library, could be implemented for other platforms.
Proceedings Article
Torch7: A Matlab-like Environment for Machine Learning
TL;DR: Torch7 is a versatile numeric computing framework and machine learning library that extends Lua that can easily be interfaced to third-party software thanks to Lua’s light interface.
Proceedings ArticleDOI
From captions to visual concepts and back
Hao Fang,Saurabh Gupta,Forrest Iandola,Rupesh Kumar Srivastava,Li Deng,Piotr Dollár,Jianfeng Gao,Xiaodong He,Margaret Mitchell,John Platt,C. Lawrence Zitnick,Geoffrey Zweig +11 more
TL;DR: This paper used multiple instance learning to train visual detectors for words that commonly occur in captions, including many different parts of speech such as nouns, verbs, and adjectives, which serve as conditional inputs to a maximum-entropy language model.