Book ChapterDOI
Data Aggregation Aware Routing for Distributed Training
Zhaohong Chen,Xin Long,Yalan Wu,Long Chen,Jigang Wu,Shuangyin Liu +5 more
- pp 241-250
TLDR
In this article, a data aggregation aware routing problem is formulated as a mixed-integer non-linear programming problem, and it is proved to be NP-hard, and then a greedy routing algorithm is proposed to solve the formulated problem, by transmitting the data to the closest aggregation node in greedy to reduce the network overhead.Abstract:
For distributed training, the communication overhead for parameter synchronization is heavy in the network. Data aggregation can efficiently alleviate network overheads. However, existing works on data aggregation are based on the streaming message data, which can not well adapt to the discrete communication for parameter synchronization. This paper formulates a data aggregation aware routing problem, with the objective of minimizing training finishing time for global model under the constraint of cache capacity. The problem is formulated as a mixed-integer non-linear programming problem, and it is proved to be NP-Hard. Then we propose a data aggregation aware routing algorithm to solve the formulated problem, by transmitting the data to the closest aggregation node in greedy to reduce the network overhead. Simulation results show that, the proposed algorithm can reduce average training finishing time by \(74\%\), and it can reduce the network overhead by \(33\%\) on average, compared with the shortest path algorithm.read more
Citations
More filters
References
More filters
Proceedings ArticleDOI
Going deeper with convolutions
Christian Szegedy,Wei Liu,Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich +8 more
TL;DR: Inception as mentioned in this paper is a deep convolutional neural network architecture that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
Posted Content
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard,Menglong Zhu,Bo Chen,Dmitry Kalenichenko,Weijun Wang,Tobias Weyand,M. Andreetto,Hartwig Adam +7 more
TL;DR: This work introduces two simple global hyper-parameters that efficiently trade off between latency and accuracy and demonstrates the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.
Proceedings ArticleDOI
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
TL;DR: ShuffleNet as discussed by the authors utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy, and achieves an actual speedup over AlexNet while maintaining comparable accuracy.
Proceedings ArticleDOI
Compressed residual-VGG16 CNN model for big data places image recognition
TL;DR: The claim that the proposed model inherits the best aspects of VGG16 and further improves upon it is supported and the proposed framework can be more easily adapted and fully integrated with the residual learning for compressing various other contemporary deep learning convolutional neural network models.
Journal ArticleDOI
A Note on Dijkstra's Shortest Path Algorithm
TL;DR: An assertion that Dijkstra's algorithm for shortest paths (adapted to allow arcs of negative weight) runs in O(n)(supscrpt) steps is disproved by showing a set of networks which take O (O) 2 (n) 3 steps.
Related Papers (5)
Routing Explicit Side Information for Data Compression in Wireless Sensor Networks
Huiyu Luo,Gregory J. Pottie +1 more
Research: A new distributed routing algorithm for supporting delay-sensitive applications
Quan Sun,Horst Langendörfer +1 more