Proceedings ArticleDOI
When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Shiqiang Wang,Tiffany Tuor,Theodoros Salonidis,Kin K. Leung,Christian Makaya,Ting He,Kevin S. Chan +6 more
- pp 63-71
TLDR
This paper analyzes the convergence rate of distributed gradient descent from a theoretical point of view, and proposes a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget.Abstract:
Emerging technologies and applications including Internet of Things (IoT), social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent based approaches. We analyze the convergence rate of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.read more
Citations
More filters
Journal ArticleDOI
Federated Machine Learning: Concept and Applications
TL;DR: This work introduces a comprehensive secure federated-learning framework, which includes horizontal federated learning, vertical federatedLearning, and federated transfer learning, and provides a comprehensive survey of existing works on this subject.
Posted Content
Federated Machine Learning: Concept and Applications
TL;DR: This work proposes building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.
Journal ArticleDOI
Deep Learning With Edge Computing: A Review
Jiasi Chen,Xukan Ran +1 more
TL;DR: This paper will provide an overview of applications where deep learning is used at the network edge, discuss various approaches for quickly executing deep learning inference across a combination of end devices, edge servers, and the cloud, and describe the methods for training deep learning models across multiple edge devices.
Journal ArticleDOI
All one needs to know about fog computing and related edge computing paradigms: A complete survey
Ashkan Yousefpour,Caleb Fung,Tam T. Nguyen,Krishna P. Kadiyala,Fatemeh Jalali,Amirreza Niakanlahiji,Jian Kong,Jason P. Jue +7 more
TL;DR: This paper provides a tutorial on fog computing and its related computing paradigms, including their similarities and differences, and provides a taxonomy of research topics in fog computing.
Proceedings ArticleDOI
Federated Learning over Wireless Networks: Optimization Model Design and Analysis
TL;DR: This work formulates a Federated Learning over wireless network as an optimization problem FEDL that captures both trade-offs and obtains the globally optimal solution by charactering the closed-form solutions to all sub-problems, which give qualitative insights to problem design via the obtained optimal FEDl learning time, accuracy level, and UE energy cost.
References
More filters
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Book
Deep Learning
TL;DR: Deep learning as mentioned in this paper is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts, and it is used in many applications such as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames.
Posted Content
Communication-Efficient Learning of Deep Networks from Decentralized Data
TL;DR: This work presents a practical method for the federated learning of deep networks based on iterative model averaging, and conducts an extensive empirical evaluation, considering five different model architectures and four datasets.
Book
Understanding Machine Learning: From Theory To Algorithms
TL;DR: The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way in an advanced undergraduate or beginning graduate course.
Related Papers (5)
Secure and resilient distributed machine learning under adversarial environments
Rui Zhang,Quanyan Zhu +1 more