Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Shiqiang Wang,Tiffany Tuor,Theodoros Salonidis,Kin K. Leung,Christian Makaya,Ting He,Kevin S. Chan +6 more
Reads0
Chats0
TLDR
In this paper, the authors consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place, and propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget.Abstract:
Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.read more
Citations
More filters
Proceedings ArticleDOI
Multi-Level Branched Regularization for Federated Learning
Jinkyu Kim,Geeho Kim,Bohyung Han +2 more
TL;DR: A novel architectural regularization technique is proposed that constructs multiple auxiliary branches in each local model by grafting local and global subnetworks at several different levels and that learns the representations of the main pathway in the local model congru-ent to the auxiliary hybrid pathways via online knowledge distillation.
Journal ArticleDOI
Communication-Efficient Federated Edge Learning for NR-U-Based IIoT Networks
TL;DR: In this article , the authors proposed a communication-efficient federated edge learning (FEL) mechanism for NR-U-based industrial IoT networks, which aims to select data importance IIoT devices for local training under relatively sufficient unlicensed resources.
Posted Content
Adaptive Task Allocation for Asynchronous Federated and Parallelized Mobile Edge Learning
Umair Mohammad,Sameh Sorour +1 more
TL;DR: The approach reduces the gradient staleness and can offer better accuracy than the synchronous scheme and the asynchronous scheme with equal task allocation, and can solve the NP-hard problem efficiently with available solvers.
Journal ArticleDOI
Incentive mechanism for collaborative distributed learning in Artificial Intelligence of Things
TL;DR: In this article, a collaborative distributed learning has been proposed to let the clients collaboratively train a machine learning model together with their local data in a distributed manner, where multiple parameter servers (PSs) compete with each other to motivate clients to contribute model training.
Journal ArticleDOI
Distributed Learning on Mobile Devices: A New Approach to Data Mining in the Internet of Things
TL;DR: In this paper, the authors proposed a novel architecture, GREAT, where each node chooses partners to share local model parameters according to link reliability, and an optimization problem is developed by taking the reliability threshold as the variable of controlling the resources' overhead.