Linear Regression Analysis

The communication and networking field is hungry for machine learning decision-making solutions to replace the traditional model-driven approaches that proved to be not rich enough for seizing the ever-growing complexity and heterogeneity of the modern systems in the field. Traditional machine learning solutions assume the existence of (cloud-based) central entities that are in charge of processing the data. Nonetheless, the difficulty of accessing private data, together with the high cost of transmitting raw data to the central entity gave rise to a decentralized machine learning approach called Federated Learning . The main idea of federated learning is to perform an on-device collaborative training of a single machine learning model without having to share the raw training data with any third-party entity. Although few survey articles on federated learning already exist in the literature, the motivation of this survey stems from three essential observations. The first one is the lack of a fine-grained multi-level classification of the federated learning literature, where the existing surveys base their classification on only one criterion or aspect. The second observation is that the existing surveys focus only on some common challenges, but disregard other essential aspects such as reliable client selection, resource management and training service pricing. The third observation is the lack of explicit and straightforward directives for researchers to help them design future federated learning solutions that overcome the state-of-the-art research gaps. To address these points, we first provide a comprehensive tutorial on federated learning and its associated concepts, technologies and learning approaches. We then survey and highlight the applications and future directions of federated learning in the domain of communication and networking. Thereafter, we design a three-level classification scheme that first categorizes the federated learning literature based on the high-level challenge that they tackle. Then, we classify each high-level challenge into a set of specific low-level challenges to foster a better understanding of the topic. Finally, we provide, within each low-level challenge, a fine-grained classification based on the technique used to address this particular challenge. For each category of high-level challenges, we provide a set of desirable criteria and future research directions that are aimed to help the research community design innovative and efficient future solutions. To the best of our knowledge, our survey is the most comprehensive in terms of challenges and techniques it covers and the most fine-grained in terms of the multi-level classification scheme it presents.

/pdf/federated-machine-learning-survey-multi-level-classification-qdw5jveh7p.pdf

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

The Internet of Things (IoT) will be ripe for the deployment of novel machine learning algorithms for both network and application management. However, given the presence of massively distributed and private datasets, it is challenging to use classical centralized learning algorithms in the IoT. To overcome this challenge, federated learning can be a promising solution that enables on-device machine learning without the need to migrate the private end-user data to a central cloud. In federated learning, only learning model updates are transferred between end-devices and the aggregation server. Although federated learning can offer better privacy preservation than centralized machine learning, it has still privacy concerns. In this paper, first, we present the recent advances of federated learning towards enabling federated learning-powered IoT applications. A set of metrics such as sparsification, robustness, quantization, scalability, security, and privacy, is delineated in order to rigorously evaluate the recent advances. Second, we devise a taxonomy for federated learning over IoT networks. Third, we propose two IoT use cases of dispersed federated learning that can offer better privacy preservation than federated learning. Finally, we present several open research challenges with their possible solutions.

Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges

The Internet of Things (IoT) will be ripe for the deployment of novel machine learning algorithm for both network and application management. However, given the presence of massively distributed and private datasets, it is challenging to use classical centralized learning algorithms in the IoT. To overcome this challenge, federated learning can be a promising solution that enables on-device machine learning without the need to migrate the private end-user data to a central cloud. In federated learning, only learning model updates are transferred between end-devices and the aggregation server. Although federated learning can offer better privacy preservation than centralized machine learning, it has still privacy concerns. In this paper, first, we present the recent advances of federated learning towards enabling federated learning-powered IoT applications. A set of metrics such as sparsification, robustness, quantization, scalability, security, and privacy, is delineated in order to rigorously evaluate the recent advances. Second, we devise a taxonomy for federated learning over IoT networks. Finally, we present several open research challenges with their possible solutions.

Deep Learning (DL) has been widely proposed for botnet attack detection in Internet of Things (IoT) networks. However, the traditional Centralized DL (CDL) method cannot be used to detect previously unknown (zero-day) botnet attack without breaching the data privacy rights of the users. In this paper, we propose Federated Deep Learning (FDL) method for zero-day botnet attack detection to avoid data privacy leakage in IoT edge devices. In this method, an optimal Deep Neural Network (DNN) architecture is employed for network traffic classification. A model parameter server remotely coordinates the independent training of the DNN models in multiple IoT edge devices, while Federated Averaging (FedAvg) algorithm is used to aggregate local model updates. A global DNN model is produced after a number of communication rounds between the model parameter server and the IoT edge devices. Zero-day botnet attack scenarios in IoT edge devices is simulated with the Bot-IoT and N-BaIoT data sets. Experiment results show that FDL model: (a) detects zero-day botnet attacks with high classification performance; (b) guarantees data privacy and security; (c) has low communication overhead (d) requires low memory space for the storage of training data; and (e) has low network latency. Therefore, FDL method outperformed CDL, Localized DL, and Distributed DL methods in this application scenario.

/pdf/federated-deep-learning-for-zero-day-botnet-attack-detection-45wlmxt52t.pdf

Federated Deep Learning for Zero-Day Botnet Attack Detection in IoT Edge Devices

Driven by privacy concerns and the visions of deep learning, the last four years have witnessed a paradigm shift in the applicability mechanism of machine learning (ML). An emerging model, called federated learning (FL), is rising above both centralized systems and on-site analysis, to be a new fashioned design for ML implementation. It is a privacy-preserving decentralized approach, which keeps raw data on devices and involves local ML training while eliminating data communication overhead. A federation of the learned and shared models is then performed on a central server to aggregate and share the built knowledge among participants. This article starts by examining and comparing different ML-based deployment architectures, followed by in-depth and in-breadth investigation on FL. Compared to the existing reviews in the field, we provide in this survey a new classification of FL topics and research fields based on thorough analysis of the main technical challenges and current related work. In this context, we elaborate comprehensive taxonomies covering various challenging aspects, contributions, and trends in the literature, including core system models and designs, application areas, privacy and security, and resource management. Furthermore, we discuss important challenges and open research directions toward more robust FL systems.

A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond

As an alternative centralized systems, which may prevent data to be stored in a central repository due to its privacy and/or abundance, federated learning (FL) is nowadays a game changer addressing both privacy and cooperative learning. It succeeds in keeping training data on the devices, while sharing locally computed then globally aggregated models throughout several communication rounds. The selection of clients participating in FL process is currently at complete/quasi randomness. However, the heterogeneity of the client devices within Internet-of-Things environment and their limited communication and computation resources might fail to complete the training task, which may lead to many discarded learning rounds affecting the model accuracy. In this article, we propose FedMCCS, a multicriteria-based approach for client selection in FL. All of the CPU, memory, energy, and time are considered for the clients resources to predict whether they are able to perform the FL task. Particularly, in each round, the number of clients in FedMCCS is maximized to the utmost, while considering each client resources and its capability to successfully train and send the needed updates. The conducted experiments show that FedMCCS outperforms the other approaches by: 1) reducing the number of communication rounds to reach the intended accuracy; 2) maximizing the number of clients; 3) handling the least number of discarded rounds; and 4) optimizing the network traffic.

FedMCCS: Multicriteria Client Selection Model for Optimal IoT Federated Learning

Conventional systems are usually constrained to store data in a centralized location. This restriction has either precluded sensitive data from being shared or put its privacy on the line. Alternatively, federated learning (FL) has emerged as a promising privacy-preserving paradigm for exchanging model parameters instead of private data of Internet of Things (IoT) devices known as clients. FL trains a global model by communicating local models generated by selected clients throughout many communication rounds until ensuring high learning performance. In these settings, the FL performance highly depends on selecting the best available clients. This process is strongly related to the quality of their models and their training data. Such selection-based schemes have not been explored yet, particularly regarding participating clients having high-quality data yet with limited resources. To address these challenges, we propose in this article FedAUR, a novel approach for an adaptive upgrade of clients resources in FL. We first introduce a method to measure how a locally generated model affects and improves the global model if selected for aggregation without revealing raw data. Next, based on the significance of each client parameters and the resources of their devices, we design a selection scheme that manages and distributes available resources on the server among the appropriate subset of clients. This client selection and resource allocation problem is thus formulated as an optimization problem, where the purpose is to discover and train in each round the maximum number of samples with the highest quality in order to target the desired performance. Moreover, we present a Kubernetes-based prototype that we implemented to evaluate the performance of the proposed approach.

Sawsan Abdulrahman

Papers

A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond

FedMCCS: Multicriteria Client Selection Model for Optimal IoT Federated Learning

Adaptive Upgrade of Client Resources for Improving the Quality of Federated Learning Model