Today’s artificial intelligence still faces two major challenges. One is that, in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated-learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated-learning framework, which includes horizontal federated learning, vertical federated learning, and federated transfer learning. We provide definitions, architectures, and applications for the federated-learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allowing knowledge to be shared without compromising user privacy.

Federated Machine Learning: Concept and Applications

Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.

Machine learning is widely used in practice to produce predictive models for applications such as image processing, speech and text recognition. These models are more accurate when trained on large amount of data collected from different sources. However, the massive data collection raises privacy concerns. In this paper, we present new and efficient protocols for privacy preserving machine learning for linear regression, logistic regression and neural network training using the stochastic gradient descent method. Our protocols fall in the two-server model where data owners distribute their private data among two non-colluding servers who train various models on the joint data using secure two-party computation (2PC). We develop new techniques to support secure arithmetic operations on shared decimal numbers, and propose MPC-friendly alternatives to non-linear functions such as sigmoid and softmax that are superior to prior work. We implement our system in C++. Our experiments validate that our protocols are several orders of magnitude faster than the state of the art implementations for privacy preserving linear and logistic regressions, and scale to millions of data samples with thousands of features. We also implement the first privacy preserving system for training neural networks.

/pdf/secureml-a-system-for-scalable-privacy-preserving-machine-2jlvrwyxkm.pdf

SecureML: A System for Scalable Privacy-Preserving Machine Learning

In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL

Federated Learning in Mobile Edge Networks: A Comprehensive Survey

/pdf/secureml-a-system-for-scalable-privacy-preserving-machine-3zt40tob0l.pdf

SecureML: A System for Scalable Privacy-Preserving Machine Learning.

We present a privacy-preserving deep learning system in which many learning participants perform neural network-based deep learning over a combined dataset of all, without revealing the participant...

Privacy-Preserving Deep Learning via Additively Homomorphic Encryption

/pdf/privacy-preserving-deep-learning-via-additively-homomorphic-3trwpm9kiz.pdf

Privacy-Preserving Deep Learning via Additively Homomorphic Encryption.

Logistic regression is a powerful machine learning tool to classify data. When dealing with sensitive data such as private or medical information, cares are necessary. In this paper, we propose a secure system for protecting the training data in logistic regression via homomorphic encryption. Perhaps surprisingly, despite the non-polynomial tasks of training in logistic regression, we show that only additively homomorphic encryption is needed to build our system. Our system is secure and scalable with the dataset size.

Scalable and Secure Logistic Regression via Homomorphic Encryption

Logistic regression is a powerful machine learning tool to classify data. When dealing with sensitive data such as private or medical information, cares are necessary. In this paper, we propose a secure system for protecting both the training and predicting data in logistic regression via homomorphic encryption. Perhaps surprisingly, despite the non-polynomial tasks of training and predicting in logistic regression, we show that only additively homomorphic encryption is needed to build our system. Indeed, we instantiate our system with Paillier, LWE-based, and ring-LWE-based encryption schemes, highlighting the merits and demerits of each instance. Our system is very scalable in both the dataset size and dimension, tolerating big size for example of hundreds of millions (10s) records. Besides examining the costs of computation and communication, we carefully test our system over real datasets to demonstrate its accuracies and other related measures such as F-score and AUC.

/pdf/scalable-and-secure-logistic-regression-via-homomorphic-3o5jhm8dtr.pdf

Scalable and Secure Logistic Regression via Homomorphic Encryption.

The notion of a splitting authentication code is very important in the context of an authentication code with arbitration. Ogata et al. [Discrete Math., 279 (2004), pp. 383--405] characterized an optimal splitting authentication code in terms of a splitting balanced incomplete block design (BIBD). A $(v,u \times c,1)$-splitting BIBD is a pair $({\cal V}, {\cal B})$, where ${\cal V}$ is a v-set of points and ${\cal B}$ is a collection of $u \times c$ arrays, called blocks, with entries from ${\cal V}$, such that any point of ${\cal V}$ can occur at most once in any block, and forany two distinct points x and y of ${\cal V}$, there is exactly one block of ${\cal B}$ in which x and y occur in different rows. In this paper, we describe various combinatorial constructions for splitting BIBDs (or, equivalently, optimal splitting authentication codes). We show that the necessary conditions for the existence of a $(v,u \times c,1)$-splitting BIBD (or, equivalently, an optimal c-splitting authentication code with ...

Lihua Wang

Papers

Privacy-Preserving Deep Learning via Additively Homomorphic Encryption

Privacy-Preserving Deep Learning via Additively Homomorphic Encryption.

Scalable and Secure Logistic Regression via Homomorphic Encryption

Scalable and Secure Logistic Regression via Homomorphic Encryption.

Combinatorial Constructions for Optimal Splitting Authentication Codes