Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients’ private data from being exposed to adversaries. Nevertheless, private information can still be divulged by analyzing uploaded parameters from clients, e.g., weights trained in deep neural networks. In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noise is added to parameters at the clients’ side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noise. Then we develop a theoretical convergence bound on the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) there is a tradeoff between convergence performance and privacy protection levels, i.e., better convergence performance leads to a lower protection level; 2) given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; and 3) there is an optimal number aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a $K$ -client random scheduling strategy, where $K$ ( $1\leq K ) clients are randomly selected from the $N$ overall clients to participate in each aggregation. We also develop a corresponding convergence bound for the loss function in this case and the $K$ -client random scheduling strategy also retains the above three properties. Moreover, we find that there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the design of various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

Federated Learning With Differential Privacy: Algorithms and Performance Analysis

In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a $K$-random scheduling strategy, where $K$ ($1<K<N$) clients are randomly selected from the $N$ overall clients to participate in each aggregation. We also develop the corresponding convergence bound of the loss function in this case and the $K$-random scheduling strategy can also retain the above three properties. Moreover, we find that there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the designs on various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

Federated Learning with Differential Privacy: Algorithms and Performance Analysis

Differential privacy is an essential and prevalent privacy model that has been widely explored in recent decades. This survey provides a comprehensive and structured overview of two research directions: differentially private data publishing and differentially private data analysis. We compare the diverse release mechanisms of differentially private data publishing given a variety of input data in terms of query type, the maximum number of queries, efficiency, and accuracy. We identify two basic frameworks for differentially private data analysis and list the typical algorithms used within each framework. The results are compared and discussed based on output accuracy and efficiency. Further, we propose several possible directions for future research and possible applications.

https://ieeexplore.ieee.org/ielaam/69/7970215/7911185-aam.pdf

Differentially Private Data Publishing and Analysis: A Survey

https://www.jstage.jst.go.jp/article/jsprs1975/37/6/37_6_39/_pdf

1998 IEEE International Conference on SMCに参加して

We consider the minimax estimation problem of a discrete distribution with support size $k$ under privacy constraints. A privatization scheme is applied to each raw sample independently, and we need to estimate the distribution of the raw samples from the privatized samples. A positive number $\epsilon $ measures the privacy level of a privatization scheme. For a given $\epsilon $ , we consider the problem of constructing optimal privatization schemes with $\epsilon $ -privacy level, i.e., schemes that minimize the expected estimation loss for the worst-case distribution. Two schemes known in the literature provide order optimal performance in the high privacy regime where $\epsilon $ is very close to 0, and in the low privacy regime where $e^{\epsilon }\approx k$ , respectively. In this paper, we propose a new family of schemes which substantially improve the performance of the existing schemes in the medium privacy regime when $1\ll e^{\epsilon } \ll k$ . More concretely, we prove that when $3.8 , our schemes reduce the expected estimation loss by 50% under $\ell _{2}^{2}$ metric and by 30% under $\ell _{1}$ metric over the existing schemes. We also prove a lower bound for the region $e^{\epsilon } \ll k$ , which implies that our schemes are order optimal in this regime.

Optimal Schemes for Discrete Distribution Estimation Under Locally Differential Privacy

Consider statistical learning (e.g. discrete distribution estimation) with local $\epsilon$-differential privacy, which preserves each data provider's privacy locally, we aim to optimize statistical data utility under the privacy constraints. Specifically, we study maximizing mutual information between a provider's data and its private view, and give the exact mutual information bound along with an attainable mechanism: $k$-subset mechanism as results. The mutual information optimal mechanism randomly outputs a size $k$ subset of the original data domain with delicate probability assignment, where $k$ varies with the privacy level $\epsilon$ and the data domain size $d$. After analysing the limitations of existing local private mechanisms from mutual information perspective, we propose an efficient implementation of the $k$-subset mechanism for discrete distribution estimation, and show its optimality guarantees over existing approaches.

Mutual Information Optimally Local Private Discrete Distribution Estimation.

For the purpose of improving the quality of services, softwares or online services are collecting various of user data, such as personal information and locations. Such data facilitates mining statistical knowledge of users, but threatens users’ privacy as it may reveal sensitive information (e.g., identities and activities) about individuals. This work considers distribution estimation over user-contributed data meanwhile providing rigid protection of their data with local $\epsilon$e-differential privacy ($\epsilon$e-LDP), which sanitizes each user's data on the client's side (e.g, on the user's mobile device). Our privacy protection covers both qualitative data (e.g., categorical data) and discrete quantitative data (e.g., location data). Specifically, for categorical data, we derive an optimal $\epsilon$e-LDP mechanism (termed as $k$k-subset mechanism) from mutual information perspective, and further show its optimality over existing approaches within the context of discrete distribution estimation; for discrete quantitative data that have arbitrary distance metric, we provide an efficient extension of $k$k-subset mechanism by proposing a variant of the popular Exponential Mechanism (EM) to tackle the asymmetry issue on the data domain. Experiments on real-world datasets and simulated scenarios show that our mechanism is highly efficient and reduces nearly a fraction of $\exp (-\frac{\epsilon }{2})$exp(-e2) error for distribution estimation when compared to existing approaches.

Local Differential Private Data Aggregation for Discrete Distribution Estimation

Set-valued data is useful for representing a rich family of information in numerous areas, such as market basket data of online shopping, apps on mobile phones and web browsing history. By analyzing set-valued data that are collected from users, service providers could learn the demographics of the users, the patterns of their usages, and finally, improve the quality of services for them. However, privacy has been an increasing concern in collecting and analyzing users' set-valued data, since these data may reveal sensitive information (e.g., identities, preferences and diseases) about individuals. In this work, we propose a privacy preserving aggregation mechanism for set-valued data: PrivSet. It provides rigorous data privacy protection locally (e.g., on mobile phones or wearable devices) and efficiently (its computational overhead is linear to the item domain size) for each user, and meanwhile allowing effective statistical analyses (e.g., distribution estimation of items, distribution estimation of set cardinality) on set-valued data for service providers. More specifically, in PrivSet, within the constraints of local e-differential privacy, each user independently responses with a subset of the set-valued data domain with calibrated probabilities, hence the true positive/false positive rate of each item is balanced and the performance of distribution estimation is optimized. Besides presenting theoretical error bounds of PrivSet and proving its optimality over existing approaches, we experimentally validate the mechanism, the experimental results illustrate that the estimation error in PrivSet has been reduced by half when compared to state-of-the-art approaches.

PrivSet: Set-Valued Data Analyses with Locale Differential Privacy

The categorical data that have natural ordering between categories are termed ordinal data, which are pervasive in numerous areas, including discrete sensor readings, metering data or preference options. Though aggregating such ordinal data from the population is facilitating plenty of crowdsourcing applications, contributing such data is privacy risky and may reveal sensitive information (e.g. locations, identities) about individuals. This work studies ordinal data aggregation for distribution estimation meanwhile locally preserving individuals' data privacy (such as on their mobile devices). Under e-geo-indistinguishable constraints, which capture intrinsic dissimilarity between ordinal categories in the framework of differential privacy, we provide an efficient and effective locally private mechanism: Subset Exponential Mechanism (SEM) for ordinal data distribution estimation. The mechanism randomly responds with a fixed-size subset of the categories with calibrated probability assignment. Specially for uniform ordinal data, we propose a circling technique to symmetrically randomizing categories and estimating frequencies of categories, hence the computational/space costs and estimation performance of SEM are further optimized. Besides contributing theoretical error bounds of SEM, we also evaluate the mechanism on extensive scenarios, the evaluation results show that SEM reduces distribution estimation error on average by exp(∊/2) factor over existing private mechanisms.

Local private ordinal data distribution estimation

Recently, local differential privacy (LDP), as a strong and practical notion, has been applied to deal with privacy issues in data collection. However, existing LDP-based strategies mainly focus on utility optimization at a single privacy level while ignoring various privacy preferences of data providers and multilevel privacy demands for statistics. In this paper, we for the first time propose a framework to optimize the utility of histogram estimation with these two privacy requirements. To clarify the goal of privacy protection, we personalize the traditional definition of LDP. We design two independent approaches to minimize the utility loss: Advanced Combination , which composes multilevel results for utility optimization, and Data Recycle with Personalized Privacy , which enlarges the sample size for an estimation. We demonstrate their effectiveness on privacy and utility, respectively. Moreover, we embed these approaches within a Recycle and Combination Framework and prove that the framework stably achieves the optimal utility by quantifying its error bounds. On real-world datasets, our approaches are experimentally validated and remarkably outperform baseline methods.

Yiwen Nie

Papers

Mutual Information Optimally Local Private Discrete Distribution Estimation.

Local Differential Private Data Aggregation for Discrete Distribution Estimation

PrivSet: Set-Valued Data Analyses with Locale Differential Privacy

Local private ordinal data distribution estimation

A Utility-Optimized Framework for Personalized Private Histogram Estimation