Statistical Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1, 124, 254, 297 History Corner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20, 179 Teacher’s Corner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26, 173, 263, 335 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40, 147, 211, 366 Statistical Computing and Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Statistical Computing and Software Reviews . . . . . . . . . . . . . . . . . . . . . . . . 75, 187 Reviews of Books and Teaching Materials . . . . . . . . . . . . . . . . . 92, 189, 281, 401 Brief Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100, 195, 292, 404 Letters to the Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102, 197, 294, 406 Special Section: Statistical Training and Curricular Revision . . . . . . . . . . . . . 105 Errata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Special Section: Opportunities and Challenges for the Discipline . . . . . . . . . 201 Software Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

The American statistician

The objective of machine learning is to extract useful information from data, while privacy is preserved by concealing information. Thus it seems hard to reconcile these competing interests. However, they frequently must be balanced when mining sensitive data. For example, medical research represents an important application where it is necessary both to extract useful information and protect patient privacy. One way to resolve the conflict is to extract general characteristics of whole populations without disclosing the private information of individuals. 
In this paper, we consider differential privacy, one of the most popular and powerful definitions of privacy. We explore the interplay between machine learning and differential privacy, namely privacy-preserving machine learning algorithms and learning-based data release mechanisms. We also describe some theoretical results that address what can be learned differentially privately and upper bounds of loss functions for differentially private algorithms. 
Finally, we present some open questions, including how to incorporate public data, how to deal with missing data in private datasets, and whether, as the number of observed samples grows arbitrarily large, differentially private machine learning algorithms can be achieved at no cost to utility as compared to corresponding non-differentially private algorithms.

Differential Privacy and Machine Learning: a Survey and Review.

Releasing high-dimensional data enables a wide spectrum of data mining tasks. Yet, individual privacy has been a major obstacle to data sharing. In this paper, we consider the problem of releasing high-dimensional data with differential privacy guarantees. We propose a novel solution to preserve the joint distribution of a high-dimensional dataset. We first develop a robust sampling-based framework to systematically explore the dependencies among all attributes and subsequently build a dependency graph. This framework is coupled with a generic threshold mechanism to significantly improve accuracy. We then identify a set of marginal tables from the dependency graph to approximate the joint distribution based on the solid inference foundation of the junction tree algorithm while minimizing the resultant error. We prove that selecting the optimal marginals with the goal of minimizing error is NP-hard and, thus, design an approximation algorithm using an integer programming relaxation and the constrained concave-convex procedure. Extensive experiments on real datasets demonstrate that our solution substantially outperforms the state-of-the-art competitors.

Differentially Private High-Dimensional Data Publication via Sampling-Based Inference

Differential privacy has become the dominant standard in the research community for strong privacy protection. There has been a flood of research into query answering algorithms that meet this standard. Algorithms are becoming increasingly complex, and in particular, the performance of many emerging algorithms is data dependent, meaning the distribution of the noise added to query answers may change depending on the input data. Theoretical analysis typically only considers the worst case, making empirical study of average case performance increasingly important. In this paper we propose a set of evaluation principles which we argue are essential for sound evaluation. Based on these principles we propose DPBench, a novel evaluation framework for standardized evaluation of privacy algorithms. We then apply our benchmark to evaluate algorithms for answering 1- and 2-dimensional range queries. The result is a thorough empirical study of 15 published algorithms on a total of 27 datasets that offers new insights into algorithm behavior---in particular the influence of dataset scale and shape---and a more complete characterization of the state of the art. Our methodology is able to resolve inconsistencies in prior empirical studies and place algorithm performance in context through comparison to simple baselines. Finally, we pose open research questions which we hope will guide future algorithm design.

/pdf/principled-evaluation-of-differentially-private-algorithms-4xe4dwg1h6.pdf

Principled Evaluation of Differentially Private Algorithms using DPBench

Graph data publishing under node-differential privacy (node-DP) is challenging due to the huge sensitivity of queries. However, since a node in graph data oftentimes represents a person, node-DP is necessary to achieve personal data protection. In this paper, we investigate the problem of publishing the degree distribution of a graph under node-DP by exploring the projection approach to reduce the sensitivity. We propose two approaches based on aggregation and cumulative histogram to publish the degree distribution. The experiments demonstrate that our approaches greatly reduce the error of approximating the true degree distribution and have significant improvement over existing works. We also present the introspective analysis for understanding the factors of publishing the degree distribution with node-DP.

https://dl.acm.org/doi/pdf/10.1145/2882903.2926745

Publishing Graph Degree Distribution with Node Differential Privacy

Histograms are the workhorse of data mining and analysis. This paper considers the problem of publishing histograms under differential privacy, one of the strongest privacy models. Existing differentially private histogram publication schemes have shown that clustering (or grouping) is a promising idea to improve the accuracy of sanitized histograms. However, none of them fully exploits the benefit of clustering. In this paper, we introduce a new clustering framework. It features a sophisticated evaluation of the trade-off between the approximation error due to clustering and the Laplace error due to Laplace noise injected, which is normally overlooked in prior work. In particular, we propose three clustering strategies with different orders of run-time complexities. We prove the superiority of our approach by theoretical utility comparisons with the competitors. Our extensive experiments over various standard real-life and synthetic datasets confirm that our technique consistently outperforms existing competitors.

https://epubs.siam.org/doi/pdf/10.1137/1.9781611973440.68

Towards Accurate Histogram Publication under Differential Privacy

Publication of the private set-valued data will provide enormous opportunities for counting queries and various data mining tasks. Compared to those previous methods based on partition-based privacy models (e.g., k-anonymity), differential privacy provides strong privacy guarantees against adversaries with arbitrary background knowledge. However, the existing solutions based on differential privacy for data publication are currently limited to static datasets, and do not adequately address today’s demand for up-to-date information. In this paper, we address the problem of differentially private set-valued data release on an incremental scenario in which the data need to be transformed are not static. Motivated by this, we propose an efficient algorithm, called IncTDPart, to incrementally generate a series of differentially private releases. The proposed algorithm is based on top-down partitioning model with the help of item-free taxonomy tree and update-bounded mechanism. Extensive experiments on real datasets confirm that our approach maintains high utility and scalability for counting query.

Differentially Private Set-Valued Data Release against Incremental Updates

With the rapid development of big data technology, the issue of preserving personal privacy has attracted more and more attention. It has been shown that protecting the published graph with the differential privacy can ensure not only the quantitative privacy, but also the data usability. In this paper, we first put forward an optimized randomized response algorithm for generating synthetic graph, which dose not depend on a trusted third party in charge of collecting data. Furthermore, based on multi-party computation clustering, we propose a generated graph model under local differential privacy (LDPGM). The experiment indicates that LDPGM can effectively control the density of the synthetic graph so that significantly reduce the error between the synthetic graph and the original graph. Therefore, it maintains the properties of the original graph well and ensures high usability.

A Two-Phase Algorithm for Generating Synthetic Graph Under Local Differential Privacy

Discovering that Map-Reduce framework is a popular way to deal with a large scale of data, but there is a significant risk to leak out users' personal information, especially when the data is sensitive, for example, including users' health records, salary information, etc. Differential privacy has recently emerged as a new paradigm for preserving private data. This makes it possible to provide strong theoretical guarantees on the privacy and utility of the query results. In this paper, we focus on top-k query which is one of the most useful queries in Map-Reduce framework over big data sets.Motivated by this, we propose an efficient algorithm, called DiffMRDifferentially private Top-kquery over MapReduce), for processing top-k query as well as satisfying differential privacy. In our algorithm, to avoid the private leak in middle process, we use exponential mechanism to select top-k records from big data sets by using score function. When the data set is too large to get a reasonably accurate result, we can reduce the reject rate and execute several more times Map-Reduce to get a more accurate top-k query result. After getting a final top-k candidate result, we will add Laplace noise to each record and adopt post-processing technique to improve the accuracy of query answers. Our experimental study demonstrates that DiffMR algorithm can be used to answer the top-k query accurately in Map-Reduce framework.

Differentially private top-k query over MapReduce

Differential privacy has emerged as one of the most promising privacy models for releasing the results of statistical queries on sensitive data, with strong privacy guarantees. Existing works on differential privacy mostly focus on simple aggregations such as counts. This paper investigates the spatial OLAP queries, which combines GIS and OLAP queries at the same time. We employ a differentially private R-tree(DiffR-Tree) to help spatial OLAP queries. In our method, several steps need to be carefully designed to equip the spatial data warehouse structure with differential privacy requirements. Our experiments results demonstrate the efficiency of our spatial OLAP query index structure and the accuracy of answering queries.

Xiaojian Zhang

Papers

Towards Accurate Histogram Publication under Differential Privacy

Differentially Private Set-Valued Data Release against Incremental Updates

A Two-Phase Algorithm for Generating Synthetic Graph Under Local Differential Privacy

Differentially private top-k query over MapReduce

DiffR-Tree: a differentially private spatial index for OLAP query