Machine-learning (ML) algorithms are increasingly utilized in privacy-sensitive applications such as predicting lifestyle choices, making medical diagnoses, and facial recognition. In a model inversion attack, recently introduced in a case study of linear classifiers in personalized medicine by Fredrikson et al., adversarial access to an ML model is abused to learn sensitive genomic information about individuals. Whether model inversion attacks apply to settings outside theirs, however, is unknown. We develop a new class of model inversion attack that exploits confidence values revealed along with predictions. Our new attacks are applicable in a variety of settings, and we explore two in depth: decision trees for lifestyle surveys as used on machine-learning-as-a-service systems and neural networks for facial recognition. In both cases confidence values are revealed to those with the ability to make prediction queries to models. We experimentally show attacks that are able to estimate whether a respondent in a lifestyle survey admitted to cheating on their significant other and, in the other context, show how to recover recognizable images of people's faces given only their name and access to the ML model. We also initiate experimental exploration of natural countermeasures, investigating a privacy-aware decision tree training algorithm that is a simple variant of CART learning, as well as revealing only rounded confidence values. The lesson that emerges is that one can avoid these kinds of MI attacks with negligible degradation to utility.

https://dl.acm.org/doi/pdf/10.1145/2810103.2813677

Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

There has been much progress in genomics in the ten years since a draft sequence of the human genome was published. Opportunities for understanding health and disease are now unprecedented, as advances in genomics are harnessed to obtain robust foundational knowledge about the structure and function of the human genome and about the genetic contributions to human health and disease. Here we articulate a 2011 vision for the future of genomics research and describe the path towards an era of genomic medicine.

/pdf/charting-a-course-for-genomic-medicine-from-base-pairs-to-ajw5texulr.pdf

Charting a course for genomic medicine from base pairs to bedside

Presentacio sobre l'Oficina de Proteccio de Dades Personals de la UAB i la politica Open Science. Va formar part de la conferencia "Les politiques d'Open Data / Open Acces: Implicacions a la recerca" orientada a investigadors i gestors de projectes europeus que va tenir lloc el 20 de setembre de 2018 a la Universitat Autonoma de Barcelona

/pdf/general-data-protection-regulation-2z11cloyp7.pdf

General data protection regulation

We initiate the study of privacy in pharmacogenetics, wherein machine learning models are used to guide medical treatments based on a patient's genotype and background. Performing an in-depth case study on privacy in personalized warfarin dosing, we show that suggested models carry privacy risks, in particular because attackers can perform what we call model inversion: an attacker, given the model and some demographic information about a patient, can predict the patient's genetic markers.

As differential privacy (DP) is an oft-proposed solution for medical settings such as this, we evaluate its effectiveness for building private versions of pharmacogenetic models. We show that DP mechanisms prevent our model inversion attacks when the privacy budget is carefully selected. We go on to analyze the impact on utility by performing simulated clinical trials with DP dosing models. We find that for privacy budgets effective at preventing attacks, patients would be exposed to increased risk of stroke, bleeding events, and mortality. We conclude that current DP mechanisms do not simultaneously improve genomic privacy while retaining desirable clinical efficacy, highlighting the need for new mechanisms that should be evaluated in situ using the general methodology introduced by our work.

/pdf/privacy-in-pharmacogenetics-an-end-to-end-case-study-of-43ecv8aqsf.pdf

Privacy in pharmacogenetics: an end-to-end case study of personalized warfarin dosing

The Electronic Medical Records and Genomics (eMERGE) Network: past, present, and future

Publishing data from electronic health records while preserving privacy: a survey of algorithms.

The disclosure of diagnosis codes can breach research participants' privacy.

Genome-wide association studies (GWAS) facilitate the discovery of genotype–phenotype relations from population-based sequence databases, which is an integral facet of personalized medicine. The increasing adoption of electronic medical records allows large amounts of patients’ standardized clinical features to be combined with the genomic sequences of these patients and shared to support validation of GWAS findings and to enable novel discoveries. However, disseminating these data “as is” may lead to patient reidentification when genomic sequences are linked to resources that contain the corresponding patients’ identity information based on standardized clinical features. This work proposes an approach that provably prevents this type of data linkage and furnishes a result that helps support GWAS. Our approach automatically extracts potentially linkable clinical features and modifies them in a way that they can no longer be used to link a genomic sequence to a small number of patients, while preserving the associations between genomic sequences and specific sets of clinical features corresponding to GWAS-related diseases. Extensive experiments with real patient data derived from the Vanderbilt's University Medical Center verify that our approach generates data that eliminate the threat of individual reidentification, while supporting GWAS validation and clinical case analysis tasks.

/pdf/anonymization-of-electronic-medical-records-for-validating-jm70o294di.pdf

Anonymization of electronic medical records for validating genome-wide association studies

The collection and sharing of person-specific biospecimens has raised significant questions regarding privacy. In particular, the question of identifiability, or the degree to which materials stored in biobanks can be linked to the name of the individuals from which they were derived, is under scrutiny. The goal of this paper is to review the extent to which biospecimens and affiliated data can be designated as identifiable. To achieve this goal, we summarize recent research in identifiability assessment for DNA sequence data, as well as associated demographic and clinical data, shared via biobanks. We demonstrate the variability of the degree of risk, the factors that contribute to this variation, and potential ways to mitigate and manage such risk. Finally, we discuss the policy implications of these findings, particularly as they pertain to biobank security and access policies. We situate our review in the context of real data sharing scenarios and biorepositories.

/pdf/identifiability-in-biobanks-models-measures-and-mitigation-m4slbxji51.pdf

Identifiability in biobanks: models, measures, and mitigation strategies

K-anonymisation is an approach to protecting privacy contained within a data set A good k-anonymisation algorithm should anonymise a data set in such a way that private information contained within it is hidden, yet anonymised data is still useful in intended applications Maximising both data usefulness and privacy protection in k-anonymisation is however difficult In this paper, we suggest a metric that attempts to quantify these two properties and introduce a clustering based algorithm that can achieve a balance between them in k-anonymisation

Grigorios Loukides

Papers

Publishing data from electronic health records while preserving privacy: a survey of algorithms.

The disclosure of diagnosis codes can breach research participants' privacy.

Anonymization of electronic medical records for validating genome-wide association studies

Identifiability in biobanks: models, measures, and mitigation strategies

Capturing data usefulness and privacy protection in K-anonymisation