Open AccessPosted Content
An Investigation of Data Poisoning Defenses for Online Learning.
Yizhen Wang,Kamalika Chaudhuri +1 more
TLDR
This work undertake a rigorous study of defenses against data poisoning for online learning, and studies four standard defenses in a powerful threat model, and provides conditions under which they can allow or resist rapid poisoning.Abstract:
Data poisoning attacks -- where an adversary can modify a small fraction of training data, with the goal of forcing the trained classifier to high loss -- are an important threat for machine learning in many applications. While a body of prior work has developed attacks and defenses, there is not much general understanding on when various attacks and defenses are effective. In this work, we undertake a rigorous study of defenses against data poisoning for online learning. First, we study four standard defenses in a powerful threat model, and provide conditions under which they can allow or resist rapid poisoning. We then consider a weaker and more realistic threat model, and show that the success of the adversary in the presence of data poisoning defenses there depends on the "ease" of the learning problem.read more
Citations
More filters
Posted Content
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Luis Muñoz-González,Battista Biggio,Ambra Demontis,Andrea Paudice,Vasin Wongrassamee,Emil Lupu,Fabio Roli +6 more
TL;DR: In this article, a back-gradient optimization algorithm is proposed to compute the gradient of interest through automatic gradient extraction, while also reversing the learning procedure to drastically reduce the attack complexity, which is able to target a wider class of learning algorithms, trained with gradient-based procedures.
Journal Article
Robustly Learning a Gaussian: Getting Optimal Error, Efficiently
Alistair Stewart,Ilias Diakonikolas,Gautam Kamath,Daniel M. Kane,Jerry Zheng Li,Ankur Moitra +5 more
TL;DR: In this paper, the authors studied the problem of learning the parameters of a high-dimensional Gaussian in the presence of noise and gave robust estimators that achieve estimation error O(varepsilon) in the total variation distance, which is optimal up to a universal constant.
Posted Content
Influence Based Defense Against Data Poisoning Attacks in Online Learning.
TL;DR: In this paper, the authors proposed a defense mechanism to minimize the degradation caused by the poisoned training data on a learner's model in an online setup, which utilizes an influence function which is a classic technique in robust statistics.
On the Permanence of Backdoors in Evolving Models
TL;DR: In this article , the authors explore the behavior of backdoor attacks in time-varying models, whose model weights are continually updated via fine-tuning to adapt to data drifts.
Posted Content
Gradient-based Data Subversion Attack Against Binary Classifiers.
TL;DR: In this article, a gradient-based data poisoning attack was proposed to achieve model degradation under the assumption that the attacker has limited-knowledge of the victim model, where the gradients of a differentiable convex loss function (residual errors) with respect to the predicted label were exploited as a warm-start and formulated different strategies to find a set of data instances to contaminate.
References
More filters
Posted Content
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
TL;DR: Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms, as it shares the same image size, data format and the structure of training and testing splits.
Software Framework for Topic Modelling with Large Corpora
Radim Řehůřek,Petr Sojka +1 more
TL;DR: This work describes a Natural Language Processing software framework which is based on the idea of document streaming, i.e. processing corpora document after document, in a memory independent fashion, and implements several popular algorithms for topical inference, including Latent Semantic Analysis and Latent Dirichlet Allocation in a way that makes them completely independent of the training corpus size.
Proceedings Article
Learning Word Vectors for Sentiment Analysis
TL;DR: This work presents a model that uses a mix of unsupervised and supervised techniques to learn word vectors capturing semantic term--document information as well as rich sentiment content, and finds it out-performs several previously introduced methods for sentiment classification.
Book ChapterDOI
Adversarial examples in the physical world
TL;DR: It is found that a large fraction of adversarial examples are classified incorrectly even when perceived through the camera, which shows that even in physical world scenarios, machine learning systems are vulnerable to adversarialExamples.
Book
Online Learning and Online Convex Optimization
TL;DR: A modern overview of online learning is provided to give the reader a sense of some of the interesting ideas and in particular to underscore the centrality of convexity in deriving efficient online learning algorithms.