Open AccessProceedings Article
Decoupling "when to update" from "how to update"
Eran Malach,Shai Shalev-Shwartz +1 more
- Vol. 30, pp 960-970
Reads0
Chats0
TLDR
In this paper, a meta algorithm for tackling the noisy labels problem is proposed, which decouples ''when to update'' from ''how to update''. And they demonstrate the effectiveness of their algorithm by mining data for gender classification by combining the Labeled Faces in the Wild (LFW) face recognition dataset with a textual genderizing service, which leads to a noisy dataset.Abstract:
Deep learning requires data. A useful approach to obtain data is to be creative and mine data from various sources, that were created for different purposes. Unfortunately, this approach often leads to noisy labels. In this paper, we propose a meta algorithm for tackling the noisy labels problem. The key idea is to decouple ``when to update'' from ``how to update''. We demonstrate the effectiveness of our algorithm by mining data for gender classification by combining the Labeled Faces in the Wild (LFW) face recognition dataset with a textual genderizing service, which leads to a noisy dataset. While our approach is very simple to implement, it leads to state-of-the-art results. We analyze some convergence properties of the proposed algorithm.read more
Citations
More filters
Posted Content
Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
TL;DR: Co-teaching as discussed by the authors trains two deep neural networks simultaneously, and let them teach each other given every mini-batch: first, each network feeds forward all data and selects some data of possibly clean labels; secondly, two networks communicate with each other what data in this minibatch should be used for training; finally, each networks back propagates the data selected by its peer network and updates itself.
Posted Content
Learning from Noisy Labels with Deep Neural Networks: A Survey
TL;DR: A comprehensive review of 62 state-of-the-art robust training methods, all of which are categorized into five groups according to their methodological difference, followed by a systematic comparison of six properties used to evaluate their superiority.
Proceedings Article
DivideMix: Learning with Noisy Labels as Semi-supervised Learning
TL;DR: DivideMix as mentioned in this paper models the per-sample loss distribution with a mixture model to dynamically divide the training data into clean samples and noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner.
Proceedings ArticleDOI
Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization
TL;DR: In this article, the authors propose a robust learning paradigm called JoCoR, which aims to reduce the diversity of two networks during training by calculating a joint loss with co-regularization for each training example.
Journal ArticleDOI
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis
TL;DR: In this article, a review of the state-of-the-art in handling label noise in deep learning for medical image analysis is presented, where the authors conducted experiments with three medical imaging datasets with different types of label noise, where they investigated several existing strategies and developed new methods to combat the negative effect of labels.