Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Posted Content
An Empirical Study of Invariant Risk Minimization
TL;DR: By extending the ColoredMNIST experiment in different ways, it is found that IRMv1 performs better as the spurious correlation varies more widely between training environments, and can be extended to an analogous setting for text classification.
Posted Content
Overinterpretation reveals image classification model pathologies
TL;DR: It is demonstrated that state of the art neural networks for CIFar-10 and ImageNet suffer from overinterpretation, and CIFAR-10 trained models make confident predictions even when 95% of an input image has been masked and humans are unable to discern salient features in the remaining pixel subset.
Proceedings ArticleDOI
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
TL;DR: This paper proposes a method that can automatically detect and ignore dataset-specific patterns, which it hypothesize are likely to reflect dataset bias, and trains a lower capacity model in an ensemble with a higher capacity model.
Proceedings ArticleDOI
Causality Inspired Representation Learning for Domain Generalization
TL;DR: A general structural causal model is introduced and a Causality Inspired Representation Learning (CIRL) algorithm is proposed that enforces the representations to satisfy the above properties and then uses them to simulate the causal factors, which yields improved generalization ability.
Proceedings ArticleDOI
The Effects of Regularization and Data Augmentation are Class Dependent
TL;DR: It is demonstrated that techniques such as DA or weight decay produce a model with a reduced complexity that is unfair across classes, demonstrating that the search for ever increasing generalization performance has left us with models and regularizers that silentlysacrifice performances on some classes.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.