Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Proceedings ArticleDOI
GOOD: A Graph Out-of-Distribution Benchmark
TL;DR: This work aims at developing an OOD benchmark, known as GOOD, for graphs specifically, and explicitly makes distinctions between covariate and concept shifts and design data splits that accurately reflect different shifts.
Proceedings Article
A causal view of compositional zero-shot recognition
TL;DR: In this paper, a causal-inspired embedding model is proposed to disentangle representations of elementary components of visual objects from correlated (confounded) training data for predicting new combinations of attribute-object pairs.
Journal Article
Learning Robust Models using the Principle of Independent Causal Mechanisms
TL;DR: It is shown theoretically and experimentally that neural networks trained in this framework focus on relations remaining invariant across environments and ignore unstable ones, and it is proved that the recovered stable relations correspond to the true causal mechanisms under certain conditions.
Exchanging Lessons Between Algorithmic Fairness and Domain Generalization
TL;DR: This work proposes a novel domain generalization method and shows theoretically and empirically how different partitioning schemes can lead to increased or decreased generalization performance, enabling it to outperform Invariant Risk Minimization with handcrafted environments in multiple cases.
Journal ArticleDOI
ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets
TL;DR: It is shown that inverse correlations between ID and OOD performance do happen in real-world benchmarks, and are particularly striking on models trained with a regularizer that diversifies the solutions to the ERM objective.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.