Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Proceedings ArticleDOI
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses
TL;DR: Modifying the network architecture to impose inductive biases that make the network robust to dataset bias is proposed, which is biased to favor simpler solutions by design and demonstrates that when the state-of-the-art debiasing methods are combined with OccamNets 4 results further improve.
Posted Content
NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization
TL;DR: In this article, a data generator is learned to synthesize OoD data by maximizing losses computed by different neural architectures, while the goal for architecture search is to find the optimal architecture parameters that minimize the synthetic OoD dataset losses.
Journal ArticleDOI
Deep Causal Learning: Representation, Discovery and Inference
TL;DR: In this article , the authors comprehensively review how deep learning can contribute to causal learning by addressing conventional challenges from three aspects: representation, discovery, and inference, and point out that deep causal learning is important for the theoretical extension and application expansion of causal science and is also an indispensable part of general artificial intelligence.
Posted Content
Identifying Invariant Factors Across Multiple Environments with KL Regression
Jaime Roquero Gimenez,James Zou +1 more
TL;DR: It is proved that KL regression recovers the true invariant factors under a flexible confounding setup and is computationally efficient as it derive an analytic solution for its global optimum.
Journal ArticleDOI
On the Impact of Spurious Correlation for Out-of-Distribution Detection
TL;DR: In this paper , a new formalization and model of the data shifts by taking into account both the invariant and environmental (spurious) features is presented, and the authors systematically investigate how spurious correlation in the training set impacts OOD detection.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.