Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Proceedings ArticleDOI
Distilling Model Failures as Directions in Latent Space
TL;DR: This work presents a scalable method for automatically distilling a model’s failure modes and harnesses linear classifiers to identify consistent error patterns, and induces a natural representation of these failure modes as directions within the feature space.
Proceedings ArticleDOI
Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP
TL;DR: It is demonstrated that simply gathering a large amount of data from the web is not the most effective way to build a pre-training dataset for robust generalization, necessitating further study into dataset design.
Posted Content
How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels
Hua Shen,Ting-Hao Kenneth Huang +1 more
TL;DR: An investigation on whether or not showing machine-generated visual interpretations helps users understand the incorrectly predicted labels produced by image classifiers demonstrates that displaying the visual interpretations did not increase, but rather decreased, the average guessing accuracy by roughly 10%.
Posted Content
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
TL;DR: The authors proposed a method that can automatically detect and ignore these kinds of dataset-specific correlations created by idiosyncrasies in the data collection process, which they call dataset biases. But their method does not require the bias to be known in advance.
Proceedings ArticleDOI
On Feature Learning in the Presence of Spurious Correlations
TL;DR: This paper evaluates the amount of information about the core (non-spurious) features that can be decoded from the representations learned by standard empirical risk minimization (ERM) and specialized group robustness training and finds that strong regularization is not necessary for learning high quality feature representations.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.