Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Journal ArticleDOI
A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings
TL;DR: A semantic-aware contrastive learning framework for sentence embeddings, termed Pseudo-Token BERT (PT-BERT), which is able to explore the pseudo-token space (i.e., latent semantic space) representation of a sentence while eliminating the impact of superficial features such as sentence length and syntax is proposed.
Posted Content
Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization
TL;DR: The authors train a set of similar models to fit the data in different ways using a penalty on the alignment of their input gradients and show theoretically and empirically that this induces the learning of more complex predictive patterns.
Journal Article
Out-of-Distribution Generalization Analysis via Influence Function
TL;DR: In this article, the authors introduce Influence Function, a classical tool from robust statistics, into the OOD generalization problem and suggest the variance of influence function to measure the stability of a model on training environments.
Posted Content
Time Series Alignment with Global Invariances
TL;DR: This work addresses the problem of comparing time series while taking into account both feature space transformation and temporal variability, and proposes a latent global transformation of the feature space with the widely used Dynamic Time Warping (DTW).
Journal ArticleDOI
A Unified Causal View of Domain Invariant Representation Learning
TL;DR: The purpose of this paper is to show how the different methods of domain shift relate to each other, and clarify the real-world circumstances under which each is expected to succeed.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.