Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Posted Content
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
TL;DR: This tutorial article aims to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcementlearning algorithms that utilize previously collected data, without additional online data collection.
Journal ArticleDOI
Shortcut learning in deep neural networks
Robert Geirhos,Jörn-Henrik Jacobsen,Claudio Michaelis,Richard S. Zemel,Wieland Brendel,Matthias Bethge,Felix A. Wichmann +6 more
TL;DR: A set of recommendations for model interpretation and benchmarking is developed, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications.
Journal ArticleDOI
Toward Causal Representation Learning
Bernhard Schölkopf,Francesco Locatello,Stefan Bauer,Nan Rosemary Ke,Nal Kalchbrenner,Anirudh Goyal,Yoshua Bengio +6 more
TL;DR: The authors reviewed fundamental concepts of causal inference and related them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research.
Posted Content
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Pang Wei Koh,Shiori Sagawa,Henrik Marklund,Sang Michael Xie,Marvin Zhang,Akshay Balsubramani,Weihua Hu,Michihiro Yasunaga,Richard Lanas Phillips,Irena Gao,Tony Lee,Etienne David,Ian Stavness,Wei Guo,Berton A. Earnshaw,Imran S. Haque,Sara Beery,Jure Leskovec,Anshul Kundaje,Emma Pierson,Sergey Levine,Chelsea Finn,Percy Liang +22 more
TL;DR: WILDS is presented, a benchmark of in-the-wild distribution shifts spanning diverse data modalities and applications, and is hoped to encourage the development of general-purpose methods that are anchored to real-world distribution shifts and that work well across different applications and problem settings.
Posted Content
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization.
TL;DR: The results suggest that regularization is important for worst-group generalization in the overparameterized regime, even if it is not needed for average generalization, and introduce a stochastic optimization algorithm, with convergence guarantees, to efficiently train group DRO models.
References
More filters
Journal ArticleDOI
A Simple Method to Determine if a Music Information Retrieval System is a "Horse"
TL;DR: A simple method is proposed and demonstrated to explain the figure of merit (FoM) of a music information retrieval system evaluated in a dataset, specifically, whether the FoM comes from the system using characteristics confounded with the “ground truth” of the dataset.
Proceedings ArticleDOI
Discovering Causal Signals in Images
TL;DR: The existence of observable footprints that reveal the causal dispositions of the object categories appearing in collections of images are established and a causal direction classifier is built that achieves state-of-the-art performance on finding the causal direction between pairs of random variables, given samples from their joint distribution.
Journal Article
Invariant models for causal transfer learning
TL;DR: In this article, the authors relax the usual covariate shift assumption and assume that it holds true for a subset of predictor variables: the conditional distribution of the target variable given this subset of predictors is invariant over all tasks.
Posted Content
Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach
TL;DR: In this article, the authors develop a generalized empirical likelihood framework based on distributional uncertainty sets constructed from nonparametric $f$-divergence balls for Hadamard differentiable functionals, and in particular, stochastic optimization problems.