Open AccessPosted Content
Invariant Risk Minimization
Reads0
Chats0
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.Abstract:
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.read more
Citations
More filters
Posted Content
Generalization and Invariances in the Presence of Unobserved Confounding.
TL;DR: It is argued that generalization must be defined with respect to a broader class of distribution shifts, irrespective of their origin (arising from changes in observed, unobserved or target variables), and a new learning principle is proposed from which an explicit notion of generalization to certain new environments is expected, even in the presence of hidden confounding.
Posted Content
Selecting Data Augmentation for Simulating Interventions
TL;DR: It is argued that causal concepts can be used to explain the success of data augmentation by describing how they can weaken the spurious correlation between the observed domains and the task labels.
Journal ArticleDOI
Building Thinking Machines by Solving Animal Cognition Tasks
TL;DR: It is argued that to fix shortcomings with modern AI systems a nonverbal operationalisation is required and this is provided by the recent Animal-AI Testbed, which translates animal cognition tests for AI and provides a bottom-up research pathway for building thinking machines that create predictive models of their environment from sensory input.
Journal Article
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
Zhecan Wang,Noel C. F. Codella,Yen-Chun Chen,Luowei Zhou,Jianwei Yang,Xiyang Dai,Bin Xiao,Haoxuan You,Shih-Fu Chang,Lu Yuan +9 more
TL;DR: An evaluation protocol that includes Visual Commonsense Reasoning, Visual Entailment, and Visual Question Answering is introduced, and an approach, named CLIP Targeted Distillation (CLIP-TD), to intelligently distill knowledge from CLIP into existing architectures using a dynamically weighted objective applied to adaptively selected tokens per instance is proposed.
Journal ArticleDOI
From Statistical to Causal Learning
TL;DR: Basic ideas underlying research to build and understand artificially intelligent systems are described: from symbolic approaches via statistical learning to interventional models relying on concepts of causality.
References
More filters
Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.
Book
Introduction to Smooth Manifolds
TL;DR: In this paper, a review of topology, linear algebra, algebraic geometry, and differential equations is presented, along with an overview of the de Rham Theorem and its application in calculus.