The hardness of conditional independence testing and the generalised covariance measure
Rajen D. Shah,Jonas Peters +1 more
Reads0
Chats0
TLDR
In this paper, the authors propose a test statistic based on the sample covariance between the residuals, which they call the generalised covariance measure (GCM) and prove that the validity of this form of test relies almost entirely on the weak requirement that the regression procedures are able to estimate the conditional means $X$ given $Z$, and $Y$ given £Z$ at a slow rate.Abstract:
It is a common saying that testing for conditional independence, that is, testing whether whether two random vectors $X$ and $Y$ are independent, given $Z$, is a hard statistical problem if $Z$ is a continuous random variable (or vector). In this paper, we prove that conditional independence is indeed a particularly difficult hypothesis to test for. Valid statistical tests are required to have a size that is smaller than a pre-defined significance level, and different tests usually have power against a different class of alternatives. We prove that a valid test for conditional independence does not have power against any alternative. Given the nonexistence of a uniformly valid conditional independence test, we argue that tests must be designed so their suitability for a particular problem may be judged easily. To address this need, we propose in the case where $X$ and $Y$ are univariate to nonlinearly regress $X$ on $Z$, and $Y$ on $Z$ and then compute a test statistic based on the sample covariance between the residuals, which we call the generalised covariance measure (GCM). We prove that validity of this form of test relies almost entirely on the weak requirement that the regression procedures are able to estimate the conditional means $X$ given $Z$, and $Y$ given $Z$, at a slow rate. We extend the methodology to handle settings where $X$ and $Y$ may be multivariate or even high dimensional. While our general procedure can be tailored to the setting at hand by combining it with any regression technique, we develop the theoretical guarantees for kernel ridge regression. A simulation study shows that the test based on GCM is competitive with state of the art conditional independence tests. Code is available as the R package $\mathtt{GeneralisedCovarianceMeasure}$ on CRAN.read more
Citations
More filters
Journal ArticleDOI
Toward Causal Representation Learning
Bernhard Schölkopf,Francesco Locatello,Stefan Bauer,Nan Rosemary Ke,Nal Kalchbrenner,Anirudh Goyal,Yoshua Bengio +6 more
TL;DR: The authors reviewed fundamental concepts of causal inference and related them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research.
Proceedings ArticleDOI
50 Years of Test (Un)fairness: Lessons for Machine Learning
Ben Hutchinson,Margaret Mitchell +1 more
TL;DR: This work traces how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged.
Journal ArticleDOI
Invariant Causal Prediction for Nonlinear Models
TL;DR: This work presents and evaluates an array of methods for nonlinear and nonparametric versions of ICP for learning the causal parents of given target variables and finds that an approach which first fits a nonlinear model with data pooled over all environments and then tests for differences between the residual distributions across environments is quite robust across a large variety of simulation settings.
Posted Content
Causality for Machine Learning
TL;DR: It is argued that the hard open problems of machine learning and AI are intrinsically related to causality, and how the field is beginning to understand them is explained.
Posted Content
Learning Neural Causal Models from Unknown Interventions.
Nan Rosemary Ke,Olexa Bilaniuk,Anirudh Goyal,Stephan Bauer,Hugol Larochelle,Chris Pal,Yoshua Bengio +6 more
TL;DR: This paper provides a general framework based on continuous optimization and neural networks to create models for the combination of observational and interventional data and establishes strong benchmark results on several structure learning tasks.
References
More filters
Journal ArticleDOI
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Proceedings ArticleDOI
XGBoost: A Scalable Tree Boosting System
Tianqi Chen,Carlos Guestrin +1 more
TL;DR: XGBoost as discussed by the authors proposes a sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning to achieve state-of-the-art results on many machine learning challenges.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Pattern Recognition and Machine Learning
TL;DR: Probability distributions of linear models for regression and classification are given in this article, along with a discussion of combining models and combining models in the context of machine learning and classification.
Book
Probabilistic graphical models : principles and techniques
Daniel L. Koller,Nir Friedman +1 more
TL;DR: The framework of probabilistic graphical models, presented in this book, provides a general approach for causal reasoning and decision making under uncertainty, allowing interpretable models to be constructed and then manipulated by reasoning algorithms.