Open AccessJournal Article
Counterfactual reasoning and learning systems: the example of computational advertising
Léon Bottou,Jonas Peters,Joaquin Quiñonero-Candela,Denis X. Charles,D. Max Chickering,Elon Portugaly,Dipankar Ray,Patrice Y. Simard,Ed Snelson +8 more
Reads0
Chats0
TLDR
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system and allow both humans and algorithms to select the changes that would have improved the system performance.Abstract:
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system. Such predictions allow both humans and algorithms to select the changes that would have improved the system performance. This work is illustrated by experiments on the ad placement system associated with the Bing search engine.read more
Citations
More filters
Proceedings ArticleDOI
Off-policy Bandits with Deficient Support
TL;DR: This work systematically analyzed the statistical and computational properties of three approaches that provide various guarantees for IPS-based learning despite the inherent limitations of support-deficient data: restricting the action space, reward extrapolation, and restricting the policy space.
Proceedings Article
Predictive Off-Policy Policy Evaluation for Nonstationary Decision Problems, with Applications to Digital Marketing.
TL;DR: It is argued that off-policy policy evaluation for nonstationary MDPs can be phrased as a time series prediction problem, which results in predictive methods that can anticipate changes before they happen and results in a drastic reduction of mean squared error when evaluating policies using real digital marketing data set.
Proceedings Article
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
TL;DR: A bias-reduced augmentation of Liu et al. (2018a)'s method, which can take advantage of a learned value function to obtain higher accuracy and yields significant advantages over previous methods.
Posted Content
Large-scale Validation of Counterfactual Learning Methods: A Test-Bed
TL;DR: The results show experimental evidence that recent off-policy learning methods can improve upon state-of-the-art supervised learning techniques on a large-scale real-world data set.
Proceedings ArticleDOI
Toward Predicting the Outcome of an A/B Experiment for Search Relevance
TL;DR: This work studies an alternative that uses historical search log to reliably predict online click-based metrics of a ranking function, without actually running it on live users, and replaces exact matching by fuzzy matching to increase data efficiency.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.