Open AccessJournal Article
Counterfactual reasoning and learning systems: the example of computational advertising
Léon Bottou,Jonas Peters,Joaquin Quiñonero-Candela,Denis X. Charles,D. Max Chickering,Elon Portugaly,Dipankar Ray,Patrice Y. Simard,Ed Snelson +8 more
Reads0
Chats0
TLDR
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system and allow both humans and algorithms to select the changes that would have improved the system performance.Abstract:
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system. Such predictions allow both humans and algorithms to select the changes that would have improved the system performance. This work is illustrated by experiments on the ad placement system associated with the Bing search engine.read more
Citations
More filters
Proceedings Article
Deep Learning with Logged Bandit Feedback
TL;DR: A Counterfactual Risk Minimization (CRM) approach for training deep networks using an equivariant empirical risk estimator with variance regularization, BanditNet, is proposed and it is shown how the resulting objective can be decomposed in a way that allows Stochastic Gradient Descent (SGD) training.
Proceedings ArticleDOI
Counterfactual Story Reasoning and Generation
TL;DR: This paper proposes Counterfactual Story Rewriting: given an original story and an intervening counterfactual event, the task is to minimally revise the story to make it compatible with the given counterfactually event.
Posted Content
Horizon: Facebook’s Open Source Applied Reinforcement Learning Platform
Jason Gauci,Edoardo Conti,Yitao Liang,Kittipat Virochsiri,Yuchen He,Zachary Kaden,Vivek Narayanan,Xiaohui Ye +7 more
TL;DR: Facebook's open source applied reinforcement learning (RL) platform Horizon is presented, which contains workflows to train popular deep RL algorithms and includes data preprocessing, feature transformation, distributed training, counterfactual policy evaluation, optimized serving, and a model-based data understanding tool.
Proceedings Article
{Toward Minimax Off-policy Value Estimation}
TL;DR: It is shown that while the so-called regression estimator is asymptotically optimal, for small sample sizes it may perform suboptimally compared to an ideal oracle up to a multiplicative factor that depends on the number of actions.
Posted Content
Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
TL;DR: A new off-policy estimation method that applies importance sampling directly on the stationary state-visitation distributions to avoid the exploding variance issue faced by existing estimators is proposed.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.