Open AccessJournal Article
Counterfactual reasoning and learning systems: the example of computational advertising
Léon Bottou,Jonas Peters,Joaquin Quiñonero-Candela,Denis X. Charles,D. Max Chickering,Elon Portugaly,Dipankar Ray,Patrice Y. Simard,Ed Snelson +8 more
Reads0
Chats0
TLDR
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system and allow both humans and algorithms to select the changes that would have improved the system performance.Abstract:
This work shows how to leverage causal inference to understand the behavior of complex learning systems interacting with their environment and predict the consequences of changes to the system. Such predictions allow both humans and algorithms to select the changes that would have improved the system performance. This work is illustrated by experiments on the ad placement system associated with the Bing search engine.read more
Citations
More filters
Proceedings ArticleDOI
Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding
Ethan K. Gordon,Sumegh Roychowdhury,Tapomayukh Bhattacharjee,Kevin Jamieson,Siddhartha S. Srinivasa +4 more
TL;DR: In this paper, a modified linear contextual bandit framework augmented with post hoc context observed after action selection is proposed to increase learning speed and reduce cumulative regret, which is more pronounced when the dimensionality of the context is large relative to the post-hoc context.
Posted Content
Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits
TL;DR: The authors improved the DR estimator by adaptively weighting observations to control its variance, and showed that a t-statistic based on the improved estimator is asymptotically normal under certain conditions, allowing them to form confidence intervals and test hypotheses.
Proceedings ArticleDOI
Monte Carlo Estimates of Evaluation Metric Error and Bias
Mucun Tian,Michael D. Ekstrand +1 more
TL;DR: Simulation of the recommender data generation and evaluation processes is used to quantify the extent of evaluation metric errors and assess their sensitivity to various assumptions.
Book ChapterDOI
Online Experimentation for Information Retrieval
TL;DR: Online experimentation for information retrieval (IR) focuses on insights that can be gained from user interactions with IR systems, such as web search engines, through A/B testing.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
MonographDOI
Causality: models, reasoning, and inference
TL;DR: The art and science of cause and effect have been studied in the social sciences for a long time as mentioned in this paper, see, e.g., the theory of inferred causation, causal diagrams and the identification of causal effects.
Journal ArticleDOI
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.