Showing papers by "Alexander Peysakhovich published in 2016"

Multi-Agent Cooperation and the Emergence of (Natural) Language

Abstract: The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitrary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents' code into natural language. Both of these are necessary steps towards developing machines that are able to communicate with humans productively.

Habits of Virtue: Creating Norms of Cooperation and Defection in the Laboratory

Abstract: What explains variability in norms of cooperation across organizations and cultures? One answer comes from the tendency of individuals to internalize typically successful behaviors as norms. Different institutional structures can cause different behavioral norms to be internalized. These norms are then carried over into atypical situations beyond the reach of the institution. Here, we experimentally demonstrate such spillovers. First, we immerse subjects in environments that do or do not support cooperation using repeated prisoner’s dilemmas. Afterwards, we measure their intrinsic prosociality in one-shot games. Subjects from environments that support cooperation are more prosocial, more likely to punish selfishness, and more trusting in general. Furthermore, these effects are most pronounced among subjects who use heuristics, suggesting that intuitive processes play a key role in the spillovers we observe. Our findings help to explain variation in one-shot anonymous cooperation, linking this intrinsicall...

Multi-Agent Cooperation and the Emergence of (Natural) Language

Abstract: The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitrary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents' code into natural language. Both of these are necessary steps towards developing machines that are able to communicate with humans productively.

Asymmetric Effects of Favorable and Unfavorable Information on Decision Making Under Ambiguity

Abstract: Most daily decisions involve uncertainty about outcome probabilities arising from incomplete knowledge, i.e., ambiguity. We explore how the addition of partial information affects these types of choices using theoretical and empirical methods. Our experiments in both gain and loss domains demonstrate that when such information supports a favorable outcome, it strongly increases valuation of an ambiguous financial prospect. However, when information supports an unfavorable outcome, it has significantly less impact. We find that two mechanisms drive this asymmetry. First, unfavorable information decreases estimates of a good outcome occurring but also reduces aversive uncertainty. These factors act in opposition, minimizing the effects of unfavorable information. Second, when information can be subjectively interpreted, unfavorable information is less likely to be integrated into evaluations. Our findings reveal mechanisms not captured by traditional models of decision making under uncertainty and highlight...

Combining observational and experimental data to find heterogeneous treatment effects.

Abstract: Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is high-dimensional and our priors are weak about which particular covariates are important. However, there are often observational data sets available that are orders of magnitude larger. We propose a method to combine these two data sources to estimate heterogeneous treatment effects. First, we use observational time series data to estimate a mapping from covariates to unit-level effects. These estimates are likely biased but under some conditions the bias preserves unit-level relative rank orderings. If these conditions hold, we only need sufficient experimental data to identify a monotonic, one-dimensional transformation from observationally predicted treatment effects to real treatment effects. This reduces power demands greatly and makes the detection of heterogeneous effects much easier. As an application, we show how our method can be used to improve Facebook page recommendations.

The Good, the Bad, and the Unflinchingly Selfish: Cooperative Decision-Making Can Be Predicted with High Accuracy Using Only Three Behavioral Types

Abstract: The human willingness to pay costs to benefit anonymous others is often explained by social preferences: rather than only valuing their own material payoff, people also care in some fashion about the outcomes of others. But how successful is this concept of outcome-based social preferences for actually predicting out-of-sample behavior? We investigate this question by having 1067 human subjects each make 20 cooperation decisions, and using machine learning to predict their last 5 choices based on their first 15. We find that decisions can be predicted with high accuracy by models that include outcome-based features and allow for heterogeneity across individuals in baseline cooperativeness and the weights placed on the outcome-based features (AUC=0.89). It is not necessary, however, to have a fully heterogeneous model -- excellent predictive power (AUC=0.88) is achieved by a model that allows three different sets of baseline cooperativeness and feature weights (i.e. three behavioral types), defined based on the participant's cooperation frequency in the 15 training trials: those who cooperated at least half the time, those who cooperated less than half the time, and those who never cooperated. Finally, we provide evidence that this inclination to cooperate cannot be well proxied by other personality/morality survey measures or demographics, and thus is a natural kind (or "cooperative phenotype")

The Good, the Bad, and the Unflinchingly Selfish: Cooperative Decision-Making can be Predicted with high Accuracy when using only Three Behavioral Types

Abstract: The human willingness to pay costs to benefit anonymous others is often explained by social preferences: rather than only valuing their own material payoff, people also care in some fashion about the outcomes of others. But how successful is this concept of outcome-based social preferences for actually predicting out-of-sample behavior? We investigate this question by having 1067 human subjects each make 20 cooperation decisions, and using machine learning to predict their last 5 choices based on their first 15. We find that decisions can be predicted with high accuracy by models that include outcome-based features and allow for heterogeneity across individuals in baseline cooperativeness and the weights placed on the outcome-based features (AUC=0.89). It is not necessary, however, to have a fully heterogeneous model -- excellent predictive power (AUC=0.88) is achieved by a model that allows three different sets of baseline cooperativeness and feature weights (i.e. three behavioral types), defined based on the participant's cooperation frequency in the 15 training trials: those who cooperated at least half the time, those who cooperated less than half the time, and those who never cooperated. Finally, we provide evidence that this inclination to cooperate cannot be well proxied by other personality/morality survey measures or demographics, and thus is a natural kind (or "cooperative phenotype").

Paying (for) Attention: The Impact of Information Processing Costs on Bayesian Inference

Abstract: Human information processing is often modeled as costless Bayesian inference. However, research in psychology shows that attention is a computationally costly and potentially limited resource. We thus study Bayesian agents for whom computing posterior beliefs is costly; such agents face a tradeoffs between economizing on attention costs and having more accurate beliefs. We show that even small processing costs can lead to significant departures from the standard costless processing model. There exist situations in which beliefs can cycle persistently and never converge. In addition, when updating is costly, agents are more sensitive to signals about rare events than to signals about common events. Thus, these individuals can permanently overestimate the likelihood of rare events. There is a commonly held assumption in economics that individuals will converge to correct beliefs/optimal behavior given sufficient experience. Our results contribute to a growing literature in psychology, neuroscience, and behavioral economics suggesting that this assumption is both theoretically and empirically fragile.