How to Do Things With Words

An Introduction to MultiAgent Systems.

Nos últimos trinta trinta anos, houve profundas mudanças na forma como compreendemos o crime e a justiça criminal. O crime tornou-se um evento simbólico, um verdadeiro teste para a ordem social e para as políticas governamentais, um desafio para a sociedade civil, para a democracia e para os direitos humanos. Segundo David Garland, professor da Faculdade de Direito da New York University, um dos principais autores no campo da Sociologia da Punição e com artigo publicado na Revista de Sociologia e Política , número 13, na modernidade tardia houve uma verdadeira obsessão securitária, direcionando as políticas criminais para um maior rigor em relação às penas e maior intolerância com o criminoso. Há trinta anos, nos EUA e na Inglaterra essa tendência era insuspeita. O livro mostra que os dois países compartilham intrigantes similaridades em suas práticas criminais, a despeito da divisão racial, das desigualdades econômicas e da letalidade violenta que marcam fortemente o cenário americano. Segundo David Garland, encontram-se nos dois países os “mesmos tipos de riscos e inseguranças, a mesma percepção a respeito dos problemas de um controle social não-efetivo, as mesmas críticas da justiça criminal tradicional, e as mesmas ansiedades recorrentes sobre mudança e ordem sociais”1 (GARLAND, 2001, p. 2). O argumento principal da obra é o seguinte: a modernidade tardia, esse distintivo padrão de relações sociais, econômicas e culturais, trouxe consigo um conjunto de riscos, inseguranças e problemas de controle social que deram uma configuração específica às nossas respostas ao crime, ao garantir os altos custos das políticas criminais, o grau máximo de duração das penas e a excessivas taxas de encarceramento.

http://www.scielo.br/pdf/rsocp/n20/n20a15.pdf

The culture of control: crime and social order in contemporary, society

人教版高中英语新课程教材中,语言运用（Using Language）是每个单元必不可少的部分,提供了围绕单元中心话题的听、说、读、写的综合性练习,是单元中心话题的延续和升华.如何设计Using Language部分的教学,使自己的教学模式既不落俗套,又能真正体现新课程标准所倡导的教学理念,正是广大一线英语教师一直努力探索的问题.

打磨Using Language,倡导新理念

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

We consider the problem of dividing items between individuals in a way that is fair both in the sense of distributional fairness and in the sense of not having disparate impact across protected classes. An important existing mechanism for distributionally fair division is competitive equilibrium from equal incomes (CEEI). Unfortunately, CEEI will not, in general, respect disparate impact constraints. We consider two types of disparate impact measures: requiring that allocations be similar across protected classes and requiring that average utility levels be similar across protected classes. We modify the standard CEEI algorithm in two ways: equitable equilibrium from equal incomes, which removes disparate impact in allocations, and competitive equilibrium from equitable incomes which removes disparate impact in attained utility levels. We show analytically that removing disparate impact in outcomes breaks several of CEEI's desirable properties such as envy, regret, Pareto optimality, and incentive compatibility. By contrast, we can remove disparate impact in attained utility levels without affecting these properties. Finally, we experimentally evaluate the tradeoffs between efficiency, equity, and disparate impact in a recommender-system based market.

Fair Division Without Disparate Impact

We consider the problem of using logged data to make predictions about what would happen if we changed the `rules of the game' in a multi-agent system. This task is difficult because in many cases we observe actions individuals take but not their private information or their full reward functions. In addition, agents are strategic, so when the rules change, they will also change their actions. Existing methods (e.g. structural estimation, inverse reinforcement learning) assume that agents' behavior comes from optimizing some utility or that the system is in equilibrium. They make counterfactual predictions by using observed actions to learn the underlying utility function (a.k.a. type) and then solving for the equilibrium of the counterfactual environment. This approach imposes heavy assumptions such as the rationality of the agents being observed and a correct model of the environment and agents' utility functions. We propose a method for analyzing the sensitivity of counterfactual conclusions to violations of these assumptions, which we call robust multi-agent counterfactual prediction (RMAC). We provide a first-order method for computing RMAC bounds. We apply RMAC to classic environments in market design: auctions, school choice, and social choice.

/pdf/robust-multi-agent-counterfactual-prediction-3z6squ7uz2.pdf

Robust Multi-agent Counterfactual Prediction

Human information processing is often modeled as costless Bayesian inference. However, research in psychology shows that attention is a computationally costly and potentially limited resource. We thus study Bayesian agents for whom computing posterior beliefs is costly; such agents face a tradeoffs between economizing on attention costs and having more accurate beliefs. We show that even small processing costs can lead to significant departures from the standard costless processing model. There exist situations in which beliefs can cycle persistently and never converge. In addition, when updating is costly, agents are more sensitive to signals about rare events than to signals about common events. Thus, these individuals can permanently overestimate the likelihood of rare events. There is a commonly held assumption in economics that individuals will converge to correct beliefs/optimal behavior given sufficient experience. Our results contribute to a growing literature in psychology, neuroscience, and behavioral economics suggesting that this assumption is both theoretically and empirically fragile.

Paying (for) Attention: The Impact of Information Processing Costs on Bayesian Inference

The human willingness to pay costs to benefit anonymous others is often explained by social preferences: rather than only valuing their own material payoff, people also include the payoffs of others in their utility function. But how successful is this concept of outcome-based social preferences for actually predicting out-of-sample behavior? We investigate this question by having 1,067 participants each make 20 Dictator Game decisions with randomized parameters (e.g., outcomes for the self, for the other, benefit/cost ratio of pro-sociality). We then use machine learning to try to predict behavior by each participant in each decision. A representative agent model (a small, shared, set of parameters) predicts better than random but still quite poorly (AUC = 0.69). Allowing for full heterogeneity across individuals in the mapping from decision-parameters to outcome yields good predictive performance (AUC = 0.89). However, this heterogeneous model is complicated and unwieldy, thus we also investigate whether a simpler model can yield similar performance. We find that the vast majority of the predictive power (AUC = 0.88) is achieved by a model that allows for three behavioral types. Finally, we show that cannot be well proxied for by other measures in psychology. This final analysis adds further evidence to the literature that human “cooperative phenotypes” are indeed meaningful, relatively orthogonal person-level traits.

The Good, the Bad, and the Unflinchingly Selfish: Pro-sociality can be Well Predicted Using Payoffs and Three Behavioral Types

Many applications in preference learning assume that decisions come from the maximization of a stable utility function. Yet a large experimental literature shows that individual choices and judgements can be affected by "irrelevant" aspects of the context in which they are made. An important class of such contexts is the composition of the choice set. In this work, our goal is to discover such choice set effects from raw choice data. We introduce an extension of the Multinomial Logit (MNL) model, called the context dependent random utility model (CDM), which allows for a particular class of choice set effects. We show that the CDM can be thought of as a second-order approximation to a general choice system, can be inferred optimally using maximum likelihood and, importantly, is easily interpretable. We apply the CDM to both real and simulated choice data to perform principled exploratory analyses for the presence of choice set effects.

Alexander Peysakhovich

Papers

Fair Division Without Disparate Impact

Robust Multi-agent Counterfactual Prediction

Paying (for) Attention: The Impact of Information Processing Costs on Bayesian Inference

The Good, the Bad, and the Unflinchingly Selfish: Pro-sociality can be Well Predicted Using Payoffs and Three Behavioral Types

Discovering Context Effects from Raw Choice Data