How to Do Things With Words

An Introduction to MultiAgent Systems.

Nos últimos trinta trinta anos, houve profundas mudanças na forma como compreendemos o crime e a justiça criminal. O crime tornou-se um evento simbólico, um verdadeiro teste para a ordem social e para as políticas governamentais, um desafio para a sociedade civil, para a democracia e para os direitos humanos. Segundo David Garland, professor da Faculdade de Direito da New York University, um dos principais autores no campo da Sociologia da Punição e com artigo publicado na Revista de Sociologia e Política , número 13, na modernidade tardia houve uma verdadeira obsessão securitária, direcionando as políticas criminais para um maior rigor em relação às penas e maior intolerância com o criminoso. Há trinta anos, nos EUA e na Inglaterra essa tendência era insuspeita. O livro mostra que os dois países compartilham intrigantes similaridades em suas práticas criminais, a despeito da divisão racial, das desigualdades econômicas e da letalidade violenta que marcam fortemente o cenário americano. Segundo David Garland, encontram-se nos dois países os “mesmos tipos de riscos e inseguranças, a mesma percepção a respeito dos problemas de um controle social não-efetivo, as mesmas críticas da justiça criminal tradicional, e as mesmas ansiedades recorrentes sobre mudança e ordem sociais”1 (GARLAND, 2001, p. 2). O argumento principal da obra é o seguinte: a modernidade tardia, esse distintivo padrão de relações sociais, econômicas e culturais, trouxe consigo um conjunto de riscos, inseguranças e problemas de controle social que deram uma configuração específica às nossas respostas ao crime, ao garantir os altos custos das políticas criminais, o grau máximo de duração das penas e a excessivas taxas de encarceramento.

http://www.scielo.br/pdf/rsocp/n20/n20a15.pdf

The culture of control: crime and social order in contemporary, society

人教版高中英语新课程教材中,语言运用（Using Language）是每个单元必不可少的部分,提供了围绕单元中心话题的听、说、读、写的综合性练习,是单元中心话题的延续和升华.如何设计Using Language部分的教学,使自己的教学模式既不落俗套,又能真正体现新课程标准所倡导的教学理念,正是广大一线英语教师一直努力探索的问题.

打磨Using Language,倡导新理念

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

This paper studies how dual-self (Fudenberg and Levine, 2006) decision-makers can use commitment technologies to combat temptation and implement long-run optimal actions. I consider three types of commitment technologies: carrot contracts (rewards for ‘good’ behavior financed by borrowing from future consumption), stick contracts (self imposed fines for ‘bad’ behavior) and binding commitment. I compare the welfare implications of these contracts and show that dual-self decision-makers strictly prefer to use carrots instead of either sticks or binding commitments. This is for several reasons: sticks are highly vulnerable to trembles (while carrots are not), sticks and binding commitments create a temptation to cancel them (while carrots do not), and finally carrots allow easy tradeoffs between commitment and flexibility (while sticks and binding commitments do not).

How to commit (if you must): Commitment contracts and the dual-self model

A large body of laboratory-based research suggests that arbitrary group assignments (ie. "minimal groups") can lead to in-group bias. We use the release of a popular augmented reality game Pokemon Go to study this phenomenon in a hybrid lab-field experiment. We analyze the behavior of 940 Pokemon Go players randomly matched to other Pokemon Go players to participate in Prisoner's Dilemma games. We find that participants are much more cooperative when their partners is from the same Pokemon Go team, demonstrating an ecologically valid occurrence of the minimal group paradigm. We also use transformed outcome lasso regressions to look for heterogeneity in treatment effects. Machine learning, rather than manual data mining, minimizes overfitting and reduces susceptibility to multiple comparison issues and researcher degrees of freedom. We find one important moderator of the effect: the salience of Pokemon Go. As it's popularity wanes, so does the size of the group bias in our experiments. Thus our full set of results show that real-world minimal group bias is quick to arise but also potentially fragile.

In-Group Favoritism Caused by Pokémon Go and the Use of Machine Learning for Principled Investigation of Potential Moderators

Towards AI that Can Solve Social Dilemmas.

Inferring a person's goal from their behavior is an important problem in applications of AI (e.g. automated assistants, recommender systems). The workhorse model for this task is the rational actor model - this amounts to assuming that people have stable reward functions, discount the future exponentially, and construct optimal plans. Under the rational actor assumption techniques such as inverse reinforcement learning (IRL) can be used to infer a person's goals from their actions. A competing model is the dual-system model. Here decisions are the result of an interplay between a fast, automatic, heuristic-based system 1 and a slower, deliberate, calculating system 2. We generalize the dual system framework to the case of Markov decision problems and show how to compute optimal plans for dual-system agents. We show that dual-system agents exhibit behaviors that are incompatible with rational actor assumption. We show that naive applications of rational-actor IRL to the behavior of dual-system agents can generate wrong inference about the agents' goals and suggest interventions that actually reduce the agent's overall utility. Finally, we adapt a simple IRL algorithm to correctly infer the goals of dual system decision-makers. This allows us to make interventions that help, rather than hinder, the dual-system agent's ability to reach their true goals.

Reinforcement Learning and Inverse Reinforcement Learning with System 1 and System 2

A large body of existing work in social science as well as computer science attempts to infer preferences of individuals from the actions they take. This includes research areas such as industrial organization [4], marketing [1], political science [12], analysis of auctions [3], recommender systems [8], search engine ranking [9], and many others. The workhorse model used either implicitly or explicitly in these disparate literatures is the rational choice model.

Alexander Peysakhovich

Papers

How to commit (if you must): Commitment contracts and the dual-self model

In-Group Favoritism Caused by Pokémon Go and the Use of Machine Learning for Principled Investigation of Potential Moderators

Towards AI that Can Solve Social Dilemmas.

Reinforcement Learning and Inverse Reinforcement Learning with System 1 and System 2

Learning context-dependent preferences from raw data