How to Do Things With Words

An Introduction to MultiAgent Systems.

Nos últimos trinta trinta anos, houve profundas mudanças na forma como compreendemos o crime e a justiça criminal. O crime tornou-se um evento simbólico, um verdadeiro teste para a ordem social e para as políticas governamentais, um desafio para a sociedade civil, para a democracia e para os direitos humanos. Segundo David Garland, professor da Faculdade de Direito da New York University, um dos principais autores no campo da Sociologia da Punição e com artigo publicado na Revista de Sociologia e Política , número 13, na modernidade tardia houve uma verdadeira obsessão securitária, direcionando as políticas criminais para um maior rigor em relação às penas e maior intolerância com o criminoso. Há trinta anos, nos EUA e na Inglaterra essa tendência era insuspeita. O livro mostra que os dois países compartilham intrigantes similaridades em suas práticas criminais, a despeito da divisão racial, das desigualdades econômicas e da letalidade violenta que marcam fortemente o cenário americano. Segundo David Garland, encontram-se nos dois países os “mesmos tipos de riscos e inseguranças, a mesma percepção a respeito dos problemas de um controle social não-efetivo, as mesmas críticas da justiça criminal tradicional, e as mesmas ansiedades recorrentes sobre mudança e ordem sociais”1 (GARLAND, 2001, p. 2). O argumento principal da obra é o seguinte: a modernidade tardia, esse distintivo padrão de relações sociais, econômicas e culturais, trouxe consigo um conjunto de riscos, inseguranças e problemas de controle social que deram uma configuração específica às nossas respostas ao crime, ao garantir os altos custos das políticas criminais, o grau máximo de duração das penas e a excessivas taxas de encarceramento.

http://www.scielo.br/pdf/rsocp/n20/n20a15.pdf

The culture of control: crime and social order in contemporary, society

人教版高中英语新课程教材中,语言运用（Using Language）是每个单元必不可少的部分,提供了围绕单元中心话题的听、说、读、写的综合性练习,是单元中心话题的延续和升华.如何设计Using Language部分的教学,使自己的教学模式既不落俗套,又能真正体现新课程标准所倡导的教学理念,正是广大一线英语教师一直努力探索的问题.

打磨Using Language,倡导新理念

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

Cooperation is central to human societies. Yet relatively little is known about the cognitive underpinnings of cooperative decision-making. Does cooperation require deliberate self-restraint? Or is spontaneous prosociality reined in by calculating self-interest? Here we present a theory of why (and for whom) intuition favors cooperation: cooperation is typically advantageous in everyday life, leading to the formation of generalized cooperative intuitions. Deliberation, by contrast, adjusts behavior towards the optimum for a given situation. Thus, in one-shot anonymous interactions where selfishness is optimal, intuitive responses tend to be more cooperative than deliberative responses. We test this “Social Heuristics Hypothesis” by aggregating across every cooperation experiment using time pressure we conducted over a two-year period (15 studies and 6,910 decisions), as well as performing a novel time pressure experiment. Doing so demonstrates a positive average effect of time pressure on cooperation. We also find substantial variation in this effect, and show that this variation is partly explained by previous experience with one-shot lab experiments.

Social Heuristics Shape Intuitive Cooperation

Cooperation is central to human societies. Yet relatively little is known about the cognitive underpinnings of cooperative decision making. Does cooperation require deliberate self-restraint? Or is spontaneous prosociality reined in by calculating self-interest? Here we present a theory of why (and for whom) intuition favors cooperation: cooperation is typically advantageous in everyday life, leading to the formation of generalized cooperative intuitions. Deliberation, by contrast, adjusts behaviour towards the optimum for a given situation. Thus, in one-shot anonymous interactions where selfishness is optimal, intuitive responses tend to be more cooperative than deliberative responses. We test this ‘social heuristics hypothesis’ by aggregating across every cooperation experiment using time pressure that we conducted over a 2-year period (15 studies and 6,910 decisions), as well as performing a novel time pressure experiment. Doing so demonstrates a positive average effect of time pressure on cooperation. We also find substantial variation in this effect, and show that this variation is partly explained by previous experience with one-shot lab experiments.

/pdf/social-heuristics-shape-intuitive-cooperation-4yop6bk2xp.pdf

Social heuristics shape intuitive cooperation

The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitrary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents' code into natural language. Both of these are necessary steps towards developing machines that are able to communicate with humans productively.

/pdf/multi-agent-cooperation-and-the-emergence-of-natural-2wdlz7bymx.pdf

Multi-Agent Cooperation and the Emergence of (Natural) Language

An intergenerational cooperation game has been developed to study decision-making regarding resource use: when decisions about resource extraction were made individually the resource was rapidly depleted by a minority of defectors; the resource was sustainably maintained across generations, however, when decisions were made democratically by voting. Cooperation is often seen in experimental economic games because actions can be reciprocated. But this trait is of no help in one of the most important types of cooperation: cooperation with future generations who cannot reciprocate if we refrain from overexploiting their resources. To test the conditions under which cooperation with the future can occur, Oliver Hauser et al. developed a laboratory model of cooperation — the Intergenerational Goods Game (IGG) — that differs from previous games in which selfishness creates social efficiency losses for group members. Instead, selfishness negatively impacts subsequent groups. Experiments involving more than 2,000 subjects demonstrate that when decisions on resource extraction are made individually, the resource is rapidly depleted by defectors. But when participants are forced to vote on how the resource should be exploited, it is exploited sustainably across generations. Voting works for two reasons. It allows a majority of cooperators to constrain a minority of defectors, and as all players receive the same amount after a vote, cooperators need not worry about losing out relative to others. Overexploitation of renewable resources today has a high cost on the welfare of future generations1,2,3,4,5. Unlike in other public goods games6,7,8,9, however, future generations cannot reciprocate actions made today. What mechanisms can maintain cooperation with the future? To answer this question, we devise a new experimental paradigm, the ‘Intergenerational Goods Game’. A line-up of successive groups (generations) can each either extract a resource to exhaustion or leave something for the next group. Exhausting the resource maximizes the payoff for the present generation, but leaves all future generations empty-handed. Here we show that the resource is almost always destroyed if extraction decisions are made individually. This failure to cooperate with the future is driven primarily by a minority of individuals who extract far more than what is sustainable. In contrast, when extractions are democratically decided by vote, the resource is consistently sustained. Voting10,11,12,13,14,15 is effective for two reasons. First, it allows a majority of cooperators to restrain defectors. Second, it reassures conditional cooperators16 that their efforts are not futile. Voting, however, only promotes sustainability if it is binding for all involved. Our results have implications for policy interventions designed to sustain intergenerational public goods.

/pdf/cooperating-with-the-future-1bkrnqvbcy.pdf

Cooperating with the future

A key element of human morality is prosocial behavior. Humans are unique among animals in their willingness to pay costs to benefit unrelated friends and strangers. This cooperation is a critical part of our identity as moral beings. Here, we ask why humans cooperate, and what can explain variation in cooperative behavior across social groups, individuals, and situations. We begin by defining cooperation, and discussing economic games as laboratory models for studying cooperative behavior. Next we provide an overview of several mechanisms that can make cooperation advantageous in the long run: direct reciprocity, indirect reciprocity, and institutions. We then turn to cooperative behavior that occurs beyond the reach of such mechanisms and thus is truly costly. We discuss the Social Heuristics Hypothesis, which contends that when individuals learn that cooperative strategies are generally advantageous, they become internalized as intuitive defaults. These defaults can spill over to produce “irrational” cooperative behavior even when externally imposed incentives are absent. Thus, we suggest that daily experience plays an important role in shaping culturally varying conceptions of cooperation as a virtuous behavior. Finally, we close by discussing open questions for future research on cooperation.

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2418439_code1038894.pdf?abstractid=2418439&mirid=1

Alexander Peysakhovich

Papers

Social Heuristics Shape Intuitive Cooperation

Social heuristics shape intuitive cooperation

Multi-Agent Cooperation and the Emergence of (Natural) Language

Cooperating with the future

Why We Cooperate