Dimensions of Reinforcement Learning

http://dklevine.com/archive/refs4122247000000001156.pdf

If multi-agent learning is the answer, what is the question?

https://dialnet.unirioja.es/descarga/articulo/4511388.pdf

Rules of encounter: designing conventions for automated negotiation among computers

There is a veritable menagerie of auctions—single-dimensional, multi-dimensional, single-sided, double-sided, first-price, second-price, English, Dutch, Japanese, sealed-bid—and these have been extensively discussed and analyzed in the economics literature. The main purpose of this article is to survey this literature from a computer science perspective, primarily from the viewpoint of computer scientists who are interested in learning about auction theory, and to provide pointers into the economics literature for those who want a deeper technical understanding. In addition, since auctions are an increasingly important topic in computer science, we also look at work on auctions from the computer science literature. Overall, our aim is to identifying what both these bodies of work these tell us about creating electronic auctions.

/pdf/auctions-and-bidding-a-guide-for-computer-scientists-25bt8rz0ij.pdf

Auctions and bidding: A guide for computer scientists

NetLogo [41, 40] is a freely downloadable, agent-based software package that was created at the Center for Connected Learning and Computer-Based Modeling (CCL) at Northwestern University, directed by Uri Wilensky. It is the latest in a series of multi-agent simulation environments that includes StarLogo [28, 33], which was developed at the MIT Media Lab by Wilensky and Mitchel Resnick. In these environments, agents are represented as ‘‘turtles’’ and are programmed with a variant of Seymour Papert’s LOGO [8, 19]. Designed to be simple enough for children to program, NetLogo illustrates what can happen when populations of turtles are given sets of rules to obey. Despite the entry-level programming interface, NetLogo is capable of quite sophisticated modeling and allows experienced programmers to add their own Java extensions. As a result, NetLogo has been widely used by a broad audience, from elementary school children to academics in the social, computer, and ‘‘hard’’ sciences; and the on-line community page includes models constructed by a wide range of representatives from each of these segments of the population. The NetLogo home page (http://ccl.northwestern.edu/netlogo/) includes a download area, models pages, sample downloadable extensions, user manuals, a FAQ, and links to various resources. The software package comes with a large number of demonstration models, each of which can be executed and modified, since the source for each demo is included in the package. These demos are also available on the ‘‘models’’ portion of the home page, where they can be run in a Web browser. They are contributed by community members and are tested and supported by the development team at CCL. Model application areas range from biology to computer science to art to artificial life and include simulations such as predator-prey, spread of disease, rumor mill, traffic jams, small worlds, fractals, erosion, dining philosophers, cellular automata, crystallization, and radioactivity. In addition, on the ‘‘community models’’ portion of the home page, an even larger and more diverse set of models is available. These, too, have been contributed by community members, but they are not checked by CCL staff; they are supported by their authors, whose contact details are listed on the Web page. NetLogo was first released in 1999, and the most recent release (as of this writing) is version 3.1.2 (August 9, 2006). NetLogo runs on virtually any of today’s most popular platforms—Microsoft Windows, Mac OS-X, and Linux. For each version released, the Web page (URL) enumerates the operating systems on which NetLogo has been successfully installed and executed. NetLogo is written in Java, which aids its portability and compatibility. One of the most convenient features of NetLogo allows programmers to save models as Java applets, permitting seamless publication of simulations from NetLogo’s built-in integrated development environment to a Web page.

/pdf/netlogo-a-multi-agent-simulation-environment-31k7dacee4.pdf

NetLogo, a Multi-agent Simulation Environment

The 2002 Trading Agent Competition (TAC) presented a challenging market game in the domain of travel shopping. One of the pivotal issues in this domain is uncertainty about hotel prices, which have a significant influence on the relative cost of alternative trip schedules. Thus, virtually all participants employ some method for predicting hotel prices. We survey approaches employed in the tournament, finding that agents apply an interesting diversity of techniques, taking into account differing sources of evidence bearing on prices. Based on data provided by entrants on their agents' actual predictions in the TAC-02 finals and semifinals, we analyze the relative efficacy of these approaches. The results show that taking into account game-specific information about flight prices is a major distinguishing factor. Machine learning methods effectively induce the relationship between flight and hotel prices from game data, and a purely analytical approach based on competitive equilibrium analysis achieves equal accuracy with no historical data. Employing a new measure of prediction quality, we relate absolute accuracy to bottom-line performance in the game.

/pdf/price-prediction-in-a-trading-agent-competition-rqcbvdpr1c.pdf

Price prediction in a trading agent competition

/pdf/price-prediction-in-a-trading-agent-competition-41rpr1jwjf.pdf

Price Prediction in a Trading Agent Competition

To deal with exponential growth in the size of a game with the number of agents, we propose an approximation based on a hierarchy of reduced games. The reduced game achieves savings by restricting the number of agents playing any strategy to fixed multiples. We validate the idea through experiments on randomly generated local-effect games. An extended application to strategic reasoning about a complex trading scenario motivates the approach, and demonstrates methods for game-theoretic reasoning over incompletely-specified games at multiple levels of granularity. Motivation Consider the task of selecting among a large set of strategies to play in an 8-player game. Through careful judgment you manage to narrow down the candidates to a reasonable number of strategies (say 35). Because the performance of a strategy for one agent depends on the strategies of the other seven, you wish to undertake a game-theoretic analysis of the situation. Determining the payoff for a particular strategy profile is expensive, however, as your observations of prior game instances are quite limited, and the only operational description of the game is in the form of a simulator that takes a non-negligible time (say 10 minutes) to produce one outcome. Moreover, since the environment is stochastic, numerous samples (say 12) are required to produce a reliable estimate for even one profile. At two hours per profile, ex

/pdf/approximate-strategic-reasoning-through-hierarchical-2nbah6p7e6.pdf

Approximate strategic reasoning through hierarchical reduction of large symmetric games

TAC-02 was the third in a series of Trading Agent Competition events fostering research in automating trading strategies by showcasing alternate approaches in an open-invitation market game. TAC presents a challenging travel-shopping scenario where agents must satisfy client preferences for complementary and substitutable goods by interacting through a variety of market types. Michigan's entry, Walverine, bases its decisions on a competitive (Walrasian) analysis of the TAC travel economy. Using this Walrasian model, we construct a decision-theoretic formulation of the optimal bidding problem, which Walverine solves in each round of bidding for each good. Walverine's optimal bidding approach, as well as several other features of its overall strategy, are potentially applicable in a broad class of trading environments.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1046&context=sis_research

Walverine: a Walrasian trading agent

We investigate the space of two-sided multiattribute auctions, focusing on the relationship between constraints on the offers traders can express through bids, and the resulting computational problem of determining an optimal set of trades. We develop a formal semantic framework for characterizing expressible offers, and show conditions under which the allocation problem can be separated into first identifying optimal pairwise trades and subsequently optimizing combinations of those trades. We analyze the bilateral matching problem while taking into consideration relevant results from multiattribute utility theory. Network flow models we develop for computing global allocations facilitate classification of the problem space by computational complexity, and provide guidance for developing solution algorithms. Experimental trials help distinguish tractable problem classes for proposed solution techniques.

Kevin M. Lochner

Papers

Price prediction in a trading agent competition

Price Prediction in a Trading Agent Competition

Approximate strategic reasoning through hierarchical reduction of large symmetric games

Walverine: a Walrasian trading agent

Bid expressiveness and clearing algorithms in multiattribute double auctions