Minority game with arbitrary cutoffs
TL;DR: In this paper, a model of a competing population of N adaptive agents, with similar capabilities, repeatedly deciding whether to attend a bar with an arbitrary cutoff L. Decisions are based upon past outcomes.
Abstract: We study a model of a competing population of N adaptive agents, with similar capabilities, repeatedly deciding whether to attend a bar with an arbitrary cutoff L. Decisions are based upon past outcomes. The agents are only told whether the actual attendance is above or below L. For L-> N/2, the game reproduces the main features of Challet and Zhang's minority game. As L is lowered, however, the mean attendances in different runs tend to divide into two groups. The corresponding standard deviations for these two groups are very different. This grouping effect results from the dynamical feedback governing the game's time-evolution, and is not reproduced if the agents are fed a random history.
Citations
More filters
TL;DR: In this paper, a prototype model of stock market is introduced and studied numerically, in which traders trade according to their own strategy, to accumulate their assets by speculating on the price's fluctuations which are produced by themselves.
Abstract: A prototype model of stock market is introduced and studied numerically. In this self-organized system, we consider only the interaction among traders without external influences. Agents trade according to their own strategy, to accumulate his assets by speculating on the price's fluctuations which are produced by themselves. The model reproduced rather realistic price histories whose statistical properties are also similar to those observed in real markets.
149 citations
TL;DR: In this paper, a model of heterogeneous, inductive rational agents inspired by the El Farol Bar problem and the Minority Game is discussed, where agents follow a simple reinforcement learning dynamics where the reinforcement, for each of their available strategies, is related to the payoff delivered by that strategy.
Abstract: We discuss a model of heterogeneous, inductive rational agents inspired by the El Farol Bar problem and the Minority Game. As in markets, agents interact through a collective aggregate variable — which plays a role similar to price — whose value is fixed by all of them. Agents follow a simple reinforcement-learning dynamics where the reinforcement, for each of their available strategies, is related to the payoff delivered by that strategy. We derive the exact solution of the model in the “thermodynamic” limit of infinitely many agents using tools of statistical physics of disordered systems. Our results show that the impact of agents on the market price plays a key role: even though price has a weak dependence on the behavior of each individual agent, the collective behavior crucially depends on whether agents account for such dependence or not. Remarkably, if the adaptive behavior of agents accounts even “infinitesimally” for this dependence they can, in a whole range of parameters, reduce global fluctuations by a finite amount. Both global efficiency and individual utility improve with respect to a “price taker” behavior if agents account for their market impact.
128 citations
Cites background from "Minority game with arbitrary cutoff..."
...Finally note that the El Farol bar problem has a similar structure but with A replaced by (A − A0) in Eq. (1) where A0 is related to the bar’s comfort level [1, 19 ]....
[...]
19 Jul 2004
TL;DR: The results of the experiments suggest that reinforcement learning can be used to improve the quality of resource allocation in large scale heterogenous system.
Abstract: In this paper we study a minimalist decentralized algorithm for resource allocation in a simplified Grid-like environment. We consider a system consisting of large number of heterogenous reinforcement learning agents that share common resources for their computational needs. There is no communication between the agents: the only information that agents receive is the (expected) completion time of a job it submitted to a particular resource and which serves as a reinforcement signal for the agent. The results of our experiments suggest that reinforcement learning can be used to improve the quality of resource allocation in large scale heterogenous system.
128 citations
Cites background from "Minority game with arbitrary cutoff..."
...Congestion games [22] and minority games [2, 5, 4, 16] are just two examples of applying game dynamics to the resource allocation problem....
[...]
TL;DR: A framework that combines Unmanned Aerial Vehicle-support with wireless powered communication techniques to further improve energy efficiency in a distributed non-orthogonal multiple access (NOMA) PSN is proposed and the numerical results demonstrate its energy efficiency, robustness, and scalability.
89 citations
TL;DR: The results of the experiments suggest that even simple reinforcement learning can indeed be used to achieve load balanced resource allocation in large scale heterogenous system.
Abstract: One of the main challenges in Grid computing is efficient allocation of resources (CPU – hours, network bandwidth, etc.) to the tasks submitted by users. Due to the lack of centralized control and the dynamic/stochastic nature of resource availability, any successful allocation mechanism should be highly distributed and robust to the changes in the Grid environment. Moreover, it is desirable to have an allocation mechanism that does not rely on the availability of coherent global information. In this paper we examine a simple algorithm for distributed resource allocation in a simplified Grid-like environment that meets the above requirements. Our system consists of a large number of heterogenous reinforcement learning agents that share common resources for their computational needs. There is no explicit communication or interaction between the agents: the only information that agents receive is the expected response time of a job it submitted to a particular resource, which serves as a reinforcement signal for the agent. The results of our experiments suggest that even simple reinforcement learning can indeed be used to achieve load balanced resource allocation in large scale heterogenous system.
78 citations
References
More filters
TL;DR: Interesting cooperation and competition patterns of the society seem to arise and to be responsive to the payoff function.
Abstract: A binary game is introduced and analysed. N players have to choose one of the two sides independently and those on the minority side win. Players use a finite set of ad hoc strategies to make their decision, based on the past record. The analysing power is limited and can adapt when necessary. Interesting cooperation and competition patterns of the society seem to arise and to be responsive to the payoff function.
1,123 citations
TL;DR: In this article, the authors argue that large variations in stock prices happen with sufficient frequency to raise doubts about existing models, which all fail to account for non-Gaussian statistics, and argue that the large variations may be due to a crowd effect, where agents imitate each other's behavior.
Abstract: Large variations in stock prices happen with sufficient frequency to raise doubts about existing models, which all fail to account for non-Gaussian statistics. We construct simple models of a stock market, and argue that the large variations may be due to a crowd effect, where agents imitate each other's behavior. The variations over different time scales can be related to each other in a systematic way, similar to the Levy stable distribution proposed by Mandelbrot to describe real market indices. In the simplest least realistic case, exact results for the statistics of the variations are derived by mapping onto a model of diffusing and annihilating particles, which has been solved by quantum field theory methods. When the agents imitate each other and respond to recent market volatility, different scaling behavior is obtained. In this case, the statistics of price variations is consistent with empirical observations. The interplay between “rational” traders whose behavior is derived from fundamental analysis of the stock, including dividends, and “noise traders”, whose behavior is governed solely by studying the market dynamics and the behavior of other traders, is investigated. When the relative number of rational traders is small, “bubbles” often occur, where the market price moves outside the range justified by fundamental market analysis. When the number of rational traders is larger, the market price is generally locked within the price range they define.
415 citations
TL;DR: In this article, the authors investigated several properties of the minority game and gave an analytical expression of σ 2/N in the N ⪡ 2 M region. But they did not consider the influence of identical players on their gain and on the systems performance.
Abstract: We investigate further several properties of the minority game we have recently introduced. We explain the origin of the phase transition and give an analytical expression of σ2/N in the N ⪡ 2 M region. The ability of the players to learn a given payoff is also analyzed, and we show that the Darwinian evolution process tends to a self-organized state, in particular, the lifetime distribution is a power-law with exponent −2. Furthermore, we study the influence of identical players on their gain and on the systems performance. Finally, we show that large brains always take advantage of small brains.
383 citations
TL;DR: In this paper, the authors study the dynamics of a system composed of interacting units each with a complex internal structure comprising many subunits and treat the case in which each subunit grows in a multiplicative manner.
Abstract: We study the dynamics of a system composed of interacting units each with a complex internal structure comprising many subunits and treat the case in which each subunit grows in a multiplicative manner. We propose a model for such systems in which the interaction among the units is treated in a mean field approximation and the interaction among subunits is nonlinear. We test the model and find agreement between our predictions and empirical results based on a large economics database spanning 20 years. [S0031-9007(98)05355-1]
255 citations