Showing papers on "Stochastic game published in 2017"

Information Design: A Unified Perspective

Abstract: Fixing a game with uncertain payoffs, information design identifies the information structure and equilibrium that maximizes the payoff of an information designer. We show how this perspective unifies existing work, including that on communication in games (Myerson (1991)), Bayesian persuasion (Kamenica and Gentzkow (2011)) and some of our own recent work. Information design has a literal interpretation, under which there is a real information designer who can commit to the choice of the best information structure (from her perspective) for a set of participants in a game. We emphasize a metaphorical interpretation, under which the information design problem is used by the analyst to characterize play in the game under many different information structures.

Formulating an n-person noncooperative game as a tensor complementarity problem

Abstract: In this paper, we consider a class of n-person noncooperative games, where the utility function of every player is given by a homogeneous polynomial defined by the payoff tensor of that player, which is a natural extension of the bimatrix game where the utility function of every player is given by a quadratic form defined by the payoff matrix of that player. We will call such a problem the multilinear game. We reformulate the multilinear game as a tensor complementarity problem, a generalization of the linear complementarity problem; and show that finding a Nash equilibrium point of the multilinear game is equivalent to finding a solution of the resulted tensor complementarity problem. Especially, we present an explicit relationship between the solutions of the multilinear game and the tensor complementarity problem, which builds a bridge between these two classes of problems. We also apply a smoothing-type algorithm to solve the resulted tensor complementarity problem and give some preliminary numerical results for solving the multilinear games.

Risk-Averse Energy Trading in Multienergy Microgrids: A Two-Stage Stochastic Game Approach

Abstract: Multienergy microgrids are a promising solution to improve overall energy (electricity, cooling, heating, etc.) efficiency. In this paper, a new optimal energy trading strategy is developed considering the risk from uncertain energy supply and demand in a set of individual multienergy microgrids. According to the historical data about energy supply of each microgrid, an aggregator aims to maximize each microgrid's profit while minimizing the risk of overbidding for renewable energy resources trading based microgrids. A novel two-stage stochastic game model with Cournot Nash pricing mechanism and the conditional value-at-risk criterion is proposed to characterize the payoff function of each microgrid. The sample average approximation (SAA) technique is employed to approximate the stochastic Nash equilibrium of the game model. The existence of the SAA Nash equilibrium is investigated and the corresponding Nash equilibrium seeking algorithm is also realized in a distributed manner. The proposed method is validated by numerical simulations on real-world data collected in Australia, and the results show that the SAA Nash equilibrium based strategy can effectively reduce the risk of not meeting the demand and improve the economic benefits for each microgrid.

A quest toward a mathematical theory of the dynamics of swarms

Abstract: This paper addresses some preliminary steps toward the modeling and qualitative analysis of swarms viewed as living complex systems. The approach is based on the methods of kinetic theory and statistical mechanics, where interactions at the microscopic scale are nonlocal, nonlinearly additive and modeled by theoretical tools of stochastic game theory. Collective learning theory can play an important role in the modeling approach. We present a kinetic equation incorporating the Cucker–Smale flocking force and stochastic game theoretic interactions in collision operators. We also present a sufficient framework leading to the asymptotic velocity alignment and global existence of smooth solutions for the proposed kinetic model with a special kernel. Analytic results on the global existence and flocking dynamics are presented, while the last part of the paper looks ahead to research perspectives.

Risk Analysis and Enhancement of Cooperation Yielded by the Individual Reputation in the Spatial Public Goods Game

Abstract: In this paper, an evolutionary game model is put forward to investigate the evolution and risk analysis of cooperation under the spatial public goods game (PGG), in which the individual reputation is obviously utilized to cut down the individual risk of being exploited during the evolution of cooperation. In this model, based on the individual utility, the strategy state will be asynchronously updated according to the Fermi-like rule. Among them, each individual will be initially endowed with an integral reputation value, and then it evolves during the evolution of strategy; while for the individual utility, it is characterized as the product of the game payoff and a power function of reputation value. Monte Carlo simulation (MCS) method is adopted here to verify the system’s evolutionary characteristics, and large quantities of simulations demonstrate that the cooperation behavior can be greatly varied and enhanced when the reputation is incorporated into the utility evaluation. Detailed strategy distribution proves that the individual with large reputation value renders the cooperators to lower their risks to be reaped by defectors, and dominates the evolution of cooperation within the whole population. In addition, the whole cooperation phase diagrams show that the coexistence region of cooperators and defectors becomes narrower and narrower as the reputation is introduced more and more. Meanwhile, it is also displayed that the reputation effect favors the evolution of cooperation, and greatly fosters the cooperators to form the compact clusters so as to reduce the risk of being invaded by defectors. To summarize, current results are conducive to making a deeper insight into the evolution of collective cooperation within many real-world biological and man-made systems.

Multi-Agent Optimal Allocation of Energy Storage Systems in Distribution Systems

Abstract: A variety of optimal methods for the allocation of a battery energy storage system (BESS) have been proposed for a distribution company (DISCO) to mitigate the transaction risk in a power market. All the distributed devices are assumed to be owned by the DISCO. However, in future power systems, more parties in a distribution system will have incentives to integrate BESS to reduce operational cost. In this paper, an enhanced BESS optimal allocation method is proposed for multiple agents in a distribution system. First, the electricity market mechanism is extended to a distribution system, and the corresponding energy transaction process is modeled for different agents, such as wind farms, solar power stations, demand aggregators, and the DISCO. The uncertainties of renewable energy and demand are addressed using stochastic methods. In the proposed transaction model, the integration of BESS can help an agent to reduce the operational cost, also defined as the payoff function. Next, game theory is introduced in this paper to investigate the interactions among the agents and to determine the BESS integration plans. The agents are built as players who are willing to minimize their payoff functions in the proposed non-cooperative game. The Nash equilibrium, which is the best strategy for the players, is proved to exist. Such equilibrium can be solved using an iterative algorithm. The proposed BESS allocation method for the multi-agent system is verified for two cases, and the payoff reductions are quantified based on the proposed distribution energy transaction mechanism.

Reinforcement Learning for Constrained Energy Trading Games With Incomplete Information

Abstract: This paper considers the problem of designing adaptive learning algorithms to seek the Nash equilibrium (NE) of the constrained energy trading game among individually strategic players with incomplete information. In this game, each player uses the learning automaton scheme to generate the action probability distribution based on his/her private information for maximizing his own averaged utility. It is shown that if one of admissible mixed-strategies converges to the NE with probability one, then the averaged utility and trading quantity almost surely converge to their expected ones, respectively. For the given discontinuous pricing function, the utility function has already been proved to be upper semicontinuous and payoff secure which guarantee the existence of the mixed-strategy NE. By the strict diagonal concavity of the regularized Lagrange function, the uniqueness of NE is also guaranteed. Finally, an adaptive learning algorithm is provided to generate the strategy probability distribution for seeking the mixed-strategy NE.

Learning with Bandit Feedback in Potential Games

Abstract: This paper examines the equilibrium convergence properties of no-regret learning with exponential weights in potential games. To establish convergence with minimal information requirements on the players' side, we focus on two frameworks: the semi-bandit case (where players have access to a noisy estimate of their payoff vectors, including strategies they did not play), and the bandit case (where players are only able to observe their in-game, realized payoffs). In the semi-bandit case, we show that the induced sequence of play converges almost surely to a Nash equilibrium at a quasi-exponential rate. In the bandit case, the same result holds for approximate Nash equilibria if we introduce a constant exploration factor that guarantees that action choice probabilities never become arbitrarily small. In particular, if the algorithm is run with a suitably decreasing exploration factor, the sequence of play converges to a bona fide Nash equilibrium with probability 1.

Cascading Failure Attacks in the Power System: A Stochastic Game Perspective

Abstract: Electric power systems are critical infrastructure and are vulnerable to contingencies including natural disasters, system errors, malicious attacks, etc. These contingencies can affect the world’s economy and cause great inconvenience to our daily lives. Therefore, security of power systems has received enormous attention for decades. Recently, the development of the Internet of Things (IoT) enables power systems to support various network functions throughout the generation, transmission, distribution, and consumption of energy with IoT devices (such as sensors, smart meters, etc.). On the other hand, it also incurs many more security threats. Cascading failures, one of the most serious problems in power systems, can result in catastrophic impacts such as massive blackouts. More importantly, it can be taken advantage by malicious attackers to launch physical or cyber attacks on the power system. In this paper, we propose and investigate cascading failure attacks (CFAs) from a stochastic game perspective. In particular, we formulate a zero-sum stochastic attack/defense game for CFAs while considering the attack/defense costs, budget constraints, diverse load shedding costs, and dynamic states in the system. Then, we develop a Q-CFA learning algorithm that works efficiently in power systems without any a priori information. We also formally prove that the convergence of the proposed algorithm achieves a Nash equilibrium. Simulation results validate the efficacy and efficiency of the proposed scheme by comparisons with other state-of-the-art approaches.

Offloading in HetNet: A Coordination of Interference Mitigation, User Association, and Resource Allocation

Abstract: The use of heterogeneous small cell-based networks to offload the traffic of existing cellular systems has recently attracted significant attention. One main challenge is solving the joint problems of interference mitigation, user association, and resource allocation. These problems are formulated as an optimization which is then analyzed using two different approaches: Markov approximation and log-linear learning. However, finding the optimal solutions of both approaches requires complete information of the whole network which is not scalable with the network size. Thus, an approach based on a Markov approximation with a novel Markov chain design and transition probabilities is proposed. This approach enables the Markov chain to converge to the bounded near optimal distribution without complete information. In the game-theoretic approach, the payoff-based log-linear learning is used, and it converges in probability to a mixed-strategy $\epsilon$ -Nash equilibrium. Based on the principles of these two approaches, a highly randomized self-organizing algorithm is proposed to reduce the gap between optimal and converged distributions. Simulation results show that all of the proposed algorithms effectively offload more than 90 percent of the traffic from the macrocell base station to small cell base stations. Moreover, the results also show that the algorithms converge quickly irrespective of the number of possible configurations.

On engineering game theory with its application in power systems

Abstract: Due to its capability of solving decision-making problems involving multiple entities and objectives, as well as complex action sequences, game theory has been a basic mathematical tool of economists, politicians, and sociologists for decades. It helps them understand how strategic interactions impact rational decisions of individual players in competitive and uncertain environment, if each player aims to get the best payoff. This situation is ubiquitous in engineering practices. This paper streamlines the foundations of engineering game theory, which uses concepts, theories and methodologies to guide the resolution of engineering design, operation, and control problems in a more canonical and systematic way. An overview of its application in smart grid technologies and power systems related topics is presented, and intriguing research directions are also envisioned.

An Evolutionary Game for User Access Mode Selection in Fog Radio Access Networks

Abstract: The fog radio access network (F-RAN) is a promising paradigm to provide high spectral efficiency and energy efficiency. Characterizing users to select an appropriate communication mode in F-RANs is critical for performance optimization. With evolutionary game theory, a dynamic mode selection is proposed for F-RANs, in which the competition among the groups of potential users’ space is formulated as a dynamic evolutionary game, and the game is solved by an evolutionary equilibrium. Stochastic geometry tool is used to derive the proposals’ payoff expressions for both fog access point and device-to-device users by considering node location, cache sizes, as well as the delay cost. The analytical results for the proposed game model and the corresponding solution are evaluated, which show that the evolutionary game-based access mode selection algorithm has a better payoff than the max rate-based algorithm.

Bayesian equilibria for uncertain bimatrix game with asymmetric information

Abstract: In an uncertain bimatrix game, there are two solution concepts of $$(\alpha ,\beta )$$(ź,β)-optimistic equilibrium strategy and $$(u,v)$$(u,v)-maximum chance equilibrium strategy. This paper goes further by assuming that the confidence levels $$\alpha , \beta $$ź,β and payoff levels $$u, v$$u,v are private information. Then, the so-called uncertain bimatrix game with asymmetric information is investigated. Two solution concepts of Bayesian optimistic equilibrium strategy and Bayesian maximum chance equilibrium strategy as well as their existence theorems are presented. Moreover, sufficient and necessary conditions are given for finding the Bayesian equilibrium strategies. Finally, a two-firm advertising problem is analyzed for illustrating our modelling idea.

Two-stage non-cooperative games with risk-averse players

Abstract: This paper formally introduces and studies a non-cooperative multi-agent game under uncertainty. The well-known Nash equilibrium is employed as the solution concept of the game. While there are several formulations of a stochastic Nash equilibrium problem, we focus mainly on a two-stage setting of the game wherein each agent is risk-averse and solves a rival-parameterized stochastic program with quadratic recourse. In such a game, each agent takes deterministic actions in the first stage and recourse decisions in the second stage after the uncertainty is realized. Each agent's overall objective consists of a deterministic first-stage component plus a second-stage mean-risk component defined by a coherent risk measure describing the agent's risk aversion. We direct our analysis towards a broad class of quantile-based risk measures and linear-quadratic recourse functions. For this class of non-cooperative games under uncertainty, the agents' objective functions can be shown to be convex in their own decision variables, provided that the deterministic component of these functions have the same convexity property. Nevertheless, due to the non-differentiability of the recourse functions, the agents' objective functions are at best directionally differentiable. Such non-differentiability creates multiple challenges for the analysis and solution of the game, two principal ones being: (1) a stochastic multi-valued variational inequality is needed to characterize a Nash equilibrium, provided that the players' optimization problems are convex; (2) one needs to be careful in the design of algorithms that require differentiability of the objectives. Moreover, the resulting (multi-valued) variational formulation cannot be expected to be of the monotone type in general. The main contributions of this paper are as follows: (a) Prior to addressing the main problem of the paper, we summarize several approaches that have existed in the literature to deal with uncertainty in a non-cooperative game. (b) We introduce a unified formulation of the two-stage SNEP with risk-averse players and convex quadratic recourse functions and highlight the technical challenges in dealing with this game. (c) To handle the lack of smoothness, we propose smoothing schemes and regularization that lead to differentiable approximations. (d) To deal with non-monotonicity, we impose a generalized diagonal dominance condition on the players' smoothed objective functions that facilitates the application and ensures the convergence of an iterative best-response scheme. (e) To handle the expectation operator, we rely on known methods in stochastic programming that include sampling and approximation. (f) We provide convergence results for various versions of the best-response scheme, particularly for the case of private recourse functions. Overall, this paper lays the foundation for future research into the class of SNEPs that provides a constructive paradigm for modeling and solving competitive decision making problems with risk-averse players facing uncertainty; this paradigm is very much at an infancy stage of research and requires extensive treatment in order to meet its broad applications in many engineering and economics domains.

Modeling Information Diffusion over Social Networks for Temporal Dynamic Prediction

Abstract: Modeling the process of information diffusion is a challenging problem. Although numerous attempts have been made in order to solve this problem, very few studies are actually able to simulate and predict temporal dynamics of the diffusion process. In this paper, we propose a novel information diffusion model, namely GT model, which treats the nodes of a network as intelligent and rational agents and then calculates their corresponding payoffs, given different choices to make strategic decisions. By introducing time-related payoffs based on the diffusion data, the proposed GT model can be used to predict whether or not the user's behaviors will occur in a specific time interval. The user’s payoff can be divided into two parts: social payoff from the user’s social contacts and preference payoff from the user’s idiosyncratic preference. We here exploit the global influence of the user and the social influence between any two users to accurately calculate the social payoff. In addition, we develop a new method of presenting social influence that can fully capture the temporal dynamics of social influence. Experimental results from two different datasets, Sina Weibo and Flickr demonstrate the rationality and effectiveness of the proposed prediction method with different evaluation metrics.

Environmental feedback drives cooperation in spatial social dilemmas

Abstract: Exploiting others is beneficial individually but it could also be detrimental globally. The reverse is also true: a higher cooperation level may change the environment in a way that is beneficial for all competitors. To explore the possible consequence of this feedback we consider a coevolutionary model where the local cooperation level determines the payoff values of the applied prisoner's dilemma game. We observe that the coevolutionary rule provides a significantly higher cooperation level comparing to the traditional setup independently of the topology of the applied interaction graph. Interestingly, this cooperation supporting mechanism offers lonely defectors a high surviving chance for a long period hence the relaxation to the final cooperating state happens logarithmically slow. As a consequence, the extension of the traditional evolutionary game by considering interactions with the environment provides a good opportunity for cooperators, but their reward may arrive with some delay.

Matching while Learning

Abstract: We consider the problem faced by a service platform that needs to match supply with demand but also to learn attributes of new arrivals in order to match them better in the future. We introduce a benchmark model with heterogeneous workers and jobs that arrive over time. Job types are known to the platform, but worker types are unknown and must be learned by observing match outcomes. Workers depart after performing a certain number of jobs. The payoff from a match depends on the pair of types and the goal is to maximize the steady-state rate of accumulation of payoff. Our main contribution is a complete characterization of the structure of the optimal policy in the limit that each worker performs many jobs. The platform faces a trade-off for each worker between myopically maximizing payoffs (exploitation) and learning the type of the worker (exploration). This creates a multitude of multi-armed bandit problems, one for each worker, coupled together by the constraint on the availability of jobs of different types (capacity constraints). We find that the platform should estimate a shadow price for each job type, and use the payoffs adjusted by these prices, first, to determine its learning goals and then, for each worker, (i) to balance learning with payoffs during the exploration phase, and (ii) to myopically match after it has achieved its learning goals during the exploitation phase.

A Framework for Community Detection in Large Networks Using Game-Theoretic Modeling

Abstract: Community detection is a fundamental component of large network analysis. In both academia and industry, progressive research has been made on problems related to community network analysis. Community detection is gaining significant attention and importance in the area of network science. Regular and synthetic complex networks have motivated intense interest in studying the fundamental unifying principles of various complex networks. This paper presents a new game-theoretic approach towards community detection in large-scale complex networks based on modified modularity; this method was developed based on modified adjacency, modified Laplacian matrices and neighborhood similarity. This approach was used to partition a given network into dense communities. It is based on determining a Nash stable partition, which is a pure strategy Nash equilibrium of an appropriately defined strategic game in which the nodes of the network were the players and the strategy of a node was to decide to which community it ought to belong. Players chose to belong to a community according to a maximized fitness/payoff. Quality of the community networks was assessed using modified modularity along with a new fitness function. Community partitioning was performed using Normalized Mutual Information and a ‘modularity measure’, which involved comparing the new game-theoretic community detection algorithm (NGTCDA) with well-studied and well-known algorithms, such as Fast Newman, Fast Modularity Detection, and Louvain Community. The quality of a network partition in communities was evaluated by looking at the contribution of each node and its neighbors against the strength of its community.

Towards a collaborative connected, automated driving environment: A game theory based decision framework for unprotected left turn maneuvers

Abstract: Connectivity and automation provide the opportunity to enhance safety and mitigate congestion in transportation systems. In fact, these technologies can enhance the efficiency of drivers/vehicles' decision-making by managing and coordinating the interactions among human-driven and connected, automated vehicles. Such management and coordination can lead to developing a collaborative connected, automated driving environment. Game theory, as a methodology to model the outcome of the interactions among multiple players, is a perfect tool to characterize the interaction between these vehicles. One of the most challenging maneuvers to model is drivers/vehicles' tactical decisions at intersections. Focusing on unprotected left turn maneuvers, this study aims at developing a game theory based framework to characterize driver behavior in unprotected left turn maneuvers in a connected, automated driving environment. A two-person non-zero-sum non-cooperative game under complete information is selected to model the underlying decision-making. NGSIM data is used to calibrate the payoff functions based on Maximum Likelihood Estimation. Validation results indicate that this framework can effectively capture vehicle interactions when performing conflicting turning movements while achieving a relatively high accuracy in predicting vehicles' real choice.

On the Existence of Nash Equilibrium in Bayesian Games

Abstract: We furnish conditions on the primitives of a Bayesian game that guarantee the existence of a Bayes-Nash equilibrium. By allowing for payoff discontinuities in actions, we cover various applications that cannot be handled by extant results.

A Variable Impacts Measurement in Random Forest for Mobile Cloud Computing

Abstract: Recently, the importance of mobile cloud computing has increased. Mobile devices can collect personal data from various sensors within a shorter period of time and sensor-based data consists of valuable information from users. Advanced computation power and data analysis technology based on cloud computing provide an opportunity to classify massive sensor data into given labels. Random forest algorithm is known as black box model which is hardly able to interpret the hidden process inside. In this paper, we propose a method that analyzes the variable impact in random forest algorithm to clarify which variable affects classification accuracy the most. We apply Shapley Value with random forest to analyze the variable impact. Under the assumption that every variable cooperates as players in the cooperative game situation, Shapley Value fairly distributes the payoff of variables. Our proposed method calculates the relative contributions of the variables within its classification process. In this paper, we analyze the influence of variables and list the priority of variables that affect classification accuracy result. Our proposed method proves its suitability for data interpretation in black box model like a random forest so that the algorithm is applicable in mobile cloud computing environment.

On an approach to constructing a characteristic function in cooperative differential games

Abstract: We propose a novel approach to constructing characteristic functions in cooperative differential games. A characteristic function of a coalition S is computed in two stages: first, optimal control strategies maximizing the total payoff of the players are found, and next, these strategies are used by the players from the coalition S, while the other players, those from N S, use strategies minimizing the total payoff of the players from S. The characteristic function obtained in this way is superadditive. In addition, it possesses a number of other useful properties. As an example, we compute values of a characteristic function for a specific differential game of pollution control.

Economics of Quality Sponsored Data in Non-Neutral Networks

Abstract: The growing demand for data has driven the service providers (SPs) to provide differential treatment of traffic to generate additional revenue streams from content providers (CPs). While SPs currently only provide best-effort services to their CPs, it is plausible to envision a model in near future, where CPs are willing to sponsor quality of service for their content in exchange of sharing a portion of their profit with SPs. This quality sponsoring becomes invaluable especially when the available resources are scarce, such as in wireless networks, and can be accommodated in a non-neutral network. In this paper, we consider the problem of quality-sponsored data (QSD) in a non-neutral network. In our model, SPs allow CPs to sponsor a portion of their resources, and price it appropriately to maximize their payoff. The payoff of the SP depends on the monetary revenue and the satisfaction of end-users both for the non-sponsored and sponsored content, while CPs generate revenue through advertisement. Note that in this setting, end-users still pay for the data they use. We analyze the market dynamics and equilibria in two different frameworks, i.e., sequential and bargaining game frameworks, and provide strategies for: 1) SPs—to determine if and how to price resources and 2) CPs—to determine if and what quality to sponsor. The frameworks characterize different sets of equilibrium strategies and market outcomes depending on the parameters of the market.

Promotion of cooperation induced by heterogeneity of both investment and payoff allocation in spatial public goods game

Abstract: Heterogeneity has attracted mounting attention across multiple disciplines and is confirmed to be a greater promoter of cooperation. It is often the case that the heterogeneity always exists in investment and payoff allocation concurrently instead of separately. In addition, the factors that affect heterogeneous investment and payoff allocation are various. Inspired by this, this paper extends the previous models by incorporating heterogeneous investment and payoff allocation into the typical PGG model to further investigate the incentive mechanisms of cooperative behavior. In order to better understand the model, three different feedback mechanisms, namely the wealth-preference mechanism, the social-self-preference mechanism, and the mixed-preference mechanism, are addressed. The former two mechanisms correspond to the case of single factor and the latter corresponds to the case of double factors. The numerical simulations indicate that feedback mechanism by bridging the connections between the investment and the payoff allocation can reduce the free-rider problem. Furthermore, it is found that the cooperative frequency and average payoff perform better in the case of the mixed-preference mechanism where players will not only take previous payoff feedback as well as current investment but also their social status into their game decision-making process. In addition, full cooperation and profitability over all players can be promoted by means of increasing r or reducing α . At last, compared with another two classic networks, the extent of cooperation is promoted under the structures of the BA scale free networks.