scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Performance Evaluation of Peer-to-Peer Gaming Overlays

13 Sep 2010-pp 1-2
TL;DR: This demo consists of a 3D first person shooter game that is designed to run in a simulated network environment as well as on a real network as a performance evaluation testbed for peer-to-peer gaming overlays.
Abstract: In this demo we present a performance evaluation testbed for peer-to-peer gaming overlays. It consists of a 3D first person shooter game that is designed to run in a simulated network environment as well as on a real network. Simulation with autonomous players (bots) guarantees scalability, a controlled workload, and reproducible results; a prototype deployment on a real network can then validate the simulation results. The information dissemination overlay pSense is implemented as a first subject for evaluation.

Summary (1 min read)

Introduction

  • In the last years gaming has become an attractive field for peer-to-peer research.
  • In particular the strict timing requirements of first person shooter games (FPS) demand the development of suitable network overlays.
  • Specific peer-to-peer overlays, such as pSense [1], VON [2], or Donnybrook [3], have been developed to address these needs.
  • Only a few overlays were analyzed with real games.
  • On the other hand, a standalone application cannot compete against a simulator in terms of parameter flexibility, reproducibility, and precise measurement.

II. PSENSE

  • PSense [1] is an information dissemination overlay addressing the specific need for exchanging updates of game states based on virtual world proximity.
  • To prevent network partitions in sparse density areas, each node keeps a list of eight sensor nodes outside its vision range.
  • In simulation, the hardware is abstracted, and the behavior is mapped to discrete-event mode.
  • A frame is then rendered each 100 simulated milliseconds.
  • Planet π4 and the pSense implementation are written in C++, while the simulator and network subsystem which the authors are currently using come from the BubbleStorm [5] project1 and are written in Standard ML.

IV. WORKLOAD: BOTS

  • The measurement of overlay properties in the game requires appropriate workloads.
  • The obvious way to generate workload is using human players.
  • But setting up a testbed environment for human players takes a lot of effort, especially in larger scales (100 or more players).
  • The traces measured on one system cannot represent a realistic workload on a different system.
  • For the currently very simple gameplay, the bot implementations have simple aim-and-shoot mechanisms.

A. Real-World LAN Game

  • The PCs are available for human players to play against each other and against the bots which can be added to the session.
  • Status information like network traffic and overlay neighbor lists is made available through an in-game head-up display.

B. Simulated Internet Game

  • The discrete-event simulation runs several game instances (each controlled by a bot) within one operating system process.
  • The game can be watched from the perspective of one of the instances.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Performance Evaluation of Peer-to-Peer Gaming Overlays
Max Lehn* Tonio Triebel Christof Leng* Alejandro Buchmann* Wolfgang Effelsberg
Databases and Distributed Systems Praktische Informatik IV
Technische Universit
¨
at Darmstadt Universit
¨
at Mannheim
Germany Germany
{max
lehn, cleng, buchmann}@dvs.tu-darmstadt.de {triebel, effelsberg}@pi4.informatik.uni-mannheim.de
Abstract—In this demo we present a performance evaluation
testbed for peer-to-peer gaming overlays. It consists of a 3D first
person shooter game that is designed to run in a simulated net-
work environment as well as on a real network. Simulation with
autonomous players (bots) guarantees scalability, a controlled
workload, and reproducible results; a prototype deployment on
a real network can then validate the simulation results. The
information dissemination overlay pSense is implemented as a
first subject for evaluation.
I. INTRODUCTION
In the last years gaming has become an attractive field
for peer-to-peer research. In particular the strict timing re-
quirements of first person shooter games (FPS) demand the
development of suitable network overlays. The main chal-
lenge consists of the combination of high dynamic movement
of players and low latencies that are required for update
messages. Specific peer-to-peer overlays, such as pSense [1],
VON [2], or Donnybrook [3], have been developed to address
these needs. For the evaluation of their systems most research
groups use simple custom game simulations. Only a few
overlays were analyzed with real games.
The simulation of a simplified game is hard to validate. The
simulator might abstract away important details that affect the
outcome of the simulation significantly. On the other hand,
a standalone application cannot compete against a simulator
in terms of parameter flexibility, reproducibility, and precise
measurement.
We propose a benchmark platform for the evaluation of
peer-to-peer gaming overlays that brings the two approaches
together. The platform is based on the 3D multiplayer space-
ship shooter Planet π4 [4]. A key feature of our system is
the ability to run the game both as a standalone application
using the real network and in a discrete-event simulation on
a virtual network. Hence, it is possible to evaluate an overlay
in a discrete-event simulation with a workload generated by
autonomous players (bots) and to validate the results with
the prototype deployment using a real network. In addition,
our system provides the possibility to replace the peer-to-peer
overlay implementation in order to compare different overlays.
For the demonstration an implementation of pSense is used to
show the two main functionalities of our system.
Fig. 1. A screenshot of Planet π 4
II. PSENSE
pSense [1] is an information dissemination overlay ad-
dressing the specific need for exchanging updates of game
states based on virtual world proximity. Each player has a
vision range which determines the interest of activities of
other players (e.g. movements and shots). Thus, in the pSense
topology a node knows all neighbors in its vision range. To
prevent network partitions in sparse density areas, each node
keeps a list of eight sensor nodes outside its vision range.
These nodes also introduce new nodes approaching the vision
range.
In its original publication, like many other peer-to-peer
systems, pSense was only evaluated with simulations lacking
most networking detail. This work is the first to show that
pSense works in a real game application.
III. I
MPLEMENTATION
In standalone application mode, like in conventional game
implementations, the game’s main loop repeatedly renders
a frame, processes user input, runs the game mechanics,
and performs network I/O. The frame rate is only limited
by the hardware capabilities. In simulation, the hardware is
abstracted, and the behavior is mapped to discrete-event mode.
The simulated instances usually do not render frames since
978-1-4244-7141-6/10/$26.00 ©2010 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE P2P 2010 proceedings.

Fig. 2. High level system architecture
there is no user who needs to see them. For watching the
simulation, frame rendering can still be enabled. A frame
is then rendered each 100 simulated milliseconds. Thus, the
game has a virtual frame rate of 10 frames per second.
Planet π4 and the pSense implementation are written in
C++, while the simulator and network subsystem which we
are currently using come from the BubbleStorm [5] project
1
and are written in Standard ML. To switch between simulation
and real application all that is necessary is to replace the
small component controlling the main loop in Planet π4’s
core. All other components remain unchanged, provided that
they fulfill certain requirements. Most importantly, I/O and
wait operations are restricted to pure asynchronous mode,
since blocking calls cannot be executed in a discrete-event
simulation.
IV. W
ORKLOAD:BOTS
The measurement of overlay properties in the game requires
appropriate workloads. The obvious way to generate workload
is using human players. But setting up a testbed environment
for human players takes a lot of effort, especially in larger
scales (100 or more players). Furthermore, the generated
workload depends on the behavior of the players and thus is
not reproducible. The alternative of using traces from human
player matches is not convincing, since the highly interactive
gameplay is influenced by system properties (e.g., latency
and consistency). The traces measured on one system cannot
represent a realistic workload on a different system.
We chose to use bots to generate reproducible and easily
scalable workloads. For the currently very simple gameplay,
the bot implementations have simple aim-and-shoot mecha-
nisms. More complex gameplay modes of course need more
complex bot implementations. Those will then have to be
tuned to an activity level similar to human players.
V. D
EMONSTRATION
The demonstration consists of two parts: a physical local
area network and a simulated Internet game.
A. Real-World LAN Game
A small set of PC runs the application in standalone mode,
joining a common game session. The PCs are available for
human players to play against each other and against the bots
which can be added to the session. Status information like
network traffic and overlay neighbor lists is made available
through an in-game head-up display.
1
http://www.bubblestorm.net
B. Simulated Internet Game
The discrete-event simulation runs several game instances
(each controlled by a bot) within one operating system process.
The number of instances in the simulation is limited by the
available amount of RAM; 4GB currently suffice for up to 128
instances. The game can be watched from the perspective of
one of the instances.
VI. C
ONCLUSION AND FUTURE WORK
Our system provides a reproducible setup in a discrete-event
simulation with a workload generated by bots and prototype
deployment for real networks with a workload generated by
human players and/or bots, both using the same peer-to-
peer system implementation. Planet π 4 provides the base for
a realistic benchmark for peer-to-peer systems focusing on
realtime capabilities. And as a game Planet π4 is still simple
enough to concentrate on the core aspects.
For the further evaluation of peer-to-peer overlays we plan
to extend Planet π4’s gameplay, replacing the simple death-
match mode with a team mode in which the teams have to
defend certain strategic points of interest. The richer gameplay
is supposed to increase the realism of the game workload, the
fun factor for human players, and the scalability (number of
players) of the gameplay.
We like to extend our measurement infrastructure to the
standalone mode. While system-wide aggregation of statistics
is trivial in the simulation environment, the distributed nature
of the standalone mode makes the aggregation of highly
timing-sensitive statistics challenging.
Finally, we want to add support for the widely used ns-3
2
network simulator as a third option in addition to our custom
simulator and the real network stack.
This demo proposal and the related implementations were
developed in the QuaP2P research group
3
funded by the
Deutsche Forschungsgemeinschaft (DFG).
R
EFERENCES
[1] A. Schmieg, M. Stieler, S. Jeckel, P. Kabus, B. Kemme, and A. Buch-
mann, “pSense-Maintaining a Dynamic Localized Peer-to-Peer Structure
for Position Based Multicast in Games, in P2P’08. Eighth International
Conference on Peer-to-Peer Computing, 2008, pp. 247–256.
[2] S.-Y. Hu and G.-M. Liao, “Scalable Peer-to-Peer Networked Virtual
Environment, in NetGames ’04: Proceedings of 3rd ACM SIGCOMM
workshop on Network and system support for games, 2004, pp. 129–133.
[3] A. Bharambe, J. R. Douceur, J. R. Lorch, T. Moscibroda, J. Pang,
S. Seshan, and X. Zhuang, “Donnybrook: Enabling Large-Scale, High-
Speed, Peer-to-Peer Games, ACM SIGCOMM Computer Communication
Review, vol. 38, no. 4, pp. 389–400, 2008.
[4] T. Triebel, B. Guthier, R. S
¨
uselbeck, G. Schiele, and W. Effelsberg, “Peer-
to-Peer Infrastructures for Games, in NOSSDAV ’08: Proceedings of the
18th International Workshop on Network and Operating Systems Support
for Digital Audio and Video, 2008, pp. 123–124.
[5] W. Terpstra, J. Kangasharju, C. Leng, and A. Buchmann, “BubbleStorm:
Resilient, Probabilistic, and Exhaustive Peer-to-Peer Search, in SIG-
COMM ’07: Proceedings of the 2007 conference on Applications, tech-
nologies, architectures, and protocols for computer communications,
2007, pp. 49–60.
2
http://www.nsnam.org/
3
http://www.quap2p.tu-darmstadt.de/
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE P2P 2010 proceedings.
Citations
More filters
Proceedings ArticleDOI
15 Aug 2011
TL;DR: This demo presents a testbed environment for Peer-to-Peer (P2P) game architectures based on Planet PI4, an online multiplayer game whose gameplay provides a standard workload for a set of gaming-specific network interfaces.
Abstract: In this demo we present a testbed environment for Peer-to-Peer (P2P) game architectures. It is based on Planet PI4, an online multiplayer game whose gameplay provides a standard workload for a set of gaming-specific network interfaces. Its pluggable architecture allows for the evaluation and comparison of existing and new P2P networking approaches. Planet PI4 can run on a real network for prototypical evaluation as well as in a discrete-event simulator providing a reproducible environment.

11 citations


Cites methods from "Performance Evaluation of Peer-to-P..."

  • ...We are developing the 3D real-time massively multiplayer online game Planet PI4 [10, 7], which serves as a benchmark for various P2P networking components....

    [...]

Proceedings ArticleDOI
06 Oct 2011
TL;DR: RTSenv is introduced, an environment and associated set of tools that can configure and manage the main aspects of RTS games, such as maps, computer-controlled units, and game scenarios, and gives evidence that several common assumptions made by researchers about game workloads do not hold in general for RTSGames and thus warrant a more detailed investigation.
Abstract: Today, Real-Time Strategy (RTS) games entertain tens of millions of players world-wide. This growing population expects new game designs and more scalable games every year. However, few tools and environments exist for game designers and implementers; of these, even fewer are available to researchers and game communities. In this work, we introduce RTSenv, an environment and associated set of tools for RTS games. Our environment can configure and manage the main aspects of RTS games, such as maps, computer-controlled units, and game scenarios. RTSenv leverages multi-cluster systems and reactive fault tolerance mechanisms to perform robust, multi-machine, and multi-instance game experiments. Using our reference implementation of RTSenv in DAS-4, a real multi-cluster system, and Amazon EC2, a commercial cloud, we show that our approach can be used in a variety of scenarios. Our results give evidence that several common assumptions made by researchers about game workloads do not hold in general for RTS games and thus warrant a more detailed investigation.

8 citations

Proceedings ArticleDOI
15 Aug 2011
TL;DR: A testbed for Internet applications that supports real-network prototypes and multiple simulators with unchanged application code is presented and a compact but flexible system interface is defined to ensure maximum portability between runtimes.
Abstract: Different evaluation methods for distributed systems like prototyping, simulation and emulation have different tradeoffs. We present a testbed for Internet applications that supports real-network prototypes and multiple simulators with unchanged application code. To ensure maximum portability between runtimes, a compact but flexible system interface is defined.

5 citations


Cites methods from "Performance Evaluation of Peer-to-P..."

  • ...We have already used the testbed to implement CUSP demo applications [11], BubbleStorm [10], pSense [8], Kademlia [7], and the Planet PI4 online game [6]....

    [...]

Dissertation
14 Sep 2012
TL;DR: After analyzing the design space of replica maintenance for peer-to-peer systems, a complete solution covering all identified use cases is presented, including a maintainer-based mechanism for data managed by a single node and a collective mechanism that shall be persistent beyond any particular node’s session time.
Abstract: As distributed systems are getting more and more complex, search facilities for finding services and data within the system become crucial. Users expect search engines to deal with complex query languages like keyword search, SQL, or XPath. At the same time, application developers cannot be expected to come up with distributed versions of those query languages from scratch. Rendezvous search systems are a very scalable solution to this problem. By separating the query processing from the network communication, existing libraries for query processing can be easily reused. A wide range of rendezvous search systems for different scenarios has been proposed in the past. Their scalability and resilience make them an excellent choice for search in large-scale and dynamic peer-to-peer environments. The resilience stems mainly from the high number of replicas per datum, which however makes replica maintenance difficult. Unfortunately, most rendezvous search systems lack maintenance algorithms to sustain the desired replica count under node churn. Replica maintenance is closely related to update mechanisms for mutable data. The highly distributed nature of peer-to-peer systems in general and the high replica count of rendezvous search systems in particular require carefully designed mechanisms for consistent updates with concurrent accesses. In this thesis, replica maintenance and update mechanisms for the BubbleStorm peer-to-peer overlay and related rendezvous search systems are introduced. After analyzing the design space of replica maintenance for peer-to-peer systems, a complete solution covering all identified use cases is presented. This includes a maintainer-based mechanism for data managed by a single node and a collective mechanism for data that shall be persistent beyond any particular node’s session time. The algorithms are evaluated in BubbleStorm’s sophisticated testbed, which allows prototype experiments and simulations with the same source code.

5 citations


Cites methods from "Performance Evaluation of Peer-to-P..."

  • ...The simulation and prototyping environment developed for this work is based on extensive experience gathered in a wide range of P2P research projects, ranging from message-based Gnutella simulations [87], custom-built message/packet simulations of BubbleStorm with up to one million nodes [143, 144], numerical simulations of the maintainer-based replication [81], and benchmarking concepts for networked virtual environments [71] to prototyping P2P wikis [99] and P2P gaming overlays [74, 75]....

    [...]

  • ...It has already been used for a real-time MMOG gaming application [74, 75]....

    [...]

  • ...The system has been successfully used to evaluate the sophisticated online gaming application Planet Pi4 [74] (see Figure 9....

    [...]

Journal ArticleDOI
TL;DR: Traditional consistency models provided by reliable group communication providing causal or even total order are restricted to support only small groups and the delivery semantics of robust and scalable protocols such as gossiping is not sufficient to support consistency semantics beyond eventual consistency.
Abstract: Group communication is deployed in many evolving Internet-scale cooperative applications such as multiplayer online games and virtual worlds to efficiently support interaction on information relevant to a potentially very large number of users or objects. Especially peer-to-peer based group communication protocols have evolved as a promising approach to allow intercommunication between many distributed peers. Yet, the delivery semantics of robust and scalable protocols such as gossiping is not sufficient to support consistency semantics beyond eventual consistency because no relationship on the order of events is enforced. On the other hand, traditional consistency models provided by reliable group communication providing causal or even total order are restricted to support only small groups.This article proposes the cluster consistency model which bridges the gap between traditional and current approaches in supporting both scalability and ordered event delivery. We introduce a dynamic and fault tolerant cluster management method that can coordinate concurrent access to resources in a peer-to-peer system and can be used to establish fault-tolerant configurable cluster consistency with predictable reliability, running on top of decentralised probabilistic protocols supporting scalable group communication. This is achieved by a general two-layered architecture that can be applied on top of the standard Internet communication layers and offers a modular, layered set of services to the applications that need them. Further, we present a fault-tolerant method implementing causal cluster consistency with predictable reliability, running on top of decentralised probabilistic protocols supporting group communication.This paper provides analytical and experimental evaluation of the properties regarding the fault tolerance of the approach. Furthermore, our experimental study, conducted by implementing and evaluating the two-layered architecture on top of standard Internet transport services, shows that the approach scales well, imposes an even load on the system, and provides high-probability reliability guarantees.

4 citations


Cites background from "Performance Evaluation of Peer-to-P..."

  • ...Collaborative environments such as distributed interactive simulation [1–3], virtual worlds [4] and massively scalable online multiplayer games [5,6] allow a possibly large set of concurrently joining and leaving processes to share and interact on a set of common replicated objects....

    [...]

References
More filters
Journal ArticleDOI
17 Aug 2008
TL;DR: Donnybrook is described, a system that enables epic-scale battles without dedicated server resources, even in a fast-paced game with tight latency bounds, by disseminating updates via a multicast system designed for the special requirements of games.
Abstract: Without well-provisioned dedicated servers, modern fast-paced action games limit the number of players who can interact simultaneously to 16-32. This is because interacting players must frequently exchange state updates, and high player counts would exceed the bandwidth available to participating machines. In this paper, we describe Donnybrook, a system that enables epic-scale battles without dedicated server resources, even in a fast-paced game with tight latency bounds. It achieves this scalability through two novel components. First, it reduces bandwidth demand by estimating what players are paying attention to, thereby enabling it to reduce the frequency of sending less important state updates. Second, it overcomes resource and interest heterogeneity by disseminating updates via a multicast system designed for the special requirements of games: that they have multiple sources, are latency-sensitive, and have frequent group membership changes. We present user study results using a prototype implementation based on Quake III that show our approach provides a desirable user experience. We also present simulation results that demonstrate Donnybrook's efficacy in enabling battles of up to 900 players.

191 citations


Additional excerpts

  • ...Specific peer-to-peer overlays, such as pSense [1], VON [2], or Donnybrook [3], have been developed to address these needs....

    [...]

Proceedings ArticleDOI
30 Aug 2004
TL;DR: A fully-distributed peer-to-peer architecture to solve the scalability problem of Networked Virtual Environment in a simple and efficient manner and is based on the mathematical construct Voronoi diagram.
Abstract: We propose a fully-distributed peer-to-peer architecture to solve the scalability problem of Networked Virtual Environment in a simple and efficient manner. Our method exploits locality of user interest inherent to such systems and is based on the mathematical construct Voronoi diagram. Scalable, responsive, fault-tolerant NVE can thus be constructed and deployed in an affordable way.

172 citations


Additional excerpts

  • ...Specific peer-to-peer overlays, such as pSense [1], VON [2], or Donnybrook [3], have been developed to address these needs....

    [...]

Proceedings ArticleDOI
27 Aug 2007
TL;DR: BubbleStorm as mentioned in this paper is a probabilistic search system built on random multigraphs, which can handle up to 90% simultaneous peer departure and 50% simultaneous crash.
Abstract: Peer-to-peer systems promise inexpensive scalability, adaptability, and robustness. Thus, they are an attractive platform for file sharing, distributed wikis, and search engines. These applications often store weakly structured data, requiring sophisticated search algorithms. To simplify the search problem, most scalable algorithms introduce structure to the network. However, churn or violent disruption may break this structure, compromising search guarantees.This paper proposes a simple probabilistic search system, BubbleStorm, built on random multigraphs. Our primary contribution is a flexible and reliable strategy for performing exhaustive search. BubbleStorm also exploits the heterogeneous bandwidth of peers. However, we sacrifice some of this bandwidth for high parallelism and low latency. The provided search guarantees are tunable, with success probability adjustable well into the realm of reliable systems.For validation, we simulate a network with one million low-end peers and show BubbleStorm handles up to 90% simultaneous peer departure and 50% simultaneous crash.

160 citations

Proceedings ArticleDOI
08 Sep 2008
TL;DR: An algorithm for creating and maintaining a dynamic localized peer-to-peer overlay network with its main application to massively multiplayer games, which allows for dynamic game worlds of practically unlimited size, only limited in scale by the number of players within the vision range.
Abstract: This paper presents an algorithm for creating and maintaining a dynamic localized peer-to-peer overlay network with its main application to massively multiplayer games. In these games, players reside in a large game world with many thousands of players but each player has typically a limited vision range. In our solution, players join the network as peers and mainly connect to neighbor peers that are close to them in the virtual game world. As players move in the game they change their neighbors dynamically with very little overhead. Peers can multicast messages that are received by peers in their locality very fast (often faster than in client-server solutions) while players that are further away receive them later or not at all. Not receiving messages from remote players is important in order to not cause the load on each peer to grow with the number of players in the game. Our performance analysis confirms that our solution allows for dynamic game worlds of practically unlimited size, only limited in scale by the number of players within the vision range.

46 citations

Proceedings ArticleDOI
28 May 2008
TL;DR: Planet Π4, a Massively Multiplayer Online Game (MMOG) developed to evaluate and compare peer-to-peer-based MMOG systems with scalability in mind, is presented.
Abstract: In this demo proposal we present Planet Π4, a Massively Multiplayer Online Game (MMOG) developed to evaluate and compare peer-to-peer-based MMOG systems with scalability in mind. The game requires low network latency and creates frequent game state updates. It has a modular architecture that can be adapted and extended with new functionality. Using this modular design we have developed different peer-to-peer infrastructures. Workshop participants will be able to play the game and compare the versions with each other.

8 citations


"Performance Evaluation of Peer-to-P..." refers methods in this paper

  • ...The platform is based on the 3D multiplayer spaceship shooter Planet π4 [4]....

    [...]

Frequently Asked Questions (2)
Q1. What have the authors contributed in "Performance evaluation of peer-to-peer gaming overlays" ?

In this demo the authors present a performance evaluation testbed for peer-to-peer gaming overlays. 

For the further evaluation of peer-to-peer overlays the authors plan to extend Planet π4 ’ s gameplay, replacing the simple deathmatch mode with a team mode in which the teams have to defend certain strategic points of interest. The authors like to extend their measurement infrastructure to the standalone mode.