A Case Study of Networks Simulation Tools for Wireless Networks
25 May 2009-pp 661-666
TL;DR: To determine the state of wireless networks simulation studies, a survey of several primer conferences papers from years 2000-2008 found some important pitfalls and issues and presents the research results with the hope to aid developers in the selection of an appropriate simulation tool.
Abstract: Networking community is largely depend on simulation to evaluate the behavior and performance of protocols for wireless networks, simulations are a good compromise between cost and complexity, on the one hand, and accuracy of the results, on the other hand. Since there are many simulators for wireless networks, it is often difficult to decide which simulator to choose. To help shed light on this issue, we discuss six “Widely Used” network simulation tools along with their associated strengths and weakness. To determine the state of wireless networks simulation studies, we surveyed the several primer conferences papers from years 2000-2008. From our survey, we found some important pitfalls and issues. We then summarize simulation study pitfalls found in our survey. Finally, we present our research results with the hope to aid developers in the selection of an appropriate simulation tool and to show them how to avoid common pitfalls to improve the reliability and repeatability of simulation based studies.
Citations
More filters
TL;DR: This systematic literature review aims to analyze, basing its investigation on available literature, the adoption of a popular network simulator, namely ns-3, and its use in the scientific community, and how extensible it is in practice according to the experience of authors.
Abstract: Complexity of current computer networks, including e.g., local networks, large structured networks, wireless sensor networks, datacenter backbones, requires a thorough study to perform analysis and support design. Simulation is a tool of paramount importance to encompass all the different aspects that contribute to design quality and network performance (including as well energy issues, security management overheads, dependability), due to the fact that such complexity produces several interactions at all network layers that is not easily modellable with analytic approaches. In this systematic literature review we aim to analyze, basing our investigation on available literature, the adoption of a popular network simulator, namely ns-3, and its use in the scientific community. More in detail, we are interested in understanding what are the impacted application domains in which authors prefer ns-3 to other similar tools and how extensible it is in practice according to the experience of authors. The results of our analysis, which has been conducted by especially focusing on 128 papers published between 2009 to 2019, reveals that 10% of the evaluated papers were discarded because they represented informal literature; most of the studies presented comparisons among different network simulators, beyond ns-3 and conceptual studies related to performance assessment and validation and routing protocols. Only about 30% of considered studies present extensions of ns-3 in terms of new modules and only about 10% present effective case studies demonstrating the effectiveness of employing network simulator in real application, except conceptual and modeling studies.
59 citations
TL;DR: 130 simulation and emulation environments and frameworks, which were originally designed and adapted for USN, are surveyed and 22 that have been widely used, regularly updated, and well supported by their developers are compared based on multifarious criteria.
41 citations
22 Mar 2011
TL;DR: This paper identifies the key limitations of the reviewed simulation tools and inspect their suitability for large-scale WSNs, and investigates simulation tools based on a new set of preferred criterion, i.e. popularity, accessibility (open-source), complexity, accuracy, scalability, extensibility and availability of various models and protocols.
Abstract: Wireless Sensor Networks (WSNs) consist of a large number of wireless sensor nodes networked together. It is a complex set of applications, link technologies, communication protocols, traffic flows and routing algorithms. Simulation is a predominant technique used to study and analyse the performance and potency of a senor network design. Since there are a huge variety of simulation tools available for WSNs, which vary in their characteristics and capabilities, it is often very difficult to decide which simulation tool to choose and which one is more appropriate for large-scale WSNs. To address this issue, in this paper, we review some of the most widely-used and state-of-the-art simulation tools for WSNs. This distinguishing feature of this paper is that we identify the key limitations of the reviewed simulation tools and inspect their suitability for large-scale WSNs. We review and investigate simulation tools based on a new set of preferred criterion, i.e. popularity, accessibility (open-source), complexity, accuracy, scalability, extensibility and availability of various models and protocols.
31 citations
Cites background or methods from "A Case Study of Networks Simulation..."
...Different researchers categorized and reviewed simulation tools for WSNs in different ways [3, 5-9], however, the distinguishing feature of our work is that we define a new set of criterion for our investigation, which rigorously keep into account the specific requirement and characteristics of…...
[...]
...Besides, a visualization tool is desirable, which can quickly track down what goes wrong during the simulation [17]....
[...]
...However, it deviates by using the object-oriented approach and use J-Sim's simulator component based architecture....
[...]
01 Dec 2016
TL;DR: This paper design and further implement the integrated link-system level simulation platform, which makes it possible to evaluate the new technologies for IEEE 802.11ax and proposes a MAC protocol combining OFDMA, MU-MIMO, non-continuous channel bonding and link adaptation and further evaluates its performance.
Abstract: As the most widely used standards for wireless local area network (WLAN), IEEE 802.11 standards are continuously amended by introducing new techniques so as to meet the increasing demands. In order to verify the performance of amended protocols, network simulation is considered as a significant method. However, as far as we know, current simulation tools are only for either media access control layer (MAC) or physical layer (PHY). The separate simulation of MAC and PHY can hardly evaluate the performance of IEEE 802.11ax in whole system level for authenticity and objectivity. Hence, the next generation WLAN (IEEE 802.11ax) requires integrated system simulation to take impacts of both MAC and PHY techniques into account. Moreover, IEEE 802.11ax introduces some new techniques, such as orthogonal frequency division multiple access (OFDMA), multi-user multiple input multiple output (MU- MIMO) and non-continuous channel bonding. In this paper, we design and further implement the integrated link-system level simulation platform, which makes it possible to evaluate the new technologies for IEEE 802.11ax. Moreover, we propose a MAC protocol combining OFDMA, MU-MIMO, non-continuous channel bonding and link adaptation and further evaluate its performance. Finally, we validate performance gains of IEEE 802.11ax through simulation, and the simulation results show that IEEE 802.11ax has obviously higher throughput, better quality of service (QoS) and higher multi- channel efficiency. To the best of our knowledge, this is the first work to design and implement simulation platform for IEEE 802.11ax with an integrated link- system level framework.
31 citations
Cites background from "A Case Study of Networks Simulation..."
...[16] show survey results that NS2 is the most popular network simulator....
[...]
16 Jun 2013
TL;DR: A WAMC testing platform to facilitate the real-time simulation of dynamic power grid, the ICT infrastructure that overlays the grid and WAMS applications and the effect of network protocols on data delay is proposed and developed.
Abstract: PMU-based Wide Area Monitoring and Control (WAMC) system is introduced to improve the monitoring of power grid across large geographic areas and control the grid using more efficient and smart applications. The performance of WAMC applications in real power system scenarios and impact of their supporting Information and Communication Technology (ICT) on the data quality can be quantified and analyzed by pseudo-real co-simulation test beds. The purpose of this study is to propose and develop a WAMC testing platform to facilitate the real-time simulation of dynamic power grid, the ICT infrastructure that overlays the grid and WAMS applications. The platform consists of OPNET, a powerful communication network emulator, connected to a real-time power system simulator through virtualized PMU device. The end point stations such as Phasor Data Concentrator or PMU-based applications are also linked to the platform through OPNET's real-simulation gateway called SITL (System-In-The-Loop). To assess the performance of the platform architecture, a case study has been performed with five PMUs which collect the data from a power model and deliver to PMU-based mode-estimation application over a typical communication network. In this study, the results explicitly intend to quantify the effect of network protocols on data delay.
29 citations
Cites background from "A Case Study of Networks Simulation..."
...The detailed comparison of available network simulators and their performance in different scenarios presented in [19], [20], and [21] suggests that OPNET is the most suitable candidate software that fulfills our requirements....
[...]
References
More filters
TL;DR: This article discusses two important necessary conditions of a credible simulation study: use of appropriate pseudo-random generators of independent uniformly distributed numbers, and appropriate analysis of simulation output data.
Abstract: In telecommunication networks, as in many other areas of science and engineering, the proliferation of computers as research tools has resulted in the adoption of computer simulation as the most commonly used paradigm of scientific investigations. This, together with a plethora of existing simulation languages and packages, has created a popular opinion that simulation is mainly an exercise in computer programming. In new computing environments, programming can be minimized, or even fully replaced, by the manipulation of icons (representing prebuilt programming objects containing basic functional blocks of simulated systems) on a computer monitor. One can say that we have witnessed another success of modern science and technology: the emergence of wonderful and powerful tools for exploring and predicting the behavior of such complex stochastic dynamic systems as telecommunication networks. But this enthusiasm is not shared by all researchers in this area. An opinion is spreading that one cannot rely on the majority of the published results on performance evaluation studies of telecommunication networks based on stochastic simulation, since they lack credibility. Indeed, the spread of this phenomenon is so wide that one can speak about a deep crisis of credibility. In this article this claim is supported by the results of a survey of over 2200 publications on telecommunication networks in proceedings of IEEE INFOCOM and such journals as IEEE Transactions on Communications, IEEE/ACM Transactions on Networking, and Performance Evaluation Journal. The discussion focuses on two important necessary conditions of a credible simulation study: use of appropriate pseudo-random generators of independent uniformly distributed numbers, and appropriate analysis of simulation output data. Having considered their perils and pitfalls, we formulate guidelines that, if observed, could help to ensure a basic level of credibility of simulation studies of telecommunication networks.
461 citations
"A Case Study of Networks Simulation..." refers background in this paper
...We follow the similar kind of work line of [12] but with different aims....
[...]
...[12] Survey In general simulation study Credibility , accuracy...
[...]
...In [12] authors presented a survey study of more then 2200 research papers in the field of network simulation studies and point out several systematic flaws in that....
[...]
...The works parented in [5, 7, 8, 9, 10, 12] are the close to our work as they include some common simulators J-Sim, OMNeT++, and ns-2, which we also consider for our study....
[...]
TL;DR: A map of the main characteristics that MANETs simulation tools should feature and the current support of these is provided, including a description for each simulator, including an explanation of what make them appealing solutions.
143 citations
"A Case Study of Networks Simulation..." refers background in this paper
...A huge list of simulators is presented in [7, 9] however, they do not give a comparative study....
[...]
...The works parented in [5, 7, 8, 9, 10, 12] are the close to our work as they include some common simulators J-Sim, OMNeT++, and ns-2, which we also consider for our study....
[...]
...[7] Description GloMoSim, ns-2, DIANEmu, GTNetS,...
[...]
10 Dec 2008
TL;DR: The goal of this paper is to aid developers in the selection of an appropriate simulation tool to test new protocols that are being developed, as well as to compare old protocols.
Abstract: Sensor networks face many problems that do not arise in other types of networks. Power constraints, limited hardware, decreased reliability, and a typically higher density and number of nodes than found in conventional networks are just a small portion of the problems that have to be considered when developing protocols for use in sensor networks. Simulation is often used to test new protocols that are being developed, as well as to compare old protocols. However, there is always a danger when using simulation in testing: the results are not necessarily going to be accurate or representative. To help overcome this, it is important to possess knowledge of the simulation tools available, along with their associated strengths and weaknesses. The goal of this paper is to aid developers in the selection of an appropriate simulation tool.
137 citations
TL;DR: This survey reviews problems of WSN performance evaluation both quantitatively and qualitatively while depicting a common suitable simulation model and briefly describes the most significant simulation frameworks available.
Abstract: This work has been funded by the Economy, Industry and Innovation Council, with the SOLIDMOVIL project (2I04SU044), by Fundacion Seneca with the ARENA Project (00546/PI/04), both of them from the Region of Murcia, and
by the Spanish Research Council with the ARPaq project (TEC2004-05622-C04-02/TCM).
104 citations
"A Case Study of Networks Simulation..." refers background in this paper
...The works parented in [5, 7, 8, 9, 10, 12] are the close to our work as they include some common simulators J-Sim, OMNeT++, and ns-2, which we also consider for our study....
[...]
...[8] Comparison SSF, SWANS, J-Sim, NCTUns, ns-2, OMNeT++, Ptolemy, ATEMU, EmStar, SNAP, TOSSIM Models, type of visualization...
[...]
...In [8] authors give an overview about 2009 Third Asia International Conference on Modelling & Simulation...
[...]
TL;DR: This article describes best current practices for validating simulations and forvalidating TCP models across various simulation environments and discusses interactions between scale and model validation and future challenges for the community.
Abstract: Networking engineers increasingly depend on simulation to design and deploy complex, heterogeneous networks. Similarly, networking researchers increasingly depend on simulation to investigate the behavior and performance of new protocol designs. Despite such widespread use of simulation, today there exists little common understanding of the degree of validation required for various applications of simulation. Further, only limited knowledge exists regarding the effectiveness of known validation techniques. To investigate these issues, in May 1999 DARPA and NIST organized a workshop on Network Simulation Validation. This article reports on discussions and consensus about issues that arose at the workshop. We describe best current practices for validating simulations and for validating TCP models across various simulation environments. We also discuss interactions between scale and model validation and future challenges for the community.
95 citations