In this paper, we review the background and state-of-the-art of big data. We first introduce the general background of big data and review related technologies, such as could computing, Internet of Things, data centers, and Hadoop. We then focus on the four phases of the value chain of big data, i.e., data generation, data acquisition, data storage, and data analysis. For each phase, we introduce the general background, discuss the technical challenges, and review the latest advances. We finally examine the several representative applications of big data, including enterprise management, Internet of Things, online social networks, medial applications, collective intelligence, and smart grid. These discussions aim to provide a comprehensive overview and big-picture to readers of this exciting area. This survey is concluded with a discussion of open problems and future directions.

Big Data: A Survey

This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Recent technological advancements have led to a deluge of data from distinctive domains (e.g., health care and scientific sensors, user-generated data, Internet and financial companies, and supply chain systems) over the past two decades. The term big data was coined to capture the meaning of this emerging trend. In addition to its sheer volume, big data also exhibits other unique characteristics as compared with traditional data. For instance, big data is commonly unstructured and require more real-time analysis. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. In this paper, we present a literature survey and system tutorial for big data analytics platforms, aiming to provide an overall picture for nonexpert readers and instill a do-it-yourself spirit for advanced audiences to customize their own big-data solutions. First, we present the definition of big data and discuss big data challenges. Next, we present a systematic framework to decompose big data systems into four sequential modules, namely data generation, data acquisition, data storage, and data analytics. These four modules form a big data value chain. Following that, we present a detailed survey of numerous approaches and mechanisms from research and industry communities. In addition, we present the prevalent Hadoop framework for addressing big data challenges. Finally, we outline several evaluation benchmarks and potential research directions for big data systems.

Toward Scalable Systems for Big Data Analytics: A Technology Tutorial

Recently, Wireless Sensor Networks (WSNs) have attracted lot of attention due to their pervasive nature and their wide deployment in Internet of Things, Cyber Physical Systems, and other emerging areas. The limited energy associated with WSNs is a major bottleneck of WSN technologies. To overcome this major limitation, the design and development of efficient and high performance energy harvesting systems for WSN environments are being explored. We present a comprehensive taxonomy of the various energy harvesting sources that can be used by WSNs. We also discuss various recently proposed energy prediction models that have the potential to maximize the energy harvested in WSNs. Finally, we identify some of the challenges that still need to be addressed to develop cost-effective, efficient, and reliable energy harvesting systems for the WSN environment.

Energy harvesting in wireless sensor networks: A comprehensive review

Wireless sensor networks (WSNs) are networks of distributed autonomous devices that can sense or monitor physical or environmental conditions cooperatively. WSNs face many challenges, mainly caused by communication failures, storage and computational constraints and limited power supply. Paradigms of computational intelligence (CI) have been successfully used in recent years to address various challenges such as data aggregation and fusion, energy aware routing, task scheduling, security, optimal deployment and localization. CI provides adaptive mechanisms that exhibit intelligent behavior in complex and dynamic environments like WSNs. CI brings about flexibility, autonomous behavior, and robustness against topology changes, communication failures and scenario changes. However, WSN developers are usually not or not completely aware of the potential CI algorithms offer. On the other side, CI researchers are not familiar with all real problems and subtle requirements of WSNs. This mismatch makes collaboration and development difficult. This paper intends to close this gap and foster collaboration by offering a detailed introduction to WSNs and their properties. An extensive survey of CI applications to various problems in WSNs from various research areas and publication venues is presented in the paper. Besides, a discussion on advantages and disadvantages of CI algorithms over traditional WSN solutions is offered. In addition, a general evaluation of CI algorithms is presented, which will serve as a guide for using CI algorithms for WSNs.

Computational Intelligence in Wireless Sensor Networks: A Survey

SensorScope is a turnkey solution for environmental monitoring systems, based on a wireless sensor network and resulting from a collaboration between environmental and network researchers. Given the interest in climate change, environmental monitoring is a domain where sensor networks will have great impact by providing high resolution spatio-temporal data for long periods of time. SensorScope is such a system, which has already been successfully deployed multiple times in various environments (e.g., mountainous, urban). Here, we describe the overall hardware and software architectures and especially focus on the sensor network itself. We also describe one of our most prominent deployments, on top of a rock glacier in Switzerland, which resulted in the description of a micro-climate phenomenon leading to cold air release from a rock-covered glacier in a region of high alpine risks. Another focus of this paper is the description of what happened behind the scenes to turn SensorScope from a laboratory experiment into successful outdoor deployments in harsh environments. Illustrated by various examples, we point out many lessons learned while working on the project. We indicate the importance of simple code, well suited to the application, as well as the value of close interaction with end-users in planning and running the network and finally exploiting the data.

SensorScope: Application-specific sensor network for environmental monitoring

Environmental monitoring constitutes an important field of application for wireless sensor networks. Given the severity of potential climate changes, environmental impact on cities, and pollution, it is a domain where sensor networks can have great impact and as such, is getting more and more attention. Current data collection techniques are indeed rather limited and make use of very expensive sensing stations, leading to a lack of appropriate observations. In this paper, we present SensorScope, a collaborative project between environmental and network researchers, that aims at providing an efficient and inexpensive out-of-the-box environmental monitoring system, based on a wireless sensor network. We especially focus on data gathering and present the hardware and network architecture of SensorScope. We also describe a real-world deployment, which took place on a rock glacier in Switzerland, as well as the results we obtained.

/pdf/sensorscope-out-of-the-box-environmental-monitoring-xw95aqd7hb.pdf

SensorScope: Out-of-the-Box Environmental Monitoring

The successful deployment of a wireless sensor network is a difficult task, littered with traps and pitfalls. Even a functional network does not guarantee gathering meaningful data. In SensorScope, with its multiple campaigns in various environments (e.g., urban, high-mountain), we have acquired much knowledge in planning, conducting, and managing real-world sensor network deployments. In this paper, we share our experience by stepping through the entire process, from the preparatory hard- and software development to the actual field deployment. Illustrated by numerous real-life examples, excerpted from our own experience, we point out many potential problems along this way and their possible solutions. We also indicate the importance of a close interaction with the end-user community in planning and running the network, and finally exploiting the data.

/pdf/the-hitchhiker-s-guide-to-successful-wireless-sensor-network-59e7vmchfs.pdf

The hitchhiker's guide to successful wireless sensor network deployments

This paper outlines a vision for community-driven sensing of our environment. At its core, community sensing is a dynamic new form of mobile geosensor network. We believe that community sensing networks, in order to be widely deployable and sustainable, need to follow utilitarian approaches towards sensing and data management. Current projects exploring community sensing have paid less attention to these underlying fundamental principles. We illustrate this vision through OpenSense -- a large project that aims to explore community sensing driven by air pollution monitoring.

OpenSense: open community driven sensing of environment

While wireless sensor networks have been extensively studied in the past few years, most results are of theoretical nature and were obtained outside of a practical context. This can be problematic for real applications, especially in the area of environmental monitoring where many factors, such as harsh weather conditions, can greatly influence the performance of such a network, while reliable delivery and high-quality measurements are required. SensorScope is an interdisciplinary project, elaborated by environmental and networking researchers, that aims at narrowing the gap between theory and practice. Several successful real-world deployments have already been undertaken in rugged environments. In this paper, we analyze the particular requirements of environmental monitoring and how these requirements have been met in the SensorScope project. We also present an application example of a deployment, undertaken in a harsh mountain environment.

/pdf/wireless-sensor-networks-for-environmental-monitoring-the-17vzvxkfnt.pdf

Guillermo Barrenetxea

Papers

SensorScope: Application-specific sensor network for environmental monitoring

SensorScope: Out-of-the-Box Environmental Monitoring

The hitchhiker's guide to successful wireless sensor network deployments

OpenSense: open community driven sensing of environment

Wireless Sensor Networks for Environmental Monitoring: The SensorScope Experience