Extracting Kernel Dataset from Big Sensory Data in Wireless Sensor Networks

TLDR: The novel concept of Kernel Dataset, which can represent the vast information carried by big sensory data with the information loss rate being less than $\epsilon$, and two distributed algorithms of maintaining the correlation coefficients among sensor nodes are developed are developed.

Abstract: The amount of sensory data manifests an explosive growth due to the increasing popularity of Wireless Sensor Networks (WSNs). The scale of sensory data in many applications has already exceeded several petabytes annually, which is beyond the computation and transmission capabilities of conventional WSNs. On the other hand, the information carried by big sensory data has high redundancy because of strong correlation among sensory data. In this paper, we introduce the novel concept of $\epsilon$ -Kernel Dataset , which is only a small data subset and can represent the vast information carried by big sensory data with the information loss rate being less than $\epsilon$ , where $\epsilon$ can be arbitrarily small. We prove that drawing the minimum $\epsilon$ -Kernel Dataset is polynomial time solvable and provide a centralized algorithm with $O(n^3)$ time complexity. Furthermore, a distributed algorithm with constant complexity $O(1)$ is designed. It is shown that the result returned by the distributed algorithm can satisfy the $\epsilon$ requirement with a near optimal size. Furthermore, two distributed algorithms of maintaining the correlation coefficients among sensor nodes are developed. Finally, the extensive real experiment results and simulation results are presented. The results indicate that all the proposed algorithms have high performance in terms of accuracy and energy efficiency.