Journal ArticleDOI
LS-Decomposition for Robust Recovery of Sensory Big Data
Xiao-Yang Liu,Xiaodong Wang +1 more
Reads0
Chats0
TLDR
An LS-decomposition approach that decomposes a sensory reading matrix as the superposition of a L-rank matrix and a S-rank Matrix and aparse anomaly matrix is proposed, and it is proved that the convex surrogate of the LS- decomposition problem guarantees bounded recovery error under proper conditions.Abstract:
The emerging Internet of Things (IoT) systems are fueling an exponential explosion of sensory data. The major challenge to effective implementation of IoT systems is the presence of massive missing data entries, measurement noise, and anomaly readings , which motivates us to investigate the robust recovery of sensory big data. In this paper, we propose an LS-decomposition approach that decomposes a sensory reading matrix as the superposition of a L ow-rank matrix and a S parse anomaly matrix. First, based on data sets from three representative real-world IoT projects, i.e., the IntelLab project (indoor environment), the GreenOrbs project (mountain environment), and the NBDC-CTD project (ocean environment), we observe that anomaly readings are ubiquitous and cannot be ignored. Second, we prove that the convex surrogate of the LS-decomposition problem guarantees bounded recovery error under proper conditions. Third, we propose an accelerated proximal gradient algorithm that converges to the optimal solution at a rate that is inversely proportional to the square of the number of iterations. Evaluations on the above three data sets show that the proposed scheme achieves (relative) recovery error $\leq 0.05$ for missing data rate $\leq 50$ percent and almost exact recovery for missing data rate $\leq 40$ percent, while previous methods have (relative) recovery error $0.04\! \sim\! 0.15$ even at only 10 percent missing data rate.read more
Citations
More filters
Journal ArticleDOI
Unsupervised seismic facies analysis via deep convolutional autoencoders
TL;DR: In this paper, the most important goal of seismic stratigraphy studies is to interpret the elements of the seismic facies with respect to the geologic environment, and the results of the study are presented.
Journal ArticleDOI
Low-Tubal-Rank Tensor Completion Using Alternating Minimization
TL;DR: A fast iterative algorithm, called Tubal-AltMin, that is inspired by a similar approach for low-rank matrix completion is proposed that improves the recovery error by several orders of magnitude and is faster than TNN-ADMM by a factor of 5.
Journal ArticleDOI
Deep Learning-Based Automatic Clutter/Interference Detection for HFSWR
TL;DR: Field experimental results show that the Faster R-CNN based method can automatically detect the clutter and interference with decent performance and classify them with high accuracy.
Journal ArticleDOI
Single Image Cloud Removal Using U-Net and Generative Adversarial Networks
TL;DR: A novel two-stage cloud removal method, i.e., extracting the clouds and removing the thin clouds directly using U-Net and recovering the corresponding irregular missing regions using generative adversarial network (GAN), is proposed.
Journal ArticleDOI
DeepMDP: A Novel Deep-Learning-Based Missing Data Prediction Protocol for IoT
Ibrahim Kok,Suat Ozdemir +1 more
TL;DR: The experimental results show that the proposed DeepMDP protocol can significantly reduce the amount of data transmission and delay while accurately predicting missing data.
References
More filters
Book
Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers
TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Journal ArticleDOI
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
TL;DR: In this paper, the authors considered the model problem of reconstructing an object from incomplete frequency samples and showed that with probability at least 1-O(N/sup -M/), f can be reconstructed exactly as the solution to the lscr/sub 1/ minimization problem.
Journal ArticleDOI
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Amir Beck,Marc Teboulle +1 more
TL;DR: A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.
Journal ArticleDOI
Internet of Things (IoT): A vision, architectural elements, and future directions
TL;DR: In this article, the authors present a cloud centric vision for worldwide implementation of Internet of Things (IoT) and present a Cloud implementation using Aneka, which is based on interaction of private and public Clouds, and conclude their IoT vision by expanding on the need for convergence of WSN, the Internet and distributed computing directed at technological research community.
Journal ArticleDOI
Robust principal component analysis
TL;DR: In this paper, the authors prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the e1 norm.