Time-Series Anomaly Detection Service at Microsoft
Hansheng Ren,Bixiong Xu,Yujing Wang,Chao Yi,Congrui Huang,Xiaoyu Kou,Tony Xing,Mao Yang,Jie Tong,Qi Zhang +9 more
TLDR
Wang et al. as mentioned in this paper proposed a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN) for time-series anomaly detection.Abstract:
Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this paper, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.read more
Citations
More filters
Proceedings Article
Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting
Defu Cao,Yujing Wang,Juanyong Duan,Ce Zhang,Xia Zhu,Congrui Huang,Yunhai Tong,Bixiong Xu,Jing Bai,Jie Tong,Qi Zhang +10 more
TL;DR: Spectral Temporal Graph Neural Network (StemGNN) is proposed to further improve the accuracy of multivariate time-series forecasting and learns inter-series correlations automatically from the data without using pre-defined priors.
Journal ArticleDOI
A Survey on Machine Learning Techniques for Cyber Security in the Last Decade
TL;DR: This paper aims to provide a comprehensive overview of the challenges that ML techniques face in protecting cyberspace against attacks, by presenting a literature on ML techniques for cyber security including intrusion detection, spam detection, and malware detection on computer networks and mobile networks in the last decade.
Posted Content
Multivariate Time-series Anomaly Detection via Graph Attention Network
Hang Zhao,Yujing Wang,Juanyong Duan,Congrui Huang,Defu Cao,Yunhai Tong,Bixiong Xu,Jing Bai,Jie Tong,Qi Zhang +9 more
TL;DR: This paper proposes a novel self-supervised framework for multivariate time-series anomaly detection that outperforms other state-of-the-art models on three real-world datasets and has good interpretability and is useful for anomaly diagnosis.
Posted Content
TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks
TL;DR: TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) that can effectively detect anomalies and outperform baseline methods in most cases, and has the highest averaged F1 score across all the datasets.
Posted Content
RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks.
TL;DR: This paper proposes RobustTAD, a Robust Time series Anomaly Detection framework by integrating robust seasonal-trend decomposition and convolutional neural network for time series data and introduces label-based weight and value- based weight in the loss function by utilizing the unbalanced nature of the time series anomaly detection problem.
References
More filters
Classification and Regression by randomForest
Andy Liaw,Matthew C. Wiener +1 more
TL;DR: random forests are proposed, which add an additional layer of randomness to bagging and are robust against overfitting, and the randomForest package provides an R interface to the Fortran programs by Breiman and Cutler.
Proceedings ArticleDOI
Saliency Detection: A Spectral Residual Approach
Xiaodi Hou,Liqing Zhang +1 more
TL;DR: A simple method for the visual saliency detection is presented, independent of features, categories, or other forms of prior knowledge of the objects, and a fast method to construct the corresponding saliency map in spatial domain is proposed.
Book
Extreme value theory : an introduction
Laurens de Haan,Ana Ferreira +1 more
TL;DR: This paper presented an excellent introduction to extreme value theory at the graduate level, requiring only some mathematical maturity, focusing on the probabilistic and statistical aspects of extreme values without major emphasis on such related topics as regular variation, point processes, empirical distribution functions, and Brownian motion.
Posted Content
Tutorial on Variational Autoencoders
TL;DR: This tutorial introduces the intuitions behind VAEs, explains the mathematics behind them, and describes some empirical behavior.
Book
Computational Frameworks for the Fast Fourier Transform
TL;DR: The Radix-2 Frameworks, a collection of general and high performance FFTs designed to solve the multi-Dimensional FFT problem of Prime Factor and Convolution, are presented.