scispace - formally typeset
Search or ask a question
Author

Paolo M. Piselli

Bio: Paolo M. Piselli is an academic researcher. The author has contributed to research in topics: Supervised learning & Boom. The author has an hindex of 1, co-authored 1 publications receiving 75 citations.

Papers
More filters
Posted Content
TL;DR: The goal is to utilize Machine Learning and statistical approaches to classify anomalous drops in periodic, but noisy, traffic patterns and found that using the intersection of the two anomaly detection methods proved to be an effective method of detecting anomalies on almost all of the models.
Abstract: Google uses continuous streams of data from industry partners in order to deliver accurate results to users. Unexpected drops in traffic can be an indication of an underlying issue and may be an early warning that remedial action may be necessary. Detecting such drops is non-trivial because streams are variable and noisy, with roughly regular spikes (in many different shapes) in traffic data. We investigated the question of whether or not we can predict anomalies in these data streams. Our goal is to utilize Machine Learning and statistical approaches to classify anomalous drops in periodic, but noisy, traffic patterns. Since we do not have a large body of labeled examples to directly apply supervised learning for anomaly classification, we approached the problem in two parts. First we used TensorFlow to train our various models including DNNs, RNNs, and LSTMs to perform regression and predict the expected value in the time series. Secondly we created anomaly detection rules that compared the actual values to predicted values. Since the problem requires finding sustained anomalies, rather than just short delays or momentary inactivity in the data, our two detection methods focused on continuous sections of activity rather than just single points. We tried multiple combinations of our models and rules and found that using the intersection of our two anomaly detection methods proved to be an effective method of detecting anomalies on almost all of our models. In the process we also found that not all data fell within our experimental assumptions, as one data stream had no periodicity, and therefore no time based model could predict it.

95 citations

Journal ArticleDOI
TL;DR: In this article , a dating system for the Italian residential real estate market from 1927 to 2019 and its interaction with credit and business cycles is presented. But the honeycomb cycle is not a model for detecting booms and busts in the housing market, even if the preliminary evidence might suggest a role for volume/transactions in detecting housing market bubbles.
Abstract: PurposeThe purpose of this paper is to provide a dating system for the Italian residential real estate market from 1927 to 2019 and investigate its interaction with credit and business cycles.Design/methodology/approachTo detect the local turning point of the Italian residential real estate market, the authors apply the honeycomb cycle developed by Janssen et al. (1994) based on the joint analysis of house prices and the number of transactions. To this end, the authors use a unique historical reconstruction of house price levels by Baffigi and Piselli (2019) in addition to data on transactions.FindingsThis study confirms the validity of the honeycomb model for the last four decades of the Italian housing market. In addition, the results show that the severe downsizing of the housing market is largely associated with business and credit contraction, certainly contributing to exacerbating the severity of the recession. Finally, preliminary evidence suggests that whenever a price bubble occurs, it is coincident with the start of phase 2 of the honeycomb cycle.Originality/valueTo the best of the authors’ knowledge, this is the first time that the honeycomb approach has been tested over such a long historical period and compared to the cyclic features of financial and real aggregates. In addition, even if the honeycomb cycle is not a model for detecting booms and busts in the housing market, the preliminary evidence might suggest a role for volume/transactions in detecting housing market bubbles.

Cited by
More filters
Posted Content
TL;DR: A structured and comprehensive overview of research methods in deep learning-based anomaly detection, grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted.
Abstract: Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and comprehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. For each category, we present we also present the advantages and limitations and discuss the computational complexity of the techniques in real application domains. Finally, we outline open issues in research and challenges faced while adopting these techniques.

522 citations

Proceedings ArticleDOI
19 Jul 2018
TL;DR: The effectiveness of Long Short-Term Memory networks, a type of Recurrent Neural Network, in overcoming issues using expert-labeled telemetry anomaly data from the Soil Moisture Active Passive (SMAP) satellite and the Mars Science Laboratory (MSL) rover, Curiosity is demonstrated.
Abstract: As spacecraft send back increasing amounts of telemetry data, improved anomaly detection systems are needed to lessen the monitoring burden placed on operations engineers and reduce operational risk. Current spacecraft monitoring systems only target a subset of anomaly types and often require costly expert knowledge to develop and maintain due to challenges involving scale and complexity. We demonstrate the effectiveness of Long Short-Term Memory (LSTMs) networks, a type of Recurrent Neural Network (RNN), in overcoming these issues using expert-labeled telemetry anomaly data from the Soil Moisture Active Passive (SMAP) satellite and the Mars Science Laboratory (MSL) rover, Curiosity. We also propose a complementary unsupervised and nonparametric anomaly thresholding approach developed during a pilot implementation of an anomaly detection system for SMAP, and offer false positive mitigation strategies along with other key improvements and lessons learned during development.

440 citations

Journal ArticleDOI
TL;DR: A background on the challenges which may be encountered when applying anomaly detection techniques to IoT data is provided, with examples of applications for the IoT anomaly detection taken from the literature.
Abstract: Anomaly detection is a problem with applications for a wide variety of domains; it involves the identification of novel or unexpected observations or sequences within the data being captured. The majority of current anomaly detection methods are highly specific to the individual use case, requiring expert knowledge of the method as well as the situation to which it is being applied. The Internet of Things (IoT) as a rapidly expanding field offers many opportunities for this type of data analysis to be implemented, however, due to the nature of the IoT, this may be difficult. This review provides a background on the challenges which may be encountered when applying anomaly detection techniques to IoT data, with examples of applications for the IoT anomaly detection taken from the literature. We discuss a range of approaches that have been developed across a variety of domains, not limited to IoT due to the relative novelty of this application. Finally, we summarize the current challenges being faced in the anomaly detection domain with a view to identifying potential research opportunities for the future.

271 citations

Journal ArticleDOI
TL;DR: This work proposes a deep learning based approach for supervised multi-time series anomaly detection that combines a Convolutional Neural Network and a Recurrent Neural Network in different ways and refers to this architecture as Multi-head CNN–RNN.

196 citations

Proceedings ArticleDOI
Hansheng Ren1, Bixiong Xu1, Yujing Wang1, Chao Yi1, Congrui Huang1, Xiaoyu Kou1, Tony Xing1, Mao Yang1, Jie Tong1, Qi Zhang1 
TL;DR: Wang et al. as mentioned in this paper proposed a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN) for time-series anomaly detection.
Abstract: Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this paper, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.

158 citations