Proceedings ArticleDOI
Comparative Causal Analysis of Network Log Data in Two Large ISPs
Reads0
Chats0
TLDR
In this article , a comparative analysis approach relying on causality between log time-series is proposed, where the authors classify log messages into anonymized log time series with log templates and apply causal discovery with the PC algorithm.Abstract:
Towards a collaborative analysis of log data obtained from multiple networks, we first need to clarify what kind of information is available as transferable knowledge between different networks. However, we cannot directly compare net-work log data from different sources because the data largely depends on the network architecture and equipment. In this paper, we focus on relational information among network log events that follow standardized network protocols regardless of network environment. We propose a comparative analysis approach relying on causality between log time-series. In this approach, we classify log messages into anonymized log time-series with log templates, reduce the number of log time-series to decrease processing time, and apply causal discovery with the PC algorithm. To decrease the processing time of causal analysis, we propose a new preprocessing method that reduces the number of log time-series without any domain knowledge (i.e., available in any ISPs). We compare log data obtained from two nation-wide ISPs to demonstrate the effectiveness of the causal approach in comparative analysis. read more
Citations
More filters
Proceedings ArticleDOI
Accelerating Causal Inference Based RCA Using Prior Knowledge From Functional Connectivity Inference
TL;DR: In this article , the authors propose a functional connectivity-based root cause analysis (RCA) framework to acquire and maintain prior knowledge for causal inference-based RCA approaches in dynamic networks.
References
More filters
Proceedings ArticleDOI
What Supercomputers Say: A Study of Five System Logs
Adam J. Oliner,Jon Stearley +1 more
TL;DR: This paper examines system logs from five supercomputers with the aim of providing useful insight and direction for future research into the use of such logs, and proposes a simpler and more effective filtering algorithm.
Proceedings ArticleDOI
A data clustering algorithm for mining patterns from event logs
TL;DR: A novel clustering algorithm for log file data sets is presented which helps one to detect frequent patterns from log files, to build log file profiles, and to identify anomalous log file lines.
Proceedings ArticleDOI
Drain: An Online Log Parsing Approach with Fixed Depth Tree
TL;DR: This work proposes an online log parsing method, namely Drain, that can parse logs in a streaming and timely manner, and uses a fixed depth parse tree, which encodes specially designed rules for parsing.
Proceedings ArticleDOI
LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs
Weibin Meng,Ying Liu,Yichen Zhu,Shenglin Zhang,Dan Pei,Yuqing Liu,Yihao Chen,Ruizhi Zhang,Shimin Tao,Pei Sun,Rong Zhou +10 more
TL;DR: Empowered by template2vec, a novel, simple yet effective method to extract the semantic information hidden in log templates, LogAnomaly can detect both sequential and quantitive log anomalies simultaneously, which has not been done by any previous work.
Journal ArticleDOI
HitAnomaly: Hierarchical Transformers for Anomaly Detection in System Log
TL;DR: This article proposes HitAnomaly, a log-based anomaly detection model utilizing a hierarchical transformer structure to model both log template sequences and parameter values and assess the robustness of the proposed model on unstable log data.