Kieran Flanagan

Bio: Kieran Flanagan is an academic researcher from Athlone Institute of Technology. The author has contributed to research in topics: Anomaly detection & Computer science. The author has an hindex of 3, co-authored 5 publications receiving 22 citations.

Network anomaly detection in time series using distance based outlier detection with cluster density analysis

Abstract: It is common place in any organizational environment that data stored internally does not necessarily belong to the company storing the data. In such cases, keeping this data secured is of critical importance. If such data is compromised, it can lead to devastating effects on both the public image of the organization and the relations between said company and its business partners. To combat this surge in malicious activity in recent years, research has focused on using anomaly detection techniques to detect possible malicious activity on a network. This paper proposes an evolution of the MCOD (Micro-Clustering Outlier Detection) machine learning algorithm. Designed to implement a time-series approach along with using both distance based outlier detection and cluster density analysis, we analysis the results of this algorithm on real-world data.

Self-configuring NetFlow anomaly detection using cluster density analysis

Abstract: The growing number of malicious network attacks has resulted in the need for a fast, reliable method to identify possible malicious activity. For any organization, it is critical that confidential and proprietary data is sufficiently secured to address both legal and contractual obligations. The changing nature of security attacks has caused a surge of interest in anomaly detection mechanisms. Such mechanisms are suitable as they can dynamically adapt to changed network conditions and threats without security personnel intervention. While anomaly detection mechanisms have significant potential, they are technically limited. Many anomaly detection approaches are unsuitable for real time environments. The approaches also typically operate based on “what is common is normal”. Mechanisms are typically singular in focus analysing data on one specific type. This paper proposes a novel framework to detect anomalies previously hidden within current detection techniques. The approach is easily extensible taking input from many security assessment applications; network traffic, asset criticality. Using time based correlations with historic data; a method for generating a normalized view of activity on the network is achieved. Once normality has been established for specific time intervals an extensible environment is implemented which allows for the active monitoring of anomalies in real-time. Anomalies which had sufficient “commonality” to remain undetected by other mechanisms are identified and analysed. The proposed solution is completely autonomous, capable of acting independently with no previous knowledge required. The presented results describe NetFlow activity of the NPD Groups' network over a 24-hour period and outline real world anomalies that were detected.

2D2N: A Dynamic Degenerative Neural Network for Classification of Images of Live Network Data

Abstract: The detection of new, novel attacks on organizational networks is a problem of ever-increasing relevance in today’s society. Research in the area is focused on the detection of “Zero-Day” and “Black Swan” events through the use of machine learning technologies. Where previous technologies needed a known example of malicious behavior to detect a similar event, recent advances in anomaly detection on network activity has shown promise of detecting novel attacks. In a real word environment however, novel behavior occurs relatively frequently as users utilize new software applications and new standards in networking. Changes such as these, while of notable importance to network security technicians, may not present themselves as an imminent threat to a network. This paper proposes a novel method for the detection and classification of changes in networking behavior. Through the use of a Dynamic Degenerative Neural Network (2D2N), changes in recognizable user activity are dynamically classified and stored for future reference. Through the use of a time-based entropy function, infrequent activity can be analyzed and given precedence over frequent activity. This aids in the classification of abnormal activity for fast, efficient assessment by the relevant persons in an organization. The proposed method enables the detection, classification and scoring of any and all user activity on a network. Evaluation of the proposed method is based upon live data gathered from a large, multinational organization.

NetFlow Anomaly Detection Though Parallel Cluster Density Analysis in Continuous Time-Series

Abstract: The increase in malicious network based attacks has resulted in a growing interest in network anomaly detection. The ability to detect unauthorized or malicious activity on a network is of importance to any organization. With the increase in novel attacks, anomaly detection techniques can be more successful in detecting unknown malicious activity in comparison to traditional signature based methods. However, in a real-world environment, there are many variables that cannot be simulated. This paper proposes an architecture where parallel clustering algorithms work concurrently in order to detect abnormalities that may be lost while traversing over time-series windows. The presented results describe the NetFlow activity of the NPD Group, Inc. over a 24-hour period. The presented results contain real-world anomalies that were detected.

Learning to Classify: A Flow-Based Relation Network for Encrypted Traffic Classification

Abstract: As the size and source of network traffic increase, so does the challenge of monitoring and analyzing network traffic. The challenging problems of classifying encrypted traffic are the imbalanced property of network data, the generalization on an unseen dataset, and overly dependent on data size. In this paper, we propose an application of a meta-learning approach to address these problems in encrypted traffic classification, named Flow-Based Relation Network (RBRN). The RBRN is an end-to-end classification model that learns representative features from the raw flows and then classifies them in a unified framework. Moreover, we design “hallucinator” to produce additional training samples for the imbalanced classification, and then focus on meta-learning to classify unseen categories from few labeled samples. We validate the effectiveness of the RBRN on the real-world network traffic dataset, and the experimental results demonstrate that the RBRN can achieve an excellent classification performance and outperform the state-of-the-art methods on encrypted traffic classification. What is more interesting, our model trained on the real-world dataset can generalize very well to unseen datasets, outperforming multiple state-of-art methods.

A Benchmark Study on Time Series Clustering.

Abstract: This paper presents the first time series clustering benchmark utilizing all time series datasets currently available in the University of California Riverside (UCR) archive -- the state of the art repository of time series data. Specifically, the benchmark examines eight popular clustering methods representing three categories of clustering algorithms (partitional, hierarchical and density-based) and three types of distance measures (Euclidean, dynamic time warping, and shape-based). We lay out six restrictions with special attention to making the benchmark as unbiased as possible. A phased evaluation approach was then designed for summarizing dataset-level assessment metrics and discussing the results. The benchmark study presented can be a useful reference for the research community on its own; and the dataset-level assessment metrics reported may be used for designing evaluation frameworks to answer different research questions.

A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment

Abstract: The Internet of Things (IoT) appliances often expose sensitive data, either directly or indirectly. They may, for instance, tell whether you are at home right now or what your long or short-term habits are. Therefore, it is crucial to protect such devices against adversaries and has in place an early warning system which indicates compromised devices in a quick and efficient manner. In this paper, we propose time window embedding solutions that efficiently process a massive amount of data and have a low-memory-footprint at the same time. On top of the proposed embedding vectors, we use the core anomaly detection unit. It is a classifier that is based on the transformer’s encoder component followed by a feed-forward neural network. We have compared the proposed method with other classical machine-learning algorithms. Therefore, in the paper, we formally evaluate various machine-learning schemes and discuss their effectiveness in the IoT-related context. Our proposal is supported by detailed experiments that have been conducted on the recently published Aposemat IoT-23 dataset.

Ensemble-Based Spam Detection in Smart Home IoT Devices Time Series Data Using Machine Learning Techniques

Abstract: The number of Internet of Things (IoT) devices is growing at a fast pace in smart homes, producing large amounts of data, which are mostly transferred over wireless communication channels However, various IoT devices are vulnerable to different threats, such as cyber-attacks, fluctuating network connections, leakage of information, etc Statistical analysis and machine learning can play a vital role in detecting the anomalies in the data, which enhances the security level of the smart home IoT system which is the goal of this paper This paper investigates the trustworthiness of the IoT devices sending house appliances’ readings, with the help of various parameters such as feature importance, root mean square error, hyper-parameter tuning, etc A spamicity score was awarded to each of the IoT devices by the algorithm, based on the feature importance and the root mean square error score of the machine learning models to determine the trustworthiness of the device in the home network A dataset publicly available for a smart home, along with weather conditions, is used for the methodology validation The proposed algorithm is used to detect the spamicity score of the connected IoT devices in the network The obtained results illustrate the efficacy of the proposed algorithm to analyze the time series data from the IoT devices for spam detection

A benchmark study on time series clustering

Abstract: This paper presents the first time series clustering benchmark utilizing all time series datasets currently available in the University of California Riverside (UCR) archive — the state of the art repository of time series data. Specifically, the benchmark examines eight popular clustering methods representing three categories of clustering algorithms (partitional, hierarchical and density-based) and three types of distance measures (Euclidean, dynamic time warping, and shape-based), while adhering to six restrictions on datasets and methods to make the comparison as unbiased as possible. A phased evaluation approach was then designed for summarizing dataset-level assessment metrics and discussing the results. The benchmark study presented can be a useful reference for the research community on its own; and the dataset-level assessment metrics reported may be used for designing evaluation frameworks to answer different research questions.