Data Mining Practical Machine Learning Tools and Techniques

Machine learning techniques are being widely used to develop an intrusion detection system (IDS) for detecting and classifying cyberattacks at the network-level and the host-level in a timely and automatic manner. However, many challenges arise since malicious attacks are continually changing and are occurring in very large volumes requiring a scalable solution. There are different malware datasets available publicly for further research by cyber security community. However, no existing study has shown the detailed analysis of the performance of various machine learning algorithms on various publicly available datasets. Due to the dynamic nature of malware with continuously changing attacking methods, the malware datasets available publicly are to be updated systematically and benchmarked. In this paper, a deep neural network (DNN), a type of deep learning model, is explored to develop a flexible and effective IDS to detect and classify unforeseen and unpredictable cyberattacks. The continuous change in network behavior and rapid evolution of attacks makes it necessary to evaluate various datasets which are generated over the years through static and dynamic approaches. This type of study facilitates to identify the best algorithm which can effectively work in detecting future cyberattacks. A comprehensive evaluation of experiments of DNNs and other classical machine learning classifiers are shown on various publicly available benchmark malware datasets. The optimal network parameters and network topologies for DNNs are chosen through the following hyperparameter selection methods with KDDCup 99 dataset. All the experiments of DNNs are run till 1,000 epochs with the learning rate varying in the range [0.01-0.5]. The DNN model which performed well on KDDCup 99 is applied on other datasets, such as NSL-KDD, UNSW-NB15, Kyoto, WSN-DS, and CICIDS 2017, to conduct the benchmark. Our DNN model learns the abstract and high-dimensional feature representation of the IDS data by passing them into many hidden layers. Through a rigorous experimental testing, it is confirmed that DNNs perform well in comparison with the classical machine learning classifiers. Finally, we propose a highly scalable and hybrid DNNs framework called scale-hybrid-IDS-AlertNet which can be used in real-time to effectively monitor the network traffic and host-level events to proactively alert possible cyberattacks.

/pdf/deep-learning-approach-for-intelligent-intrusion-detection-19gksnoeyl.pdf

Deep Learning Approach for Intelligent Intrusion Detection System

Data preprocessing is widely recognized as an important stage in anomaly detection. This paper reviews the data preprocessing techniques used by anomaly-based network intrusion detection systems (NIDS), concentrating on which aspects of the network traffic are analyzed, and what feature construction and selection methods have been used. Motivation for the paper comes from the large impact data preprocessing has on the accuracy and capability of anomaly-based NIDS. The review finds that many NIDS limit their view of network traffic to the TCP/IP packet headers. Time-based statistics can be derived from these headers to detect network scans, network worm behavior, and denial of service attacks. A number of other NIDS perform deeper inspection of request packets to detect attacks against network services and network applications. More recent approaches analyze full service responses to detect attacks targeting clients. The review covers a wide range of NIDS, highlighting which classes of attack are detectable by each of these approaches.

Data preprocessing is found to predominantly rely on expert domain knowledge for identifying the most relevant parts of network traffic and for constructing the initial candidate set of traffic features. On the other hand, automated methods have been widely used for feature extraction to reduce data dimensionality, and feature selection to find the most relevant subset of features from this candidate set. The review shows a trend toward deeper packet inspection to construct more relevant features through targeted content parsing. These context sensitive features are required to detect current attacks.

Data preprocessing for anomaly based network intrusion detection : a review

Sliding mode control (SMC) is recognized as robust controller with a high stability in a wide range of operating conditions, although it suffers from chattering problem. In addition, it cannot be directly applied to multiswitches power converters. In this paper, a high performance and fixed switching frequency sliding mode controller is proposed for a single-phase unipolar inverter. The chattering problem of SMC is eliminated by smoothing the control law in a narrow boundary layer, and a pulsewidth modulator produces the fixed frequency switching law for the inverter. The smoothing procedure is based on limitation of pulsewidth modulator. Although the smoothed control law limits the performance of SMC, regulation and dynamic response of the inverter output voltage are in an acceptable superior range. The performance of the proposed controller is verified by both simulation and experiments on a prototype 6-kVA inverter. The experimental results show that the total harmonic distortion of the output voltage is less than 1.1% and 1.7% at maximum linear and nonlinear load, respectively. Furthermore, the output dynamic performance of the inverter strictly conforms the standard IEC62040-3. Moreover, the measured efficiency of the inverter in the worst condition is better than 95.5%.

Fixed Switching Frequency Sliding Mode Control for Single-Phase Unipolar Inverters

This paper presents a state-of-the-art survey of smartphone (SP)-based solutions for fall detection and prevention. Falls are considered as major health hazards for both the elderly and people with neurodegenerative diseases. To mitigate the adverse consequences of falling, a great deal of research has been conducted, mainly focused on two different approaches, namely, fall detection and fall prevention. Required hardware for both fall detection and prevention are also available in SPs. Consequently, researchers' interest in finding SP-based solutions has increased dramatically over recent years. To the best of our knowledge, there has been no published review on SP-based fall detection and prevention. Thus in this paper, we present the taxonomy for SP-based fall detection and prevention solutions and systematic comparisons of existing studies. We have also identified three challenges and three open issues for future research, after reviewing the existing articles. Our time series analysis demonstrates a trend towards the integration of external sensing units with SPs for improvement in usability of the systems.

https://www.mdpi.com/1424-8220/14/4/7181/pdf

Smartphone-based solutions for fall detection and prevention: challenges and open issues

Although KDD99 dataset is more than 15 years old, it is still widely used in academic research. To investigate wide usage of this dataset in Machine Learning Research (MLR) and Intrusion Detection Systems (IDS); this study reviews 149 research articles from 65 journals indexed in Science Citation In- dex Expanded and Emerging Sources Citation Index during the last six years (2010–2015). If we include papers presented in other indexes and conferences, number of studies would be tripled. The number of published studies shows that KDD99 is the most used dataset in IDS and machine learning areas, and it is the de facto dataset for these research areas. To show recent usage of KDD99 and the related sub-dataset (NSL-KDD) in IDS and MLR, the following de- scriptive statistics about the reviewed studies are given: main contribution of articles, the applied algorithms,

A Review of KDD99 Dataset Usage in Intrusion Detection and Machine Learning between 2010 and 2015

Nonlinear sensors and microcontrollers are used in many embedded system designs. As the input-output characteristic of most sensors is nonlinear in nature, obtaining data from a nonlinear sensor by using an integer microcontroller has always been a design challenge. This paper discusses the implementation of six software-based sensor linearization algorithms for low-cost microcontrollers. The comparative study of the linearization algorithms is performed by using a nonlinear optical distance-measuring sensor. The performance of the algorithms is examined with respect to memory space usage, linearization accuracy and algorithm execution time. The implementation and comparison results can be used for selection of a linearization algorithm based on the sensor transfer function, expected linearization accuracy and microcontroller capacity.

Implementation of software-based sensor linearization algorithms on low-cost microcontrollers.

This paper presents the design and implementation of a multi-channel remote controller (MCRC) for home and office appliances. The aim of this work is to integrate several existing remote controller channels in a common platform. The proposed controller supports access to controlled environment via Web Page, Smart-Phone (SP), PDA, GSM network and telephone lines. The suggested controller is more reliable than conventional ones especially during emergency condition such as main server failure and interruption in GSM or Internet network. The suggested system enables easy and flexible access to controlled devices. The hardware part of the controller contains a home server which is built upon PC and auxiliary microcontrollers. The feasibility of this architecture has been demonstrated with a prototype implementation and presented in details.

A multi-channel remote controller for home and office appliances

Parkinson's disease monitoring from gait analysis via foot-worn sensors

Remote care and telemonitoring have become essential component of current geriatric medicine. Intelligent use of wireless sensors is a major issue in relevant computational studies to realize these concepts in practice. While there has been a growing interest in recognizing daily activities of patients through wearable sensors, the efforts towards utilizing the streaming data from these sensors for clinical practices are limited. Here, we present a practical application of clinical data mining from wearable sensors with a particular objective of diagnosing Parkinson’s Disease from gait analysis through a sets of ground reaction force (GRF) sensors worn under the foots. We introduce a supervised learning method based on Random Forests that analyze the multi-sensor data to classify the person wearing these sensors. We offer to extract a set of time-domain and frequency-domain features that would be effective in distinguishing normal and diseased people from their gait signals. The experimental results on a benchmark dataset have shown that proposed method can significantly outperform the previous methods reported in the literature.

Hamit Erdem

Papers

A Review of KDD99 Dataset Usage in Intrusion Detection and Machine Learning between 2010 and 2015

Implementation of software-based sensor linearization algorithms on low-cost microcontrollers.

A multi-channel remote controller for home and office appliances

Parkinson's disease monitoring from gait analysis via foot-worn sensors

A Random Forest Method to Detect Parkinson’s Disease via Gait Analysis