Data Mining Practical Machine Learning Tools and Techniques

In the Internet age, malware (such as viruses, trojans, ransomware, and bots) has posed serious and evolving security threats to Internet users. To protect legitimate users from these threats, anti-malware software products from different companies, including Comodo, Kaspersky, Kingsoft, and Symantec, provide the major defense against malware. Unfortunately, driven by the economic benefits, the number of new malware samples has explosively increased: anti-malware vendors are now confronted with millions of potential malware samples per year. In order to keep on combating the increase in malware samples, there is an urgent need to develop intelligent methods for effective and efficient malware detection from the real and large daily sample collection. In this article, we first provide a brief overview on malware as well as the anti-malware industry, and present the industrial needs on malware detection. We then survey intelligent malware detection methods. In these methods, the process of detection is usually divided into two stages: feature extraction and classification/clustering. The performance of such intelligent malware detection approaches critically depend on the extracted features and the methods for classification/clustering. We provide a comprehensive investigation on both the feature extraction and the classification/clustering techniques. We also discuss the additional issues and the challenges of malware detection using data mining techniques and finally forecast the trends of malware development.

https://dl.acm.org/doi/pdf/10.1145/3073559

A Survey on Malware Detection Using Data Mining Techniques

An imbalanced training data set can pose serious problems for many real-world data mining tasks that employ SVMs to conduct supervised learning. In this paper, we propose a kernel-boundary-alignment algorithm, which considers THE training data imbalance as prior information to augment SVMs to improve class-prediction accuracy. Using a simple example, we first show that SVMs can suffer from high incidences of false negatives when the training instances of the target class are heavily outnumbered by the training instances of a nontarget class. The remedy we propose is to adjust the class boundary by modifying the kernel matrix, according to the imbalanced data distribution. Through theoretical analysis backed by empirical study, we show that our kernel-boundary-alignment algorithm works effectively on several data sets.

/pdf/kba-kernel-boundary-alignment-considering-imbalanced-data-1x43ris4xt.pdf

KBA: kernel boundary alignment considering imbalanced data distribution

In this paper, we propose the class-boundaryalignment algorithm to augment SVMs to deal with imbalanced training-data problems posed by many emerging applications (e.g., image retrieval, video surveillance, and gene profiling). Through a simple example, we first show that SVMs can be ineffective in determining the class boundary when the training instances of the target class are heavily outnumbered by the nontarget training instances. To remedy this problem, we propose to adjust the class boundary either by transforming the kernel function when the training data can be represented in a vector space, or by modifying the kernel matrix when the data do not have a vector-space representation (e.g., sequence data). Through theoretical analysis and empirical study, we show that the classboundary-alignment algorithm works effectively with images (data that have a vector-space representation) and video sequences (data that do not have a vector-space representation).

/pdf/class-boundary-alignment-for-imbalanced-dataset-learning-1i4jq53tbb.pdf

Class-Boundary Alignment for Imbalanced Dataset Learning

https://ro.uow.edu.au/cgi/viewcontent.cgi?article=1608&context=eispapers

AdaBoost with SVM-based component classifiers

The increase of malware that are exploiting the Internet daily has become a serious threat. The manual heuristic inspection of malware analysis is no longer considered effective and efficient compared against the high spreading rate of malware. Hence, automated behavior-based malware detection using machine learning techniques is considered a profound solution. The behavior of each malware on an emulated (sandbox) environment will be automatically analyzed and will generate behavior reports. These reports will be preprocessed into sparse vector models for further machine learning (classification). The classifiers used in this research are k-Nearest Neighbors (kNN), Naive Bayes, J48 Decision Tree, Support Vector Machine (SVM), and Multilayer Perceptron Neural Network (MlP). Based on the analysis of the tests and experimental results of all the 5 classifiers, the overall best performance was achieved by J48 decision tree with a recall of 95.9%, a false positive rate of 2.4%, a precision of 97.3%, and an accuracy of 96.8%. In summary, it can be concluded that a proof-of-concept based on automatic behavior-based malware analysis and the use of machine learning techniques could detect malware quite effectively and efficiently.

Analysis of Machine learning Techniques Used in Behavior-Based Malware Detection

Traffic jam in Jakarta, Indonesia has become a crucial problem for the society. A Traffic Management Center has been built by the police, in this case Polda Metro Jaya to help people to get the latest information regarding traffic jam. Twitter has been used by TMC Polda Metro Jaya to spread the news of traffic. With its limitations, Twitter doesn't provide good user interface in the case of traffic condition. In this paper, information extraction technique is used to get the data of traffic, so that the traffic information can be presented in map view as a mobile application of Android. Natural Language Processing can be used to extract information from a tweet. A tweet will be tokenized, then each of the token will be assigned to a particular part-of-speech tag while analyzing the sentence by using rule based approach. Based on the rules, information of traffic can be extracted in the form of template which consist of time, origin, destination and condition. By using Google Map, the extracted information will be presented in 3 different colors for 3 different levels of traffic condition. Early experiment with limited vocabulary and rules has showed promising result.

Traffic Condition Information Extraction & Visualization from Social Media Twitter for Android Mobile Application

Development of an accurate laboratory diagnostic tool, as recommended by WHO, is the key step to overcome the serious global health burden caused by malaria. This study aims to explore the possibility of computerized diagnosis of malaria and to develop a novel image processing algorithm to reliably detect the presence of malaria parasite from Plasmodium falciparum species in thin smears of Giemsa stained peripheral blood sample. The algorithm was designed as an expert system based on the method used by medical practitioner performing microscopy diagnosis of malaria. Digital images were acquired using a digital camera connected to a light microscope. Prior to processing, the images were subjected to gray-scale conversion to decrease image variability. Global thresholding were implemented to obtain erythrocyte and other blood cell components in each image. The segmented images were further processed to obtain possibly infected erythrocyte and the components of parasite inside the corresponding erythrocyte using multiple threshold. These parasite's constituents (nucleus and cytoplasm) were used as the preliminary basis for parasite/non parasite classification. Malaria samples prepared and provided by Eijkman Institute of Molecular Biology Indonesia were used to test the proposed algorithm.

/pdf/automated-status-identification-of-microscopic-images-2sukp85ytn.pdf

Automated status identification of microscopic images obtained from malaria thin blood smears

Face Recognition begins with extracting the coordinates of features such as width of mouth, width of eyes, pupil, and compare the result with the measurements stored in the database and return the closest record (facial metrics). Nowadays, there are a lot of face recognition techniques and algorithms found and developed around the world. Facial recognition becomes an interesting research topic. It is proven by numerous number of published papers related with facial recognition including facial feature extraction, facial algorithm improvements, and facial recognition implementations. Main purposes of this research are to get the best facial recognition algorithm (Eigenface and Fisherface) provided by the Open CV 2.4.8 by comparing the ROC (Receiver Operating Characteristics) curve and implement it in the attendance system as the main case study. Based on the experiments, the ROC curve proves that using the current training set, Eigenface achieves better result than Fisherface. Eigenface implemented inside the Attendance System returns between 70% to 90% similarity for genuine face images.

Implementation of face recognition algorithm for biometrics based time attendance system

rising of computer violence, such as Distributed Denial of Service (DDoS), web vandalism, and cyber bullying are becoming more serious issues when they are politically motivated and intentionally conducted to generate fear in society. These kinds of activity are categorized as cyber terrorism. As the number of such cases increase, the availability of information regarding these actions is required to facilitate experts in investigating cyber terrorism. This research aims to create text classification system which classifies the document using several algorithms including Naive Bayes, Nearest Neighbor, Support Vector Machine (SVM), Decision Tree, and Multilayer Perceptron. The result shows that SVM outperforms by achieving 100% of accuracy. This result concludes the excellent performance of SVM in handling high dimensional of data.

Anto Satriyo Nugroho

Papers

Analysis of Machine learning Techniques Used in Behavior-Based Malware Detection

Traffic Condition Information Extraction & Visualization from Social Media Twitter for Android Mobile Application

Automated status identification of microscopic images obtained from malaria thin blood smears

Implementation of face recognition algorithm for biometrics based time attendance system

Text Classification Techniques Used to Faciliate Cyber Terrorism Investigation