Being able to identify the groups of clients that are responsible for a significant portion of a Web site's requests can be helpful to both the Web site and the clients. In a Web application, it is beneficial to move content closer to groups of clients that are responsible for large subsets of requests to an origin server. We introduce clusters---a grouping of clients that are close together topologically and likely to be under common administrative control. We identify clusters using a ``network-aware" method, based on information available from BGP routing table snapshots.

On Network-Aware Clustering of Web Clients

We explore the task of Android malware detection based on dynamic analysis of application activity sequences using deep learning techniques. We show that analyzing a sequence of the activities is informative for detecting malware, but that analyzing longer sequences does not necessarily lead to a more accurate model. In the real-world scenario, the number of malware is low compared to that of harmless applications. Our dataset has more than 180,000 samples, two-thirds of which are malware. This dataset is significantly larger than other datasets used in previous studies. We mimic real-world cases by randomly sampling a small portion of malware samples. Using the state-of-the-art model BERT, we show that it is possible to achieve desired malware detection performance with an extremely unbalanced dataset. We find that our BERT based model achieves an F1 score of 0.919 with just 0.5% of the examples being malware, which significantly outperforms current state-of-the-art approaches. The results validate the effectiveness of our proposed method in dealing with highly imbalanced datasets.

/pdf/malware-detection-on-highly-imbalanced-data-through-sequence-1g4k4k06z0.pdf

Malware Detection on Highly Imbalanced Data through Sequence Modeling

The evolution of recent malicious software with the rising use of digital services has increased the probability of corrupting data, stealing information, or other cybercrimes by malware attacks. Therefore, malicious software must be detected before it impacts a large number of computers. Recently, many malware detection solutions have been proposed by researchers. However, many challenges limit these solutions to effectively detecting several types of malware, especially zero-day attacks due to obfuscation and evasion techniques, as well as the diversity of malicious behavior caused by the rapid rate of new malware and malware variants being produced every day. Several review papers have explored the issues and challenges of malware detection from various viewpoints. However, there is a lack of a deep review article that associates each analysis and detection approach with the data type. Such an association is imperative for the research community as it helps to determine the suitable mitigation approach. In addition, the current survey articles stopped at a generic detection approach taxonomy. Moreover, some review papers presented the feature extraction methods as static, dynamic, and hybrid based on the utilized analysis approach and neglected the feature representation methods taxonomy, which is considered essential in developing the malware detection model. This survey bridges the gap by providing a comprehensive state-of-the-art review of malware detection model research. This survey introduces a feature representation taxonomy in addition to the deeper taxonomy of malware analysis and detection approaches and links each approach with the most commonly used data types. The feature extraction method is introduced according to the techniques used instead of the analysis approach. The survey ends with a discussion of the challenges and future research directions.

/pdf/malware-detection-issues-challenges-and-future-directions-a-31epk3c9.pdf

Malware Detection Issues, Challenges, and Future Directions: A Survey

Malware variants are the major emerging threats that face cybersecurity due to the potential damage to computer systems. Many solutions have been proposed for detecting malware variants. However, accurate detection is challenging due to the constantly evolving nature of the malware variants that cause concept drift. Existing malware detection solutions assume that the mapping learned from historical malware features will be valid for new and future malware. The relationship between input features and the class label has been considered stationary, which doesn’t hold for the ever-evolving nature of malware variants. Malware features change dynamically due to code obfuscations, mutations, and the modification made by malware authors to change the features’ distribution and thus evade the detection rendering the detection model obsolete and ineffective. This study presents an Adaptive behavioral-based Incremental Batch Learning Malware Variants Detection model using concept drift detection and sequential deep learning (AIBL-MVD) to accommodate the new malware variants. Malware behaviors were extracted using dynamic analysis by running the malware files in a sandbox environment and collecting their Application Programming Interface (API) traces. According to the malware first-time appearance, the malware samples were sorted to capture the malware variants’ change characteristics. The base classifier was then trained based on a subset of historical malware samples using a sequential deep learning model. The new malware samples were mixed with a subset of old data and gradually introduced to the learning model in an adaptive batch size incremental learning manner to address the catastrophic forgetting dilemma of incremental learning. The statistical process control technique has been used to detect the concept drift as an indication for incrementally updating the model as well as reducing the frequency of model updates. Results from extensive experiments show that the proposed model is superior in terms of detection rate and efficiency compared with the static model, periodic retraining approaches, and the fixed batch size incremental learning approach. The model maintains an average of 99.41% detection accuracy of new and variants malware with a low updating frequency of 1.35 times per month.

/pdf/an-adaptive-behavioral-based-incremental-batch-learning-1pjmar6m0a.pdf

An Adaptive Behavioral-Based Incremental Batch Learning Malware Variants Detection Model Using Concept Drift Detection and Sequential Deep Learning

With the increasing variants of malware, it is of great significance to detect malware and ensure system security effectively. The existing malware dynamic detection methods are vulnerable to evasion attacks. For this situation, we propose a malware dynamic detection method based on mufti-feature ensemble learning. Firstly, the method adopts the combination of software features such as API call sequence with high detection precision and low-level hardware features such as resistance to evasion the memory dump grayscale and hardware performance counters. Secondly, we improve each feature based on the original research. We select a more advanced classifier model to improve the detection precision of a single feature. Finally, an ensemble learning algorithm composed of multiple classification algorithms detects malware, the multi-features can describe malware behavior from multi-dimensions to improve detection performance. We use a large number of malware sample dataset to experiment, and the results show that our detection method can obtain good detection precision rate, and is better than other recently proposed dynamic detection methods in anti-evasion performance.

/pdf/smash-a-malware-detection-method-based-on-multi-feature-3nmguss8za.pdf

SMASH: A Malware Detection Method Based on Multi-Feature Ensemble Learning

While increasing the threat of malware for information systems, researchers strive to find alternative malware detection methods based on static, dynamic and hybrid analysis. Due to obfuscation techniques to bypass the static analysis, dynamic methods become more useful to detect malware. Therefore, most of the researches focus on dynamic behavior analysis of malicious software. In this work, our main objective is to find more discriminative dynamic features to detect malware executables by analyzing different dynamic features with common malware detection approaches. Moreover, we analyze separately different features obtained in dynamic analysis, such as API-call, usage system library and operations, to observe the contributions of these features to malware detection and classification success. For this purpose, we evaluate the performance of some dynamic feature-based malware detection and classification approaches using four data sets that contain real and synthetic malware executables.

Analysis and Evaluation of Dynamic Feature-Based Malware Detection Methods

Metamorphic malware identification using engine-specific patterns based on co-opcode graphs

In this work, disassembler effect has been investigated on the detection performance of opcode-based malware detection methods. Opcode analysis is widely used to detect malware programs. Our experimental results presented which disassembler is more suitable with the analysis method for the best performance. Therefore, researchers will be guided with the results obtained in this work for their static opcode based PE file analysis.

Analysis and Comparison of Disassemblers for OpCode Based Malware Analysis

Sequential opcode embedding-based malware detection method

Networked data is data composed of network objects and links. Network objects are characterized by high dimensional attributes and by links indicating the relationships among these objects. However, traditional feature selection and feature extraction methods consider only attribute information, thus ignoring link information. In the presented work, we propose a new unsupervised binary feature construction method (NetBFC) for networked data that reconstructs attributes for each object by exploiting link information. By exploring similar objects in the network and associating them, our method increases the similarities between objects with high probability of being in the same group. The proposed method enables local attribute enrichment and local attribute selection for each object by aggregating the attributes of similar objects in order to deal with the sparsity of networked data. In addition, this method applies an attribute elimination phase to eliminate irrelevant and redundant attributes which decrease the performance of clustering algorithms. Experimental results on real-world data sets indicate that NetBFC significantly achieves better performance when compared to baseline methods.

Arzu Gorgulu Kakisim

Papers

Analysis and Evaluation of Dynamic Feature-Based Malware Detection Methods

Metamorphic malware identification using engine-specific patterns based on co-opcode graphs

Analysis and Comparison of Disassemblers for OpCode Based Malware Analysis

Sequential opcode embedding-based malware detection method

Unsupervised binary feature construction method for networked data