We explore the task of Android malware detection based on dynamic analysis of application activity sequences using deep learning techniques. We show that analyzing a sequence of the activities is informative for detecting malware, but that analyzing longer sequences does not necessarily lead to a more accurate model. In the real-world scenario, the number of malware is low compared to that of harmless applications. Our dataset has more than 180,000 samples, two-thirds of which are malware. This dataset is significantly larger than other datasets used in previous studies. We mimic real-world cases by randomly sampling a small portion of malware samples. Using the state-of-the-art model BERT, we show that it is possible to achieve desired malware detection performance with an extremely unbalanced dataset. We find that our BERT based model achieves an F1 score of 0.919 with just 0.5% of the examples being malware, which significantly outperforms current state-of-the-art approaches. The results validate the effectiveness of our proposed method in dealing with highly imbalanced datasets.

/pdf/malware-detection-on-highly-imbalanced-data-through-sequence-1g4k4k06z0.pdf

Malware Detection on Highly Imbalanced Data through Sequence Modeling

The evolution of recent malicious software with the rising use of digital services has increased the probability of corrupting data, stealing information, or other cybercrimes by malware attacks. Therefore, malicious software must be detected before it impacts a large number of computers. Recently, many malware detection solutions have been proposed by researchers. However, many challenges limit these solutions to effectively detecting several types of malware, especially zero-day attacks due to obfuscation and evasion techniques, as well as the diversity of malicious behavior caused by the rapid rate of new malware and malware variants being produced every day. Several review papers have explored the issues and challenges of malware detection from various viewpoints. However, there is a lack of a deep review article that associates each analysis and detection approach with the data type. Such an association is imperative for the research community as it helps to determine the suitable mitigation approach. In addition, the current survey articles stopped at a generic detection approach taxonomy. Moreover, some review papers presented the feature extraction methods as static, dynamic, and hybrid based on the utilized analysis approach and neglected the feature representation methods taxonomy, which is considered essential in developing the malware detection model. This survey bridges the gap by providing a comprehensive state-of-the-art review of malware detection model research. This survey introduces a feature representation taxonomy in addition to the deeper taxonomy of malware analysis and detection approaches and links each approach with the most commonly used data types. The feature extraction method is introduced according to the techniques used instead of the analysis approach. The survey ends with a discussion of the challenges and future research directions.

/pdf/malware-detection-issues-challenges-and-future-directions-a-31epk3c9.pdf

Malware Detection Issues, Challenges, and Future Directions: A Survey

Malware variants are the major emerging threats that face cybersecurity due to the potential damage to computer systems. Many solutions have been proposed for detecting malware variants. However, accurate detection is challenging due to the constantly evolving nature of the malware variants that cause concept drift. Existing malware detection solutions assume that the mapping learned from historical malware features will be valid for new and future malware. The relationship between input features and the class label has been considered stationary, which doesn’t hold for the ever-evolving nature of malware variants. Malware features change dynamically due to code obfuscations, mutations, and the modification made by malware authors to change the features’ distribution and thus evade the detection rendering the detection model obsolete and ineffective. This study presents an Adaptive behavioral-based Incremental Batch Learning Malware Variants Detection model using concept drift detection and sequential deep learning (AIBL-MVD) to accommodate the new malware variants. Malware behaviors were extracted using dynamic analysis by running the malware files in a sandbox environment and collecting their Application Programming Interface (API) traces. According to the malware first-time appearance, the malware samples were sorted to capture the malware variants’ change characteristics. The base classifier was then trained based on a subset of historical malware samples using a sequential deep learning model. The new malware samples were mixed with a subset of old data and gradually introduced to the learning model in an adaptive batch size incremental learning manner to address the catastrophic forgetting dilemma of incremental learning. The statistical process control technique has been used to detect the concept drift as an indication for incrementally updating the model as well as reducing the frequency of model updates. Results from extensive experiments show that the proposed model is superior in terms of detection rate and efficiency compared with the static model, periodic retraining approaches, and the fixed batch size incremental learning approach. The model maintains an average of 99.41% detection accuracy of new and variants malware with a low updating frequency of 1.35 times per month.

/pdf/an-adaptive-behavioral-based-incremental-batch-learning-1pjmar6m0a.pdf

An Adaptive Behavioral-Based Incremental Batch Learning Malware Variants Detection Model Using Concept Drift Detection and Sequential Deep Learning

With the increasing variants of malware, it is of great significance to detect malware and ensure system security effectively. The existing malware dynamic detection methods are vulnerable to evasion attacks. For this situation, we propose a malware dynamic detection method based on mufti-feature ensemble learning. Firstly, the method adopts the combination of software features such as API call sequence with high detection precision and low-level hardware features such as resistance to evasion the memory dump grayscale and hardware performance counters. Secondly, we improve each feature based on the original research. We select a more advanced classifier model to improve the detection precision of a single feature. Finally, an ensemble learning algorithm composed of multiple classification algorithms detects malware, the multi-features can describe malware behavior from multi-dimensions to improve detection performance. We use a large number of malware sample dataset to experiment, and the results show that our detection method can obtain good detection precision rate, and is better than other recently proposed dynamic detection methods in anti-evasion performance.

/pdf/smash-a-malware-detection-method-based-on-multi-feature-3nmguss8za.pdf

SMASH: A Malware Detection Method Based on Multi-Feature Ensemble Learning

Detection and mitigation of modern malware are critical for the normal operation of an organisation. Traditional defence mechanisms are becoming increasingly ineffective due to the techniques used by attackers such as code obfuscation, metamorphism, and polymorphism, which strengthen the resilience of malware. In this context, the development of adaptive, more effective malware detection methods has been identified as an urgent requirement for protecting the IT infrastructure against such threats, and for ensuring security. In this paper, we investigate an alternative method for malware detection that is based on N-grams and machine learning. We use a dynamic analysis technique to extract an Indicator of Compromise (IOC) for malicious files, which are represented using N-grams. The paper also proposes TF-IDF as a novel alternative used to identify the most significant N-grams features for training a machine learning algorithm. Finally, the paper evaluates the proposed technique using various supervised machine-learning algorithms. The results show that Logistic Regression, with a score of 98.4%, provides the best classification accuracy when compared to the other classifiers used.

/pdf/malgra-machine-learning-and-n-gram-malware-feature-3ld1jz7zyg.pdf

MALGRA: Machine Learning and N-Gram Malware Feature Extraction and Detection System

While increasing the threat of malware for information systems, researchers strive to find alternative malware detection methods based on static, dynamic and hybrid analysis. Due to obfuscation techniques to bypass the static analysis, dynamic methods become more useful to detect malware. Therefore, most of the researches focus on dynamic behavior analysis of malicious software. In this work, our main objective is to find more discriminative dynamic features to detect malware executables by analyzing different dynamic features with common malware detection approaches. Moreover, we analyze separately different features obtained in dynamic analysis, such as API-call, usage system library and operations, to observe the contributions of these features to malware detection and classification success. For this purpose, we evaluate the performance of some dynamic feature-based malware detection and classification approaches using four data sets that contain real and synthetic malware executables.

Analysis and Evaluation of Dynamic Feature-Based Malware Detection Methods

Metamorphic malware identification using engine-specific patterns based on co-opcode graphs

In this work, disassembler effect has been investigated on the detection performance of opcode-based malware detection methods. Opcode analysis is widely used to detect malware programs. Our experimental results presented which disassembler is more suitable with the analysis method for the best performance. Therefore, researchers will be guided with the results obtained in this work for their static opcode based PE file analysis.

Analysis and Comparison of Disassemblers for OpCode Based Malware Analysis

Malicious software (Malwares) become major threats for digital assets in the digital environment. Traditional malware detection systems use the signatures of the malware executables to detect them. However, the complexity and diversity of malwares increases day by day with metamorphic ones that quickly change its structure and signature. Therefore, most of the researches have focused on the detection of these kinds of malwares. In this work, five different malware detection approaches have been implemented and tested on real and synthetic malware and benign samples. We have collected a new malware data set including 6857 benign and 8701 malicious samples. Experiments have shown that the real malware executables decrease the performance of the methods.

Mert Nar

Papers

Analysis and Evaluation of Dynamic Feature-Based Malware Detection Methods

Metamorphic malware identification using engine-specific patterns based on co-opcode graphs

Analysis and Comparison of Disassemblers for OpCode Based Malware Analysis

Analysis and Comparison of Opcode-based Malware Detection Approaches