In the Internet age, malware (such as viruses, trojans, ransomware, and bots) has posed serious and evolving security threats to Internet users. To protect legitimate users from these threats, anti-malware software products from different companies, including Comodo, Kaspersky, Kingsoft, and Symantec, provide the major defense against malware. Unfortunately, driven by the economic benefits, the number of new malware samples has explosively increased: anti-malware vendors are now confronted with millions of potential malware samples per year. In order to keep on combating the increase in malware samples, there is an urgent need to develop intelligent methods for effective and efficient malware detection from the real and large daily sample collection. In this article, we first provide a brief overview on malware as well as the anti-malware industry, and present the industrial needs on malware detection. We then survey intelligent malware detection methods. In these methods, the process of detection is usually divided into two stages: feature extraction and classification/clustering. The performance of such intelligent malware detection approaches critically depend on the extracted features and the methods for classification/clustering. We provide a comprehensive investigation on both the feature extraction and the classification/clustering techniques. We also discuss the additional issues and the challenges of malware detection using data mining techniques and finally forecast the trends of malware development.

https://dl.acm.org/doi/pdf/10.1145/3073559

A Survey on Malware Detection Using Data Mining Techniques

App Store Analysis studies information about applications obtained from app stores. App stores provide a wealth of information derived from users that would not exist had the applications been distributed via previous software deployment methods. App Store Analysis combines this non-technical information with technical information to learn trends and behaviours within these forms of software repositories. Findings from App Store Analysis have a direct and actionable impact on the software teams that develop software for app stores, and have led to techniques for requirements engineering, release planning, software design, security and testing. This survey describes and compares the areas of research that have been explored thus far, drawing out common aspects, trends and directions future research should take to address open problems and challenges.

/pdf/a-survey-of-app-store-analysis-for-software-engineering-4mwd81w21w.pdf

A Survey of App Store Analysis for Software Engineering

Opcode sequences as representation of executables for data-mining-based unknown malware detection

Problem Given the number of times in which an unknown event has happened and failed: Required the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named. SECTION 1 Definition 1. Several events are inconsistent, when if one of them happens, none of the rest can. 2. Two events are contrary when one, or other of them must; and both together cannot happen. 3. An event is said to fail, when it cannot happen; or, which comes to the same thing, when its contrary has happened. 4. An event is said to be determined when it has either happened or failed. 5. The probability of any event is the ratio between the value at which an expectation depending on the happening of the event ought to be computed, and the value of the thing expected upon it’s 2 happening.

An essay towards solving a problem in the doctrine of chances. [Facsimil]

One of the major and serious threats on the
Internet today is malicious software, often referred to as a malware. The
malwares being designed by attackers are polymorphic and metamorphic which have
the ability to change their code as they propagate. Moreover, the diversity and
volume of their variants severely undermine the effectiveness of traditional
defenses which typically use signature based techniques and are unable to
detect the previously unknown malicious executables. The variants of malware
families share typical behavioral patterns reflecting their origin and purpose.
The behavioral patterns obtained either statically or dynamically can be
exploited to detect and classify unknown malwares into their known families
using machine learning techniques. This survey paper provides an overview of
techniques for analyzing and classifying the malwares.

/pdf/malware-analysis-and-classification-a-survey-390liif3we.pdf

Malware Analysis and Classification: A Survey

Malware is every malicious code that has the potential to harm any computer or network. The amount of malware is increasing faster every year and poses a serious security threat. Hence, malware detection has become a critical topic in computer security. Currently, signature-based detection is the most extended method within commercial antivirus. Although this method is still used on most popular commercial computer antivirus software, it can only achieve detection once the virus has already caused damage and it is registered. Therefore, it fails to detect new variations of known malware. In this paper, we propose a new method to detect variants of known malware families. This method is based on the frequency of appearance of opcode sequences. Furthermore, we describe a method to mine the relevance of each opcode and, thereby, weigh each opcode sequence frequency. We show that this method provides an effective way to detect variants of known malware families.

Idea: opcode-sequence-based malware detection

Malware is any computer software potentially harmful to both computers and networks. The amount of malware is growing every year and poses a serious global security threat. Signature-based detection is the most extended method in commercial antivirus software, however, it consistently fails to detect new malware. Supervised machine learning has been adopted to solve this issue. There are two types of features that supervised malware detectors use: (i) static features and (ii) dynamic features. Static features are extracted without executing the sample whereas dynamic ones requires an execution. Both approaches have their advantages and disadvantages. In this paper, we propose for the first time, OPEM, an hybrid unknown malware detector which combines the frequency of occurrence of operational codes (statically obtained) with the information of the execution trace of an executable (dynamically obtained). We show that this hybrid approach enhances the performance of both approaches when run separately.

OPEM: A Static-Dynamic Approach for Machine-Learning-Based Malware Detection

Malware is any kind of computer software potentially harmful to both computers and networks. The amount of malware is increasing every year and poses a serious global security threat. Signature-based detection is the most widely used commercial antivirus method, however, it consistently fails to detect new malware. Supervised machine-learning models have been used to solve this issue, but the usefulness of supervised learning is far to be perfect because it requires that a significant amount of malicious code and benign software to be identified and labelled beforehand. In this paper, we propose a new method of malware protection that adopts a semi-supervised learning approach to detect unknown malware. This method is designed to build a machine-learning classifier using a set of labelled (malware and legitimate software) and unlabelled instances.We performed an empirical validation demonstrating that the labelling efforts are lower than when supervised learning is used, while maintaining high accuracy rates.

Semi-supervised Learning for Unknown Malware Detection

The use of mobile phones has increased because they offer nearly the same functionality as a personal computer. In addition, the number of applications available for Android-based mobile devices has increased. Google offers programmers the opportunity to upload and sell applications in the Android Market, but malware writers upload their malicious code there. In light of this background, we present here manifest analysis for malware detection in Android MAMA, a new method that extracts several features from the Android manifest of the applications to build machine learning classifiers and detect malware.

Mama: manifest analysis for malware detection in android

The rise of the social web has brought a series of privacy concerns and threats. In particular, data leakage is a risk that affects the privacy of not only companies but individuals. Although there are tools that can prevent data losses, they require a prior step that involves the sensitive data to be properly identified. In this paper, we propose a new automatic approach that applies Named Entity Recognition (NER) to prevent data leaks. We conduct an empirical study with real-world data and show that this NER-based approach can enhance the prevention of data losses. In addition, we present and detail the implementation of a prototype built with these techniques and show how it can be used by both particulars and companies in order to handle data losses.

Javier Nieves

Papers

Idea: opcode-sequence-based malware detection

OPEM: A Static-Dynamic Approach for Machine-Learning-Based Malware Detection

Semi-supervised Learning for Unknown Malware Detection

Mama: manifest analysis for malware detection in android

Data Leak Prevention through Named Entity Recognition