Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

International Journal of Computer and Applications

In the Internet age, malware (such as viruses, trojans, ransomware, and bots) has posed serious and evolving security threats to Internet users. To protect legitimate users from these threats, anti-malware software products from different companies, including Comodo, Kaspersky, Kingsoft, and Symantec, provide the major defense against malware. Unfortunately, driven by the economic benefits, the number of new malware samples has explosively increased: anti-malware vendors are now confronted with millions of potential malware samples per year. In order to keep on combating the increase in malware samples, there is an urgent need to develop intelligent methods for effective and efficient malware detection from the real and large daily sample collection. In this article, we first provide a brief overview on malware as well as the anti-malware industry, and present the industrial needs on malware detection. We then survey intelligent malware detection methods. In these methods, the process of detection is usually divided into two stages: feature extraction and classification/clustering. The performance of such intelligent malware detection approaches critically depend on the extracted features and the methods for classification/clustering. We provide a comprehensive investigation on both the feature extraction and the classification/clustering techniques. We also discuss the additional issues and the challenges of malware detection using data mining techniques and finally forecast the trends of malware development.

https://dl.acm.org/doi/pdf/10.1145/3073559

A Survey on Malware Detection Using Data Mining Techniques

In this paper we introduce a deep neural network based malware detection system that Invincea has developed, which achieves a usable detection rate at an extremely low false positive rate and scales to real world training example volumes on commodity hardware. We show that our system achieves a 95% detection rate at 0.1% false positive rate (FPR), based on more than 400,000 software binaries sourced directly from our customers and internal malware databases. In addition, we describe a non-parametric method for adjusting the classifier’s scores to better represent expected precision in the deployment environment. Our results demonstrate that it is now feasible to quickly train and deploy a low resource, highly accurate machine learning classification model, with false positive rates that approach traditional labor intensive expert rule based malware detection, while also detecting previously unseen malware missed by these traditional approaches. Since machine learning models tend to improve with larger datasizes, we foresee deep neural network classification models gaining in importance as part of a layered network defense strategy in coming years.

Deep neural network based malware detection using two dimensional binary program features

Energy conservation is a primary concern in sensor networks. Several MAC protocols have been proposed to address this concern. However, the tradeoff between power consumption and latency has not been thoroughly studied. In this paper, we propose a sensor medium access control protocol with dynamic duty cycle, DSMAC, which achieves a good tradeoff between the two performance metrics without incurring much overhead. Moreover, DSMAC is able to adjust its duty cycle with varying traffic conditions without assuming any prior knowledge of application requirements. Both analytical and simulation results have been presented in this paper.

Medium access control with a dynamic duty cycle for sensor networks

One of the major and serious threats on the
Internet today is malicious software, often referred to as a malware. The
malwares being designed by attackers are polymorphic and metamorphic which have
the ability to change their code as they propagate. Moreover, the diversity and
volume of their variants severely undermine the effectiveness of traditional
defenses which typically use signature based techniques and are unable to
detect the previously unknown malicious executables. The variants of malware
families share typical behavioral patterns reflecting their origin and purpose.
The behavioral patterns obtained either statically or dynamically can be
exploited to detect and classify unknown malwares into their known families
using machine learning techniques. This survey paper provides an overview of
techniques for analyzing and classifying the malwares.

/pdf/malware-analysis-and-classification-a-survey-390liif3we.pdf

Malware Analysis and Classification: A Survey

We propose and study data scheduling and segmentation and reassembly (SAR) policies in Bluetooth. In such systems, the conventional scheduling policies such as round robin perform poorly as they are not suited to the tight coupling of uplink-downlink scheduling and result in slot wastage and unfairness. Scheduling in Bluetooth is complex due to (i) reserved slots for voice traffic, and, (ii) variable sized data packets. The reservation of voice slots at regular intervals results in non-contiguous TDD slots available for data. We propose two new scheduling policies that utilize information about the size of the head-of-the-line (HOL) packet at the master and slave queues to schedule the TDD slots effectively. These policies achieve high throughput and greater fairness compared to the round-robin based scheduling policies. We then study the SAR policies at the Bluetooth MAC. SAR policies have a significant effect on data scheduling as they govern the distribution of packet size. We propose two new SAR policies for Bluetooth that give good performance in terms of throughput, delay and fairness. Finally, we include channel errors in Bluetooth and propose a modified scheduling algorithm that gives good performance.

MAC scheduling and SAR policies for Bluetooth: a master driven TDD pico-cellular wireless system

Motivated by the emerging standards for indoor pico-cellular wireless systems, such as the Bluetooth, we propose and study the scheduling policies for master driven time division duplex (TDD) wireless networks. In these networks, the frequency band is divided into time slots, and each end (i.e., master or slave) takes turns in using the time slots. In Bluetooth, a slave transmits packets in the reverse slot only after the master polls the slave in a forward slot (by sending data to it). The conventional scheduling policies such as round robin do not perform well in these systems as they are not suited to the tight coupling of the uplink-downlink. We propose new scheduling policies, (i) the priority scheme, and, (ii) the K-fairness scheme that utilize the state at the master and slaves to schedule the TDD slots effectively. Active slaves are differentiated based upon the binary information (i.e., the presence or absence of packets in a slave queue) about the master-slave queue pairs. The priority scheme achieves high throughput as compared to the packet-by-packet generalized processor sharing (PGPS) based policies while guaranteeing a minimal service to each active slave while the K-fairness policy is characterized by a tight fairness bound as well as high system throughput. We then extend these policies for scheduling variable size data in the presence of voice. Further, since Bluetooth supports variable size data packets (1, 3 or 5 slots) on the same connections, the segmentation and reassembly (SAR) can significantly impact scheduling of data packets by varying packet size distribution. We propose an intelligent SAR policy (ISAR) and compare it with the naive random-SAR in which the data packet sizes (i.e., 1, 3 or 5) are assigned probabilistically. ISAR adapts MAC packet size at the master and slave queues depending on the data arrival rates at both the queues.

Divya Bansal

Papers

Malware Analysis and Classification: A Survey

MAC scheduling and SAR policies for Bluetooth: a master driven TDD pico-cellular wireless system

Data scheduling and SAR for Bluetooth MAC

Smart patrolling

A smartphone based technique to monitor driving behavior using DTW and crowdsensing