Data Mining - Concepts and Techniques.

Data Mining Practical Machine Learning Tools and Techniques

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

https://forum.stanford.edu/events/posterslides/DeepCompressionCompressingDeepNeuralNetworkswithPruning,TrainedQuantizationandHuffmanCoding.pdf

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32\(\times \) memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58\(\times \) faster convolutional operations (in terms of number of the high precision operations) and 32\(\times \) memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is the same as the full-precision AlexNet. We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than \(16\,\%\) in top-1 accuracy. Our code is available at: http://allenai.org/plato/xnornet.

/pdf/xnor-net-imagenet-classification-using-binary-convolutional-393ghc3cqd.pdf

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Software component selection is the most important part of component based software development. A large amount of time is invested in searching and selecting the most appropriate component from component repository. Different methods are used to select components quickly and efficiently. In the proposed method we have used part of off the shelf option and fuzzy logic methodology for components selection. The proposed methodology incorporates several important factors such as efficiency, reusability, portability, functionality, security, testability and maintenance. The methodology is illustrated and evaluated using hypothetical case study.

A Novel Fuzzy Logic Based Software Component Selection Modeling

The rapidly rising usage of telecommunication and information networks which inter-connect modern society through computers, smart phones and other electronic devices has led to security threats and cyber-crimes (CC) activities. These cybercrime activities has ultimately resulted in CC attack classification as a serious problem in network security domain while machine learning has been subjected to extensive research area in intrusion classification with emphasis on improving the rate of classifier's accuracy or improving the data mining model performance. This study is another attempt, using rough set theory (RST), a rule based decision making approach to extract rules for intrusion attacks classification. Experiments were performed on publicly available data to explore the performance of four different algorithms e.g. genetic algorithm, covering algorithm, LEM2 and Exhaustive algorithms. It is observed that RST classification based on genetic algorithm for rules generation yields best performance as compared to other mentioned rules generation algorithms. Moreover, by applying the proposed technique on publicly available dataset about intrusion attacks, the results show that the proposed approach can fully predict all intrusion attacks and also provides prior useful information to the security engineers or developers to conduct a mandating action.

Classification of cyber attacks based on rough set theory

This study contributes to formalize a three phase customer churn prediction technique. In the first phase, a supervised feature selection procedure is adopted to select the most relevant subset of features by laying-off the redundancy and increasing the relevance that leads to reduced and highly correlated features set. In the second phase, a knowledge based system (KBS) is built through Ripple Down Rule (RDR) learner which acquires knowledge about seen customer churn behavior and handles the problem of brittle in churn KBS through prudence analysis that will issue a prompt to the decision maker whenever a case is beyond the maintained knowledge in the knowledge database. In the final phase, a technique for Simulated Expert (SE) is proposed to evaluate the Knowledge Acquisition (KA) in KB system. Moreover, by applying the proposed approach on publicly available dataset, the results show that the proposed approach can be a worthy alternate for churn prediction in telecommunication industry.

A Prudent Based Approach for Customer Churn Prediction

Software maintenance is considered one of the most important issues in software engineering which has some serious implications in term of cost and effort. It consumes enormous amount of organization's overall resources. On the other hand, software architecture of an application has considerable effect on quality factors such as maintainability, performance, reliability and flexibility etc. Using software architecture for quantification of certain quality factor will help organizations to plan resources accordingly. This paper is an attempt to predict software maintenance effort at architecture level. The method takes requirements, domain knowledge and general software engineering knowledge as input in order to prescribe application architecture. Once application architecture is prescribed, then weighted scenarios and certain factors (i.e. system novelty, turnover and maintenance staff ability, documentation quality, testing quality etc) that affect software maintenance are applied to application architecture to quantify maintenance effort. The technique is illustrated and evaluated using web content extraction application architecture.

Software Maintenance Prediction Using Weighted Scenarios: An Architecture Perspective

Malware analysis and detection over the Android have been the focus of considerable research, during recent years, as customer adoption of Android attracted a corresponding number of malware writers. Antivirus companies commonly rely on signatures and are error‐prone. Traditional machine learning techniques are based on static, dynamic, and hybrid analysis; however, for large scale Android malware analysis, these approaches are not feasible. Deep neural architectures are able to analyze large scale static details of the applications, but static analysis techniques can ignore many malicious behaviors of applications. The study contributes to the documentation of various approaches for detection of malware, traditional and state‐of‐the‐art models, developed for analysis that facilitates the provision of basic insights for researchers working in malware analysis, and the study also provides a dynamic approach that employs deep neural network models for detection of malware. Moreover, the study uses Android permissions as a parameter to measure the dynamic behavior of around 16,900 benign and intruded applications. A dataset is created which encompasses a large set of permissions‐based dynamic behavior pertaining applications, with an aim to train deep learning models for prediction of behavior. The proposed architecture extracts representations from input sequence data with no human intervention. The state‐of‐the‐art Deep Convolutional Generative Adversarial Network extracted deep features and accomplished a general validation accuracy of 97.08% with an F1‐score of 0.973 in correctly classifying input. Furthermore, the concept of blockchain is utilized to preserve the integrity of the dataset and the results of the analysis.

Sajid Anwar

Papers

A Novel Fuzzy Logic Based Software Component Selection Modeling

Classification of cyber attacks based on rough set theory

A Prudent Based Approach for Customer Churn Prediction

Software Maintenance Prediction Using Weighted Scenarios: An Architecture Perspective

Integrity verification and behavioral classification of a large dataset applications pertaining smart OS via blockchain and generative models