Data Mining - Concepts and Techniques.

Data Mining Practical Machine Learning Tools and Techniques

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

https://forum.stanford.edu/events/posterslides/DeepCompressionCompressingDeepNeuralNetworkswithPruning,TrainedQuantizationandHuffmanCoding.pdf

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32\(\times \) memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58\(\times \) faster convolutional operations (in terms of number of the high precision operations) and 32\(\times \) memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is the same as the full-precision AlexNet. We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than \(16\,\%\) in top-1 accuracy. Our code is available at: http://allenai.org/plato/xnornet.

/pdf/xnor-net-imagenet-classification-using-binary-convolutional-393ghc3cqd.pdf

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Android malware detection through generative adversarial networks

Requirement engineering is one of the most significant phases of software engineering. Success or failure of any software project relies heavily on better requirement engineering process. Better awareness of the requirements is fundamental for requirements engineering. Requirement Prioritization is an important component of requirement engineering process. In this paper, we have highlighted some serious shortcomings related to existing requirement prioritization techniques. Based on these findings, we have proposed an intelligent fuzzy logic based technique for requirements prioritization based on the perceived value of each requirement. We have also proposed a framework for evaluation of existing as well as proposed requirement prioritization techniques.

Value Based Fuzzy Requirement Prioritization and Its Evaluation Framework

Telecom companies are facing a serious problem of customer churn due to exponential growth in the use of telecommunication based services and the fierce competition in the market. Customer churns are the customers who decide to quit or switch use of the service or even company and join another competitor. This problem can affect the revenues and reputation of the telecom company in the business market. Therefore, many Customer Churn Prediction (CCP) models have been developed; however these models, mostly study in the context of within company CCP. Therefore, these models are not suitable for a situation where the company is newly established or have recently adopted the use of advanced technology or have lost the historical data relating to the customers. In such scenarios, Just-In-Time (JIT) approach can be a more practical alternative for CCP approach to address this issue in cross-company instead of within company churn prediction. This paper has proposed a JIT approach for CCP. However, JIT approach also needs some historical data to train the classifier. To cover this gap in this study, we built JIT-CCP model using Cross-company concept (i.e., when one company (source) data is used as training set and another company (target) data is considered for testing purpose). To support JIT-CCP, the cross-company data must be carefully transformed before being applied for classification. The objective of this paper is to provide an empirical comparison and effect of with and without state-of-the-art data transformation methods on the proposed JIT-CCP model. We perform experiments on publicly available benchmark datasets and utilize Naive Bayes as an underlying classifier. The results demonstrated that the data transformation methods improve the performance of the JIT-CCP significantly. Moreover, when using well-known data transformation methods, the proposed model outperforms the model learned by using without data transformation methods.

Just-in-time Customer Churn Prediction: With and Without Data Transformation

This study contributes to formalize customer churn prediction where rough set theory is used as one-class classifier and multi-class classifier to investigate the trade-off in the selection of an effective classification model for customer churn prediction. Experiments were performed to explore the performance of four different rule generation algorithms (i.e. exhaustive, genetic, covering and LEM2). It is observed that rough set as one-class classifier and multi-class classifier based on genetic algorithm yields more suitable performance as compared to the other three rule generation algorithms. Furthermore, by applying the proposed techniques (i.e. Rough sets as one-class and multi-class classifiers) on publicly available dataset, the results show that rough set as a multi - class classifier provides more accurate results for binary/multi-class classification problems.

Customer Churn Prediction in Telecommunication Industry: With and without Counter-Example

Due to the exponential growth in technologies and a greater number of competitors in the telecom sector, the companies are facing a rigorous problem of customer churns. The customer churn is a phenomenon that highlights the customer’s intention who may switch from a certain service or even the service provider company. Many customer churn prediction (CCP) techniques are developed by academics and practitioners to handle the customer churn in order to resolve the problems pertaining to customer retention. However, CCP is not widely studied in the scenario where the company is not having enough historical data due to either been a newly established company or due to the recent start of a new technology or even because of the loss of the historical data. The just-in-time (JIT) approach can be a more practical alternative to address this issue as compared to state-of-the-art CCP techniques. Unfortunately, similar to traditional churn prediction models, JIT also requires enough historical data. To address this gap in the traditional CCP models, this study uses the cross-company data, i.e., data from another company, in the context of JIT for addressing CCP problems in the telecom sector. We empirically evaluated the performance of the proposed model using publicly available datasets of two telecom companies. It is found from the empirical evaluation that in the JIT-CCP context: (i) it is possible to evaluate the performance of the predictive model using cross-company dataset for training purposes and (ii) it is evident that heterogeneous ensemble-based JIT-CCP model is more suitable approach to use as compared to individual classifier or homogeneous ensemble-based technique.

Sajid Anwar

Papers

Android malware detection through generative adversarial networks

Value Based Fuzzy Requirement Prioritization and Its Evaluation Framework

Just-in-time Customer Churn Prediction: With and Without Data Transformation

Customer Churn Prediction in Telecommunication Industry: With and without Counter-Example

Just-in-time customer churn prediction in the telecommunication sector