Deep learning applications and challenges in big data analytics
Maryam M. Najafabadi,Flavio Villanustre,Taghi M. Khoshgoftaar,Naeem Seliya,Randall Wald,Edin Muharemagic +5 more
Reads0
Chats0
TLDR
This study explores how Deep Learning can be utilized for addressing some important problems in Big Data Analytics, including extracting complex patterns from massive volumes of data, semantic indexing, data tagging, fast information retrieval, and simplifying discriminative tasks.Abstract:
Big Data Analytics and Deep Learning are two high-focus of data science. Big Data has become important as many organizations both public and private have been collecting massive amounts of domain-specific information, which can contain useful information about problems such as national intelligence, cyber security, fraud detection, marketing, and medical informatics. Companies such as Google and Microsoft are analyzing large volumes of data for business analysis and decisions, impacting existing and future technology. Deep Learning algorithms extract high-level, complex abstractions as data representations through a hierarchical learning process. Complex abstractions are learnt at a given level based on relatively simpler abstractions formulated in the preceding level in the hierarchy. A key benefit of Deep Learning is the analysis and learning of massive amounts of unsupervised data, making it a valuable tool for Big Data Analytics where raw data is largely unlabeled and un-categorized. In the present study, we explore how Deep Learning can be utilized for addressing some important problems in Big Data Analytics, including extracting complex patterns from massive volumes of data, semantic indexing, data tagging, fast information retrieval, and simplifying discriminative tasks. We also investigate some aspects of Deep Learning research that need further exploration to incorporate specific challenges introduced by Big Data Analytics, including streaming data, high-dimensional data, scalability of models, and distributed computing. We conclude by presenting insights into relevant future works by posing some questions, including defining data sampling criteria, domain adaptation modeling, defining criteria for obtaining useful data abstractions, improving semantic indexing, semi-supervised learning, and active learning.read more
Citations
More filters
Proceedings ArticleDOI
Big Data Mining in the Presence of Concept Drifting
Terbium Siraj,Efrana Jannat,Warafta Rasul,Meher Afroz Chowdhury,Swakkhar Shatabda,Dewan Md. Farid +5 more
TL;DR: The calculated result proves proposed approach as an effective and efficient means of detecting novel class, which implies concept drift identification at the same time.
Deep Learning Method Vs. Hand-Crafted Features For Lung Cancer Diagnosis And Breast Cancer Risk Analysis
TL;DR: This paper presents a meta-analyses of breast cancer risk analysis and lung cancer diagnosis using a convolutional neural network and deep learning methodology.
Posted Content
A Study of Deep Feature Fusion based Methods for Classifying Multi-lead ECG.
TL;DR: A Multi-branch Convolution and Residual Network (MBCRNet) with three kinds of feature fusion methods for automatic detection of normal and abnormal ECG signals that outperforms previous methods under the same database.
Proceedings ArticleDOI
Class-specific pre-trained sparse autoencoders for learning effective features for document classification
Maysa I Abdulhussain,John Q. Gan +1 more
TL;DR: Experimental results have demonstrated the advantages and usefulness of the proposed class-specific pre-trained approach based on sparse autoencoder in document classification in high-dimensional feature space, in terms of the limited number of features required to achieve good classification accuracy.
Journal ArticleDOI
Big data investment and knowledge integration in academic libraries
TL;DR: This study investigates the investment of Hadoop in managing the academic libraries’ big data, focusing on three Vs (velocity, variety, and volume), and shows how Hadoops framework using the map/reduce technique can be used to manage the big data of such libraries.
References
More filters
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Proceedings ArticleDOI
Histograms of oriented gradients for human detection
Navneet Dalal,Bill Triggs +1 more
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.
Posted Content
Efficient Estimation of Word Representations in Vector Space
TL;DR: This paper proposed two novel model architectures for computing continuous vector representations of words from very large data sets, and the quality of these representations is measured in a word similarity task and the results are compared to the previously best performing techniques based on different types of neural networks.
Proceedings ArticleDOI
Object recognition from local scale-invariant features
TL;DR: Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
Journal ArticleDOI
Reducing the Dimensionality of Data with Neural Networks
TL;DR: In this article, an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data is described.