The massive growth in the scale of data has been observed in recent years being a key factor of the Big Data scenario. Big Data can be defined as high volume, velocity and variety of data that require a new high-performance processing. Addressing big data is a challenging and time-demanding task that requires a large computational infrastructure to ensure successful data processing and analysis. The presence of data preprocessing methods for data mining in big data is reviewed in this paper. The definition, characteristics, and categorization of data preprocessing approaches in big data are introduced. The connection between big data and data preprocessing throughout all families of methods and big data technologies are also examined, including a review of the state-of-the-art. In addition, research challenges are discussed, with focus on developments on different big data framework, such as Hadoop, Spark and Flink and the encouragement in devoting substantial research efforts in some families of data preprocessing methods and applications on new big data learning paradigms.

/pdf/big-data-preprocessing-methods-and-prospects-41ihs62og2.pdf

Big data preprocessing: methods and prospects

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of “quadratic-neuron-based deep learning”. Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretability. In this paper, we use quadratic neurons to construct an encoder-decoder structure, referred as the quadratic autoencoder, and apply it to low-dose CT denoising. The experimental results on the Mayo low-dose CT dataset demonstrate the utility and robustness of quadratic autoencoder in terms of image denoising and model efficiency. To our best knowledge, this is the first time that the deep learning approach is implemented with a new type of neurons and demonstrates a significant potential in the medical imaging field.

Quadratic Autoencoder (Q-AE) for Low-Dose CT Denoising

Data clustering is one of the most studied data mining tasks. It aims, through various methods, to discover previously unknown groups within the data sets. In the past years, considerable progress has been made in this field leading to the development of innovative and promising clustering algorithms. These traditional clustering algorithms present some serious issues in connection with the speed-up, the throughput, and the scalability. Thus, they can no longer be directly used in the context of Big Data, where data are mainly characterized by their volume, velocity, and variety. In order to overcome their limitations, the research today is heading to the parallel computing concept by giving rise to the so-called parallel clustering algorithms. This paper presents an overview of the latest parallel clustering algorithms categorized according to the computing platforms used to handle the Big Data, namely, the horizontal and vertical scaling platforms. The former category includes peer-to-peer networks, MapReduce, and Spark platforms, while the latter category includes Multi-core processors, Graphics Processing Unit, and Field Programmable Gate Arrays platforms. In addition, it includes a comparison of the performance of the reviewed algorithms based on some common criteria of clustering validation in the Big Data context. Therefore, it provides the reader with an overall vision of the current parallel clustering techniques.

A survey on parallel clustering algorithms for Big Data

In this paper, we propose a heuristic-based algorithm to improve the initial seeding of the k-means clustering algorithm. The proposed algorithm primarily aims to improve the initial choice of the centroids used by the k-means algorithm and also ensure that the requisite number of clusters is always returned in every run of the algorithm. Thus, the use of the proposed algorithm significantly reduces the possibility of k-means converging to a locally optimal solution. The paper explores the genetic algorithm framework to obtain the original seed points and couples this with the use of the differential evolution heuristic to obtain the requisite number of clusters. We have examined the performance of the proposed algorithm in the case of clustering text documents as such corpus often have significantly large number of data points and also require the formation of a large number of clusters. The results obtained have been compared with basic implementations of the k-means algorithm using standard parameters.

A hybrid approach using genetic algorithm and the differential evolution heuristic for enhanced initialization of the k-means algorithm with applications in text clustering

The complete binary tree as an important network structure has long been investigated for parallel and distributed computing, which has many nice properties and used to be embedded into other interconnection architectures. The parity cube is an important variant of the hypercube. It has many attractive features superior to those of the hypercube. In this paper, we prove that the complete binary tree with $$2^n-1$$ 2 n - 1 vertices can be embedded with dilation 1, congestion 1, load 1 into the $$n$$ n -dimensional parity cube $$PQ_n$$ P Q n and expansion tending to 1. Furthermore, we provide an $$O(NlogN)$$ O ( N l o g N ) algorithm to construct the complete binary tree with $$2^n-1$$ 2 n - 1 vertices in $$PQ_n$$ P Q n , where $$N$$ N denotes the number of vertices in $$PQ_n$$ P Q n and $$n\ge 1$$ n ? 1 .

Embedding complete binary trees into parity cubes

A distributed framework for trimmed Kernel k-Means clustering

In mathematical terms, an artificial neuron computes the inner product of a $d$ -dimensional input vector ${x}$ with its weight vector ${w}$ , compares it with a bias value $w_{0}$ and fires based on the result of this comparison. Therefore, its decision boundary is given by the equation ${w}^{T}{x}+w_{0}=0$ . In this paper, we propose replacing the linear hyperplane decision boundary of a neuron with a curved, paraboloid decision boundary. Thus, the decision boundary of the proposed paraboloid neuron is given by the equation $({h}^{T}{x}+h_{0})^{2}-||{x}-{p}||_{2}^{2}=0$ , where $h$ and $h_{0}$ denote the parameters of the directrix and $p$ denotes the coordinates of the focus. Such paraboloid neural networks are proven to have superior recognition accuracy in a number of applications.

Neurons With Paraboloid Decision Boundaries for Improved Neural Network Classification Performance

Data clustering is an unsupervised learning task that has found many applications in various scientific fields. The goal is to find subgroups of closely related data samples (clusters) in a set of unlabeled data. A classic clustering algorithm is the so-called k-Means. It is very popular, however, it is also unable to handle cases in which the clusters are not linearly separable. Kernel k-Means is a state of the art clustering algorithm, which employs the kernel trick, in order to perform clustering on a higher dimensionality space, thus overcoming the limitations of classic k-Means regarding the non linear separability of the input data. It has recently received a distributed implementation, named Trimmed Kernel k-Means, following the MapReduce distributed computing model. In addition to performing the computations in a distributed manner, Trimmed Kernel k-Means also trims the kernel matrix, in order to reduce the memory requirements and improve performance. The trimming of each row of the kernel matrix is achieved by attempting to estimate the cardinality of the cluster that the corresponding sample belongs to, and removing the kernel matrix entries connecting the sample to samples that probably belong to another cluster. The Spark cluster computing framework was used for the distributed implementation. In this paper, we present a distributed clustering scheme that is based on Trimmed Kernel k-Means, which employs subsampling, in order to be able to efficiently perform clustering on an extremely large dataset. The results indicate that the proposed method run much faster than the original Trimmed Kernel k-Means, while still providing clustering performance competitive with other state of the art kernel approaches.

Efficient MapReduce Kernel k-Means for Big Data Clustering

http://poseidon.csd.auth.gr/papers/PUBLISHED/JOURNAL/pdf/Tsapanos11a.pdf

Shape matching using a binary search tree structure of weak classifiers

Kernel k-Means is a basis for many state of the art global clustering approaches. When the number of samples grows too big, however, it is extremely time-consuming to compute the entire kernel matrix and it is impossible to store it in the memory of a single computer. The algorithm of Approximate Kernel k-Means has been proposed, which works using only a small part of the kernel matrix. The computation of the kernel matrix, even a part of it, remains a significant bottleneck of the process. Some types of kernel, however, can be computed using matrix multiplication. Modern CPU architectures and computational optimization methods allow for very fast matrix multiplication, thus those types of kernel matrices can be computed much faster than others.

Nikolaos Tsapanos

Papers

A distributed framework for trimmed Kernel k-Means clustering

Neurons With Paraboloid Decision Boundaries for Improved Neural Network Classification Performance

Efficient MapReduce Kernel k-Means for Big Data Clustering

Shape matching using a binary search tree structure of weak classifiers

Fast Kernel Matrix Computation for Big Data Clustering