Data Mining Practical Machine Learning Tools and Techniques

Real and Complex Analysis

Despite more than two decades of continuous development learning from imbalanced data is still a focus of intense research. Starting as a problem of skewed distributions of binary tasks, this topic evolved way beyond this conception. With the expansion of machine learning and data mining, combined with the arrival of big data era, we have gained a deeper insight into the nature of imbalanced learning, while at the same time facing new emerging challenges. Data-level and algorithm-level methods are constantly being improved and hybrid approaches gain increasing popularity. Recent trends focus on analyzing not only the disproportion between classes, but also other difficulties embedded in the nature of data. New real-life problems motivate researchers to focus on computationally efficient, adaptive and real-time methods. This paper aims at discussing open issues and challenges that need to be addressed to further develop the field of imbalanced learning. Seven vital areas of research in this topic are identified, covering the full spectrum of learning from imbalanced data: classification, regression, clustering, data streams, big data analytics and applications, e.g., in social media and computer vision. This paper provides a discussion and suggestions concerning lines of future research for each of them.

/pdf/learning-from-imbalanced-data-open-challenges-and-future-5d4mv5l28h.pdf

Learning from imbalanced data: open challenges and future directions

The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques, in order to help manage the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space. In this paper, we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-the-art in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.

/pdf/deep-learning-in-mobile-and-wireless-networking-a-survey-ct8cy2kuqi.pdf

Deep Learning in Mobile and Wireless Networking: A Survey

Recently, MOEA/D multi-objective evolutionary algorithm based on decomposition has achieved great success in the field of evolutionary multi-objective optimization and has attracted a lot of attention. It decomposes a multi-objective optimization problem MOP into a set of scalar subproblems using uniformly distributed aggregation weight vectors and provides an excellent general algorithmic framework of evolutionary multi-objective optimization. Generally, the uniformity of weight vectors in MOEA/D can ensure the diversity of the Pareto optimal solutions, however, it cannot work as well when the target MOP has a complex Pareto front PF; i.e., discontinuous PF or PF with sharp peak or low tail. To remedy this, we propose an improved MOEA/D with adaptive weight vector adjustment MOEA/D-AWA. According to the analysis of the geometric relationship between the weight vectors and the optimal solutions under the Chebyshev decomposition scheme, a new weight vector initialization method and an adaptive weight vector adjustment strategy are introduced in MOEA/D-AWA. The weights are adjusted periodically so that the weights of subproblems can be redistributed adaptively to obtain better uniformity of solutions. Meanwhile, computing efforts devoted to subproblems with duplicate optimal solution can be saved. Moreover, an external elite population is introduced to help adding new subproblems into real sparse regions rather than pseudo sparse regions of the complex PF, that is, discontinuous regions of the PF. MOEA/D-AWA has been compared with four state of the art MOEAs, namely the original MOEA/D, Adaptive-MOEA/D, -MOEA/D, and NSGA-II on 10 widely used test problems, two newly constructed complex problems, and two many-objective problems. Experimental results indicate that MOEA/D-AWA outperforms the benchmark algorithms in terms of the IGD metric, particularly when the PF of the MOP is complex.

/pdf/moea-d-with-adaptive-weight-adjustment-4jiz8yx52e.pdf

Moea/d with adaptive weight adjustment

Human mobility prediction is of great importance for a wide spectrum of location-based applications. However, predicting mobility is not trivial because of three challenges: 1) the complex sequential transition regularities exhibited with time-dependent and high-order nature; 2) the multi-level periodicity of human mobility; and 3) the heterogeneity and sparsity of the collected trajectory data. In this paper, we propose DeepMove, an attentional recurrent network for mobility prediction from lengthy and sparse trajectories. In DeepMove, we first design a multi-modal embedding recurrent neural network to capture the complicated sequential transitions by jointly embedding the multiple factors that govern the human mobility. Then, we propose a historical attention model with two mechanisms to capture the multi-level periodicity in a principle way, which effectively utilizes the periodicity nature to augment the recurrent neural network for mobility prediction. We perform experiments on three representative real-life mobility datasets, and extensive evaluation results demonstrate that our model outperforms the state-of-the-art models by more than 10%. Moreover, compared with the state-of-the-art neural network models, DeepMove provides intuitive explanations into the prediction and sheds light on interpretable mobility prediction.

DeepMove: Predicting Human Mobility with Attentional Recurrent Networks

Crowd flow prediction is of great importance in a wide range of applications from urban planning, traffic control to public safety. It aims to predict the inflow (the traffic of crowds entering a region in a given time interval) and outflow (the traffic of crowds leaving a region for other places) of each region in the city with knowing the historical flow data. In this paper, we propose DeepSTN+, a deep learning-based convolutional model, to predict crowd flows in the metropolis. First, DeepSTN+ employs the ConvPlus structure to model the longrange spatial dependence among crowd flows in different regions. Further, PoI distributions and time factor are combined to express the effect of location attributes to introduce prior knowledge of the crowd movements. Finally, we propose an effective fusion mechanism to stabilize the training process, which further improves the performance. Extensive experimental results based on two real-life datasets demonstrate the superiority of our model, i.e., DeepSTN+ reduces the error of the crowd flow prediction by approximately 8%∼13% compared with the state-of-the-art baselines.

/pdf/deepstn-context-aware-spatial-temporal-neural-network-for-yecd9r8bub.pdf

DeepSTN+: Context-Aware Spatial-Temporal Neural Network for Crowd Flow Prediction in Metropolis

Band selection is an important preprocessing step for hyperspectral data processing. It involves two crucial problems, i.e., suitable measure criterion and effective search strategy. Mutual information (MI) has been widely used as the measure criterion for its nonlinear and nonparametric characteristics. For efficient calculation, traditional MI-based criteria commonly use bivariate MI (BMI) to approximate the ideal MI-based criterion. However, these BMI-based criteria may miss the bands having discriminative information and do not give the condition of the approximation. In this paper, a novel criterion based on trivariate MI (TMI) is proposed to measure the redundancy for classification. From the multivariate MI perspective, the proposed TMI-based and traditional BMI-based criteria are proved as the low-order approximations of the ideal criterion under some assumptions. Compared with the BMI-based criteria, a more relaxed assumption condition is required for the TMI-based criterion. To alleviate the problem of few labeled samples existing in hyperspectral images, the TMI-based criterion is extended to the semisupervised TMI-based (STMI) method by adding a graph regulation term. Additionally, to search an appropriate band subset by the TMI- and STMI-based criteria, a new clonal selection algorithm (CSA) is proposed. In CSA, integer encoding and adaptive operators are devised to reduce space and time cost. Experimental results demonstrate the effectiveness of the proposed algorithms for hyperspectral band selection.

Hyperspectral Band Selection Based on Trivariate Mutual Information and Clonal Selection

Generative adversarial networks (GANs) are famous for generating samples by training a generator and a discriminator via an adversarial procedure. For hyperspectral image classification, the collection of samples is always difficult. However, directly applying GAN to hyperspectral image classification exists two problems. One is that the generated samples lack discriminative information. Meanwhile, the discriminator has no discriminative ability for multiclassification. Another is that spatial and spectral information requires to be considered in hyperspectral image classification simultaneously. To address these problems, a novel multiclass spatial–spectral GAN (MSGAN) method is proposed. In MSGAN, two generators are devised to generate the samples containing spatial and spectral information, respectively, and the discriminator is devised to extract joint spatial–spectral features and output multiclass probabilities. Moreover, novel adversarial objectives for multiclass are defined. The discriminator is devised to predict training samples belonging to true classes and generated samples belonging to all the classes with the same probability. The generators are devised to make the discriminator mistake. By adversarial learning between the discriminator and generators, the classification performance of the discriminator is promoted with the assistance of discriminative generated samples. Experimental results on hyperspectral images demonstrate that the proposed method achieves encouraging classification performance compared with several state-of-the-art methods, especially with the limited training samples.

Classification of Hyperspectral Images Based on Multiclass Spatial–Spectral Generative Adversarial Networks

With the popularity of mobile devices and location-based social network, understanding and modelling the human mobility becomes an important topic in the field of ubiquitous computing. With the model developing from personal models with own information to the joint models with population information, the prediction performance of proposed models become better and better. Meanwhile, the privacy issues of these models come into the view of community and the public: collecting and uploading private data to the centralized server without enough regulation. In this paper, we propose PMF, a privacy-preserving mobility prediction framework via federated learning, to solve this problem without significantly sacrificing the prediction performance. In our framework, based on the deep learning mobility model, no private data is uploaded into the centralized server and the only uploaded thing is the updated model parameters which are difficult to crack and thus more secure. Furthermore, we design a group optimization method for the training on local devices to achieve better trade-off between performance and privacy. Finally, we propose a fine-tuned personal adaptor for personal modelling to further improve the prediction performance. We conduct extensive experiments on three real-life mobility datasets to demonstrate the superiority and effectiveness of our methods in privacy protection settings.

Jie Feng

Papers

DeepMove: Predicting Human Mobility with Attentional Recurrent Networks

DeepSTN+: Context-Aware Spatial-Temporal Neural Network for Crowd Flow Prediction in Metropolis

Hyperspectral Band Selection Based on Trivariate Mutual Information and Clonal Selection

Classification of Hyperspectral Images Based on Multiclass Spatial–Spectral Generative Adversarial Networks

PMF: A Privacy-preserving Human Mobility Prediction Framework via Federated Learning