What is Twitter

The all-pairs-similarity-search (or similarity join) problem has been extensively studied for text and a handful of other datatypes. However, surprisingly little progress has been made on similarity joins for time series subsequences. The lack of progress probably stems from the daunting nature of the problem. For even modest sized datasets the obvious nested-loop algorithm can take months, and the typical speed-up techniques in this domain (i.e., indexing, lower-bounding, triangular-inequality pruning and early abandoning) at best produce one or two orders of magnitude speedup. In this work we introduce a novel scalable algorithm for time series subsequence all-pairs-similarity-search. For exceptionally large datasets, the algorithm can be trivially cast as an anytime algorithm and produce high-quality approximate solutions in reasonable time. The exact similarity join algorithm computes the answer to the time series motif and time series discord problem as a side-effect, and our algorithm incidentally provides the fastest known algorithm for both these extensively-studied problems. We demonstrate the utility of our ideas for two time series data mining problems, including motif discovery and novelty discovery.

Matrix Profile I: All Pairs Similarity Joins for Time Series: A Unifying View That Includes Motifs, Discords and Shapelets

Recently, Cloud-based Mobile Augmentation (CMA) approaches have gained remarkable ground from academia and industry. CMA is the state-of-the-art mobile augmentation model that employs resource-rich clouds to increase, enhance, and optimize computing capabilities of mobile devices aiming at execution of resource-intensive mobile applications. Augmented mobile devices envision to perform extensive computations and to store big data beyond their intrinsic capabilities with least footprint and vulnerability. Researchers utilize varied cloud-based computing resources (e.g., distant clouds and nearby mobile nodes) to meet various computing requirements of mobile users. However, employing cloud-based computing resources is not a straightforward panacea. Comprehending critical factors (e.g., current state of mobile client and remote resources) that impact on augmentation process and optimum selection of cloud-based resource types are some challenges that hinder CMA adaptability. This paper comprehensively surveys the mobile augmentation domain and presents taxonomy of CMA approaches. The objectives of this study is to highlight the effects of remote resources on the quality and reliability of augmentation processes and discuss the challenges and opportunities of employing varied cloud-based resources in augmenting mobile devices. We present augmentation definition, motivation, and taxonomy of augmentation types, including traditional and cloud-based. We critically analyze the state-of-the-art CMA approaches and classify them into four groups of distant fixed, proximate fixed, proximate mobile, and hybrid to present a taxonomy. Vital decision making and performance limitation factors that influence on the adoption of CMA approaches are introduced and an exemplary decision making flowchart for future CMA approaches are presented. Impacts of CMA approaches on mobile computing is discussed and open challenges are presented as the future research directions.

Cloud-Based Augmentation for Mobile Devices: Motivation, Taxonomies, and Open Challenges

In this work, we present an interactive system for visual analysis of urban traffic congestion based on GPS trajectories. For these trajectories we develop strategies to extract and derive traffic jam information. After cleaning the trajectories, they are matched to a road network. Subsequently, traffic speed on each road segment is computed and traffic jam events are automatically detected. Spatially and temporally related events are concatenated in, so-called, traffic jam propagation graphs. These graphs form a high-level description of a traffic jam and its propagation in time and space. Our system provides multiple views for visually exploring and analyzing the traffic condition of a large city as a whole, on the level of propagation graphs, and on road segment level. Case studies with 24 days of taxi GPS trajectories collected in Beijing demonstrate the effectiveness of our system.

/pdf/visual-traffic-jam-analysis-based-on-trajectory-data-57g67pjy84.pdf

Visual Traffic Jam Analysis Based on Trajectory Data

The change in the DNA is a form of genetic variation in the human genome. In addition, the DNA copy number change is also linked with the progression of many emerging diseases. Array-based Comparative Genomic Hybridization (CGH) is considered as a major task when measuring the DNA copy number change across the genome. Moreover, DNA copy number change is an essential measure to diagnose the cancer disease. Next generation sequencing is an important method for studying the spread of infectious disease qualitatively and quantitatively. CGH is widely used in continuous monitoring of copy number of thousands of genes throughout the genome. In recent years, the size of the DNA sequence data is very large. Hence, there is a need to use a scalable machine learning approach to overcome the various issues in DNA copy number change detection. In this paper, we use a Bayesian hidden Markov model (HMM) with Gaussian Mixture (GM) Clustering approach to model the DNA copy number change across the genome. The proposed Bayesian HMM with GM Clustering approach is compared with various existing approaches such as Pruned Exact Linear Time method, binary segmentation method and segment neighborhood method. Experimental results demonstrate the effectiveness of our proposed change detection algorithm.

Machine Learning Based Big Data Processing Framework for Cancer Diagnosis Using Hidden Markov Model and GM Clustering

Data clustering is an important data mining technology that plays a crucial role in numerous scientific applications. However, it is challenging due to the size of datasets has been growing rapidly to extra-large scale in the real world. Meanwhile, MapReduce is a desirable parallel programming platform that is widely applied in kinds of data process fields. In this paper, we propose an efficient parallel density-based clustering algorithm and implement it by a 4-stages MapReduce paradigm. Furthermore, we adopt a quick partitioning strategy for large scale non-indexed data. We study the metric of merge among bordering partitions and make optimizations on it. At last, we evaluate our work on real large scale datasets using Hadoop platform. Results reveal that the speedup and scale up of our work are very efficient.

/pdf/mr-dbscan-an-efficient-parallel-density-based-clustering-1fspv4fkp2.pdf

MR-DBSCAN: An Efficient Parallel Density-Based Clustering Algorithm Using MapReduce

Wukong: A cloud-oriented file service for mobile Internet devices

High-dimensional similarity join (HDSJ) is critical for many novel applications in the domain of mobile data management Nowadays, performing HDSJs efficiently faces two challenges First, the scale of datasets is increasing rapidly, making parallel computing on a scalable platform a must Second, the dimensionality of the data can be up to hundreds or even thousands, which brings about the issue of dimensionality curse In this paper, we address these challenges and study how to perform parallel HDSJs efficiently in the MapReduce paradigm Particularly, we propose a cost model to demonstrate that it is important to take both communication and computation costs into account as dimensionality and data volume increases To this end, we propose DAA (Dimension Aggregation Approximation), an efficient compression approach that can help significantly reduce both these costs when performing parallel HDSJs Moreover, we design DAA-based parallel HDSJ algorithms which can scale up to massive data sizes and very high dimensionality We perform extensive experiments using both synthetic and real datasets to evaluate the speedup and the scale up of our algorithms

Efficient Similarity Joins on Massive High-Dimensional Datasets Using MapReduce

As more and more individuals have multiple computing devices, how to maintain the consistency of user files stored on multiple devices becomes a challenge. Cloud-based file synchronization services (FSS), e.g., Drop box and iFolder, are promising approaches. However, we haven't seen any public documents discussing the consistency protocol, which plays a vital role in any FSS. This paper focuses on maintaining the consistency of user files in FSS. We define a new file abstraction layer called user view, which describes all the files and their metadata in a sync-folder. We propose a user view based file consistency protocol with a conflicts resolution mechanism for FSS. The consistency protocol automatically keeps user files synchronized among the clients and server. We implement our consistency protocol into a prototype called Sync Views and set up a LAN-based environment to evaluate its feasibility and performance. We measured the time consumption of each synchronous operation on Sync Views and compared with iFolder, an open source implementation. The results show that our protocol works well during the file synchronization processes and outperforms iFolder when the synchronized file becomes very large.

SyncViews: Toward Consistent User Views in Cloud-Based File Synchronization Services

As a leading cause of severe disability and death, stroke places an enormous burden on Chinese society. A nationwide stroke screening platform called CSDC (China Stoke Data Center) has been built to support the national stroke prevention program and stroke clinical research since 2011. This platform is composed of a data integration system and a big data analysis system. The data integration system is used to collect information on risk factors, diagnosis history, treatment, and sociodemographic characteristics and stroke patients' EMR. The big data analysis system support decision making of stroke control and prevention, clinical evaluation and research. In this paper, the design and implementation of CSDC are illustrated, and some application results are presented. This platform is expected to provide rich data and powerful tool support for stroke control and prevention in China.

Huajian Mao

Papers

MR-DBSCAN: An Efficient Parallel Density-Based Clustering Algorithm Using MapReduce

Wukong: A cloud-oriented file service for mobile Internet devices

Efficient Similarity Joins on Massive High-Dimensional Datasets Using MapReduce

SyncViews: Toward Consistent User Views in Cloud-Based File Synchronization Services

CSDC — A nationwide screening platform for stroke control and prevention in China