Data Mining - Concepts and Techniques.

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

Social systems are in a constant state of flux, with dynamics spanning from minute-by-minute changes to patterns present on the timescale of years. Accurate models of social dynamics are important for understanding the spreading of influence or diseases, formation of friendships, and the productivity of teams. Although there has been much progress on understanding complex networks over the past decade, little is known about the regularities governing the microdynamics of social networks. Here, we explore the dynamic social network of a densely-connected population of ∼1,000 individuals and their interactions in the network of real-world person-to-person proximity measured via Bluetooth, as well as their telecommunication networks, online social media contacts, geolocation, and demographic data. These high-resolution data allow us to observe social groups directly, rendering community detection unnecessary. Starting from 5-min time slices, we uncover dynamic social structures expressed on multiple timescales. On the hourly timescale, we find that gatherings are fluid, with members coming and going, but organized via a stable core of individuals. Each core represents a social context. Cores exhibit a pattern of recurring meetings across weeks and months, each with varying degrees of regularity. Taken together, these findings provide a powerful simplification of the social network, where cores represent fundamental structures expressed with strong temporal and spatial regularity. Using this framework, we explore the complex interplay between social and geospatial behavior, documenting how the formation of cores is preceded by coordination behavior in the communication networks and demonstrating that social behavior can be predicted with high precision.

/pdf/fundamental-structures-of-dynamic-social-networks-k2hteb4q5r.pdf

Fundamental structures of dynamic social networks

Mining GPS data for mobility patterns: A survey

https://www.vs.inf.ethz.ch/publ/papers/wilhelmk-predic-2014.pdf

Predicting household occupancy for smart heating control: A comparative performance analysis of state-of-the-art approaches

The ability to foresee the next moves of a user is crucial to ubiquitous computing. Disregarding major differences in individuals' routines, recent ground-breaking analysis on mobile phone data suggests high predictability in mobility. By nature, however, mobile phone data offer very low spatial and temporal resolutions. It remains largely unknown how the predictability changes with respect to different spatial/temporal scales. Using high-resolution GPS data, this paper investigates the scaling effects on predictability. Given specified spatial-temporal scales, recorded trajectories are encoded into long strings of distinct locations, and several information-theoretic measures of predictability are derived. Somewhat surprisingly, high predictability is still present at very high spatial/temporal resolutions. Moreover, the predictability is independent of the overall mobility area covered. This suggests highly regular mobility behaviors. Moreover, by varying the scales over a wide range, an invariance is observed which suggests that certain trade-offs between the predicting accuracy and spatial-temporal resolution are unavoidable. As many applications in ubiquitous computing concern mobility, these findings should have direct implications.

Predictability of individuals' mobility with high-resolution positioning data

Global positioning systems GPS logs recorded in personal devices contain rich information such as travel patterns, locations of frequent visits and place–event associations. There have been rather successful attempts in detecting the mode of transport from GPS logs such as walking, driving or taking a bus, which has found varied applications. However, the best-known schemes either require tedious manual labelling or pre-training process or both. We present MoDetect MD, a unsupervised scheme which eliminates the need of manual labelling and pre-training while attaining equal or greater accuracy compared with the best-known supervised methods. MD can also cater for differences in individual's behaviours, and hence may be more widely applicable than the existing schemes. To achieve this, MD relies on Kolmogorov–Smirnov test which offers a theoretical assurance when computing similarity between segments of records. Our analysis shows that the higher speed modes can be better differentiated through a weighted bootstrapping procedure. We also augment the decisions with reference to the transfer probabilities between different modes at locations identified from the GPS records.

Detecting modes of transport from unlabelled positioning sensor data

Biased random walk has been studied extensively over the past decade especially in the transport and communication networks communities. The mean first passage time (MFPT) of a biased random walk is an important performance indicator in those domains. While the fundamental matrix approach gives precise solution to MFPT, the computation is expensive and the solution lacks interpretability. Other approaches based on the Mean Field Theory relate MFPT to the node degree alone. However, nodes with the same degree may have very different local weight distribution, which may result in vastly different MFPT. We derive an approximate bound to the MFPT of biased random walk with short relaxation time on complex network where the biases are controlled by arbitrarily assigned node weights. We show that the MFPT of a node in this general case is closely related to not only its node degree, but also its local weight distribution. The MFPTs obtained from computer simulations also agree with the new theoretical analysis. Our result enables fast estimation of MFPT, which is useful especially to differentiate between nodes that have very different local node weight distribution even though they share the same node degrees.

https://dr.ntu.edu.sg/bitstream/10220/19549/1/Estimating%20Mean%20First%20Passage%20Time%20of%20Biased%20Random%20Walks%20with%20Short%20Relaxation%20Time%20on%20Complex%20Networks.pdf

Estimating mean first passage time of biased random walks with short relaxation time on complex networks.

Previous studies suggest that human mobility is highly regular in two respects. Firstly, individual's travels are governed by occasional exploration of new locations and preferential return to most frequently visited locations. Secondly, human mobility sequences exhibit high predictability. The existing model [1] is able to mimic exploration and preferential return, and lit actual mobility data. However, the high predictability issue is not addressed in this model. In this paper, we derive an upper bound of the predictability manifested by this model. Motivated by the incorrigible gap between the bound and the empirical results shown in [2], we further propose a new Markovian model by modifying the rule of preferential return to be conditional on the individuals' current location. We show both theoretically and empirically that the new Markovian model presents high predictability while preserving the desirable scaling properties of the original model in [1], making it the most complete model to date in capturing the essence of human mobility.

Modeling High Predictability and Scaling Laws of Human Mobility

The container stowage planning problem is known to be NP-hard and heuristic algorithms have been proposed. Conventionally, the efficiency of the stowage planning algorithms are improved by pruning or reducing the search space. We observe that constraint evaluation is the core of most algorithms. In addition, the order at which the constraints are evaluated can have significant impact on the efficiency of the constraint evaluation engine. We propose random sample model (RSM) and sequential sample model (SSM) for analysis of the problem. We present and evaluate seven strategies in optimizing the constraint evaluation engine. We show how to achieve the optimal constraint ordering with respect to RSM and SSM, respectively. However, the optimal ordering for SSM requires perfect information about the states of the constraint tests, which is impractical. We present an alternative strategy and show empirically that its efficiency is close to the optimal. Experiments show that, compared to a naive ordering, an average of 2.74 times speed up in the evaluation engine can be achieved. Note to Practitioners —Automated stowage planning has become a trend as the number of mega-scale containerships being deployed has increased drastically over the past decade. Due to the nature of the problem, it is impractical to expect the best stowage plan within a limited amount of time, and hence, many heuristics have been devised to reduce the solution space. Many heuristics algorithms share a common core-repeatedly selecting a stowage slot and a container (selection mechanism) and evaluate whether all of the constraints are satisfied. To improve the efficiency, most studies aim to reduce the number of location-container pairs being considered. We consider an alternative approach that improves the efficiency of the constraint evaluation engine. Specifically, we show how to improve the efficiency by strategically reordering the sequence in which the constraints are evaluated. With the improvements on the efficiency, the stowage algorithm is one step closer to being able to generate solutions in real time. This may enable the shipping companies to take last-minute shipment order and react to changes in demands quickly.

/pdf/a-near-optimal-algorithm-for-constraint-test-ordering-in-359bmf0l6n.pdf

Zhuo Qi Lee

Papers

Predictability of individuals' mobility with high-resolution positioning data

Detecting modes of transport from unlabelled positioning sensor data

Estimating mean first passage time of biased random walks with short relaxation time on complex networks.

Modeling High Predictability and Scaling Laws of Human Mobility

A Near-Optimal Algorithm for Constraint Test Ordering in Automated Stowage Planning