Showing papers on "Dynamic time warping published in 2015"

Time series classification with ensembles of elastic distance measures

Abstract: Several alternative distance measures for comparing time series have recently been proposed and evaluated on time series classification (TSC) problems. These include variants of dynamic time warping (DTW), such as weighted and derivative DTW, and edit distance-based measures, including longest common subsequence, edit distance with real penalty, time warp with edit, and move---split---merge. These measures have the common characteristic that they operate in the time domain and compensate for potential localised misalignment through some elastic adjustment. Our aim is to experimentally test two hypotheses related to these distance measures. Firstly, we test whether there is any significant difference in accuracy for TSC problems between nearest neighbour classifiers using these distance measures. Secondly, we test whether combining these elastic distance measures through simple ensemble schemes gives significantly better accuracy. We test these hypotheses by carrying out one of the largest experimental studies ever conducted into time series classification. Our first key finding is that there is no significant difference between the elastic distance measures in terms of classification accuracy on our data sets. Our second finding, and the major contribution of this work, is to define an ensemble classifier that significantly outperforms the individual classifiers. We also demonstrate that the ensemble is more accurate than approaches not based in the time domain. Nearly all TSC papers in the data mining literature cite DTW (with warping window set through cross validation) as the benchmark for comparison. We believe that our ensemble is the first ever classifier to significantly outperform DTW and as such raises the bar for future work in this area.

Stride Segmentation during Free Walk Movements Using Multi-Dimensional Subsequence Dynamic Time Warping on Inertial Sensor Data

Abstract: Changes in gait patterns provide important information about individuals’ health. To perform sensor based gait analysis, it is crucial to develop methodologies to automatically segment single strides from continuous movement sequences. In this study we developed an algorithm based on time-invariant template matching to isolate strides from inertial sensor signals. Shoe-mounted gyroscopes and accelerometers were used to record gait data from 40 elderly controls, 15 patients with Parkinson’s disease and 15 geriatric patients. Each stride was manually labeled from a straight 40 m walk test and from a video monitored free walk sequence. A multi-dimensional subsequence Dynamic Time Warping (msDTW) approach was used to search for patterns matching a pre-defined stride template constructed from 25 elderly controls. F-measure of 98% (recall 98%, precision 98%) for 40 m walk tests and of 97% (recall 97%, precision 97%) for free walk tests were obtained for the three groups. Compared to conventional peak detection methods up to 15% F-measure improvement was shown. The msDTW proved to be robust for segmenting strides from both standardized gait tests and free walks. This approach may serve as a platform for individualized stride segmentation during activities of daily living.

Query-by-example keyword spotting using long short-term memory networks

Abstract: We present a novel approach to query-by-example keyword spotting (KWS) using a long short-term memory (LSTM) recurrent neural network-based feature extractor. In our approach, we represent each keyword using a fixed-length feature vector obtained by running the keyword audio through a word-based LSTM acoustic model. We use the activations prior to the softmax layer of the LSTM as our keyword-vector. At runtime, we detect the keyword by extracting the same feature vector from a sliding window and computing a simple similarity score between this test vector and the keyword vector. With clean speech, we achieve 86% relative false rejection rate reduction at 0.5% false alarm rate when compared to a competitive phoneme posteriorgram with dynamic time warping KWS system, while the reduction in the presence of babble noise is 67%. Our system has a small memory footprint, low computational cost, and high precision, making it suitable for on-device applications.

Trajectory similarity measures

Abstract: Storing, querying, and analyzing trajectories is becoming increasingly important, as the availability and volumes of trajectory data increases. One important class of trajectory analysis is computing trajectory similarity. This paper introduces and compares four of the most common measures of trajectory similarity: longest common subsequence (LCSS), Frechet distance, dynamic time warping (DTW), and edit distance. These four measures have been implemented in a new open source R package, freely available on CRAN [19]. The paper highlights some of the differences between these four similarity measures, using real trajectory data, in addition to indicating some of the important emerging applications for measurement of trajectory similarity.

Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition

Abstract: Accurate and computationally efficient means for classifying human activities have been the subject of extensive research efforts. Most current research focuses on extracting complex features to achieve high classification accuracy. We propose a template selection approach based on Dynamic Time Warping, such that complex feature extraction and domain knowledge is avoided. We demonstrate the predictive capability of the algorithm on both simulated and real smartphone data.

Review of Functional Data Analysis

Abstract: With the advance of modern technology, more and more data are being recorded continuously during a time interval or intermittently at several discrete time points. They are both examples of "functional data", which have become a prevailing type of data. Functional Data Analysis (FDA) encompasses the statistical methodology for such data. Broadly interpreted, FDA deals with the analysis and theory of data that are in the form of functions. This paper provides an overview of FDA, starting with simple statistical notions such as mean and covariance functions, then covering some core techniques, the most popular of which is Functional Principal Component Analysis (FPCA). FPCA is an important dimension reduction tool and in sparse data situations can be used to impute functional data that are sparsely observed. Other dimension reduction approaches are also discussed. In addition, we review another core technique, functional linear regression, as well as clustering and classification of functional data. Beyond linear and single or multiple index methods we touch upon a few nonlinear approaches that are promising for certain applications. They include additive and other nonlinear functional regression models, such as time warping, manifold learning, and dynamic modeling with empirical differential equations. The paper concludes with a brief discussion of future directions.

Multivariate time series classification with parametric derivative dynamic time warping

Abstract: We improve DTW distance measure in multivariate time series classification.We use derivatives to improve DTW in multivariate time series classification.We test effectiveness on 18 real time series.We present a detailed comparison of proposed methods. Multivariate time series (MTS) data are widely used in a very broad range of fields, including medicine, finance, multimedia and engineering. In this paper a new approach for MTS classification, using a parametric derivative dynamic time warping distance, is proposed. Our approach combines two distances: the DTW distance between MTS and the DTW distance between derivatives of MTS. The new distance is used in classification with the nearest neighbor rule. Experimental results performed on 18 data sets demonstrate the effectiveness of the proposed approach for MTS classification.

On the non-trivial generalization of Dynamic Time Warping to the multi-dimensional case

Abstract: In the last decade, Dynamic Time Warping (DTW) has emerged as the distance measure of choice for virtually all time series data mining applications. This is the result of significant progress in improving DTW’s efficiency, and multiple empirical studies showing that DTW-based classifiers at least equal the accuracy of all their rivals across dozens of datasets. Thus far, most of the research has considered only the one-dimensional case, with practitioners generalizing to the multi-dimensional case in one of two ways. In general, it appears the community believes either that the two ways are equivalent, or that the choice is irrelevant. In this work, we show that this is not the case. The two most commonly used multidimensional DTW methods can produce different classifications, and neither one dominates over the other. This seems to suggest that one should learn the best method for a particular application. However, we will show that this is not necessary; a simple, principled rule can be used on a case-by-case basis to predict which of the two methods we should give credence to. Our method allows us to ensure that classification results are at least as accurate as the better of the two rival methods, and in many cases, our method is strictly more accurate. We demonstrate our ideas with the most extensive set of multi-dimensional time series classification experiments ever attempted. Keywords—Dynamic Time Warping; Classification

Accelerating Dynamic Time Warping Clustering with a Novel Admissible Pruning Strategy

Abstract: Clustering time series is a useful operation in its own right, and an important subroutine in many higher-level data mining analyses, including data editing for classifiers, summarization, and outlier detection. While it has been noted that the general superiority of Dynamic Time Warping (DTW) over Euclidean Distance for similarity search diminishes as we consider ever larger datasets, as we shall show, the same is not true for clustering. Thus, clustering time series under DTW remains a computationally challenging task. In this work, we address this lethargy in two ways. We propose a novel pruning strategy that exploits both upper and lower bounds to prune off a large fraction of the expensive distance calculations. This pruning strategy is admissible; giving us provably identical results to the brute force algorithm, but is at least an order of magnitude faster. For datasets where even this level of speedup is inadequate, we show that we can use a simple heuristic to order the unavoidable calculations in a most-useful-first ordering, thus casting the clustering as an anytime algorithm. We demonstrate the utility of our ideas with both single and multidimensional case studies in the domains of astronomy, speech physiology, medicine and entomology.

Multi-layered gesture recognition with Kinect

Abstract: This paper proposes a novel multi-layered gesture recognition method with Kinect. We explore the essential linguistic characters of gestures: the components concurrent character and the sequential organization character, in a multi-layered framework, which extracts features from both the segmented semantic units and the whole gesture sequence and then sequentially classifies the motion, location and shape components. In the first layer, an improved principle motion is applied to model the motion component. In the second layer, a particle-based descriptor and a weighted dynamic time warping are proposed for the location component classification. In the last layer, the spatial path warping is further proposed to classify the shape component represented by unclosed shape context. The proposed method can obtain relatively high performance for one-shot learning gesture recognition on the ChaLearn Gesture Dataset comprising more than 50, 000 gesture sequences recorded with Kinect.

An Inertial Pen With Dynamic Time Warping Recognizer for Handwriting and Gesture Recognition

Abstract: This paper presents an inertial-sensor-based digital pen (inertial pen) and its associated dynamic time warping (DTW)-based recognition algorithm for handwriting and gesture recognition. Users hold the inertial pen to write numerals or English lowercase letters and make hand gestures with their preferred handheld style and speed. The inertial signals generated by hand motions are wirelessly transmitted to a computer for online recognition. The proposed DTW-based recognition algorithm includes the procedures of inertial signal acquisition, signal preprocessing, motion detection, template selection, and recognition. We integrate signals collected from an accelerometer, a gyroscope, and a magnetometer into a quaternion-based complementary filter for reducing the integral errors caused by the signal drift or intrinsic noise of the gyroscope, which might reduce the accuracy of the orientation estimation. Furthermore, we have developed a minimal intra-class to maximal inter-class based template selection method (min-max template selection method) for a DTW recognizer to obtain a superior class separation for improved recognition. Experimental results have successfully validated the effectiveness of the DTW-based recognition algorithm for online handwriting and gesture recognition using the inertial pen.

Segmental acoustic indexing for zero resource keyword search

Abstract: The task of zero resource query-by-example keyword search has received much attention in recent years as the speech technology needs of the developing world grow. These systems traditionally rely upon dynamic time warping (DTW) based retrieval algorithms with runtimes that are linear in the size of the search collection. As a result, their scalability substantially lags that of their supervised counterparts, which take advantage of efficient word-based indices. In this paper, we present a novel audio indexing approach called Segmental Randomized Acoustic Indexing and Logarithmic-time Search (S-RAILS). S-RAILS generalizes the original frame-based RAILS methodology to word-scale segments by exploiting a recently proposed acoustic segment embedding technique. By indexing word-scale segments directly, we avoid higher cost frame-based processing of RAILS while taking advantage of the improved lexical discrimination of the embeddings. Using the same conversational telephone speech benchmark, we demonstrate major improvements in both speed and accuracy over the original RAILS system.

Method for investigating intradriver heterogeneity using vehicle trajectory data: A Dynamic Time Warping approach

Abstract: After first extending Newell’s car-following model to incorporate time-dependent parameters, this paper describes the Dynamic Time Warping (DTW) algorithm and its application for calibrating this microscopic simulation model by synthesizing driver trajectory data. Using the unique capabilities of the DTW algorithm, this paper attempts to examine driver heterogeneity in car-following behavior, as well as the driver’s heterogeneous situation-dependent behavior within a trip, based on the calibrated time-varying response times and critical jam spacing. The standard DTW algorithm is enhanced to address a number of estimation challenges in this specific application, and a numerical experiment is presented with vehicle trajectory data extracted from the Next Generation Simulation (NGSIM) project for demonstration purposes. The DTW algorithm is shown to be a reasonable method for processing large vehicle trajectory datasets, but requires significant data reduction to produce reasonable results when working with high resolution vehicle trajectory data. Additionally, singularities present an interesting match solution set to potentially help identify changing driver behavior; however, they must be avoided to reduce analysis complexity.

DTW-based kernel and rank-level fusion for 3D gait recognition using Kinect

Abstract: This paper presents a new 3D gait recognition method that utilizes the kinect skeleton data for representing the gait signature. We propose to use two new features, namely joint relative distance (JRD) and joint relative angle (JRA), which are robust against view and pose variations. The relevance of each JRD and JRA sequence in representing human gait is evaluated using a genetic algorithm. We also introduce a dynamic time warping-based kernel that takes a collection of JRD or JRA sequences as parameters and computes a dissimilarity measure between the training and the unknown sample. The proposed kernel can effectively handle variable walking speed without any need of extra pre-processing. In addition, we propose a rank-level fusion of JRD and JRA features that can boost the overall recognition performance greatly. The effectiveness of the proposed method is evaluated using a 3D skeletal gait database captured with a Kinect v2 sensor. In our experiments, rank level fusion of joint relative distance (JRD) and joint relative angle (JRA) achieves promising results, as compared against only JRD and only JRA-based gait recognition.

A hybrid dynamic time warping-deep neural network architecture for unsupervised acoustic modeling.

Abstract: We report on an architecture for the unsupervised discovery of talker-invariant subword embeddings. It is made out of two components: a dynamic-time warping based spoken term discovery (STD) system and a Siamese deep neural network (DNN). The STD system clusters word-sized repeated fragments in the acoustic streams while the DNN is trained to minimize the distance between time aligned frames of tokens of the same cluster, and maximize the distance between tokens of different clusters. We use additional side information regarding the average duration of phonemic units, as well as talker identity tags. For evaluation we use the datasets and metrics of the Zero Resource Speech Challenge. The model shows improvement over the baseline in subword unit modeling.

Real-Time Human Movement Retrieval and Assessment With Kinect Sensor

Abstract: The difficulty of vision-based posture estimation is greatly decreased with the aid of commercial depth camera, such as Microsoft Kinect. However, there is still much to do to bridge the results of human posture estimation and the understanding of human movements. Human movement assessment is an important technique for exercise learning in the field of healthcare. In this paper, we propose an action tutor system which enables the user to interactively retrieve a learning exemplar of the target action movement and to immediately acquire motion instructions while learning it in front of the Kinect. The proposed system is composed of two stages. In the retrieval stage, nonlinear time warping algorithms are designed to retrieve video segments similar to the query movement roughly performed by the user. In the learning stage, the user learns according to the selected video exemplar, and the motion assessment including both static and dynamic differences is presented to the user in a more effective and organized way, helping him/her to perform the action movement correctly. The experiments are conducted on the videos of ten action types, and the results show that the proposed human action descriptor is representative for action video retrieval and the tutor system can effectively help the user while learning action movements.

A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors

Abstract: Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing “Palm Downward” sign gestures from “Palm Inward” ones. Only the “Palm Inward” gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

Continuous Action Recognition Based on Sequence Alignment

Abstract: Continuous action recognition is more challenging than isolated recognition because classification and segmentation must be simultaneously carried out. We build on the well known dynamic time warping framework and devise a novel visual alignment technique, namely dynamic frame warping (DFW), which performs isolated recognition based on per-frame representation of videos, and on aligning a test sequence with a model sequence. Moreover, we propose two extensions which enable to perform recognition concomitant with segmentation, namely one-pass DFW and two-pass DFW. These two methods have their roots in the domain of continuous recognition of speech and, to the best of our knowledge, their extension to continuous visual action recognition has been overlooked. We test and illustrate the proposed techniques with a recently released dataset (RAVEL) and with two public-domain datasets widely used in action recognition (Hollywood-1 and Hollywood-2). We also compare the performances of the proposed isolated and continuous recognition algorithms with several recently published methods.

Dynamic time warping and sparse representation classification for birdsong phrase classification using limited training dataa)

Abstract: Annotation of phrases in birdsongs can be helpful to behavioral and population studies. To reduce the need for manual annotation, an automated birdsong phrase classification algorithm for limited data is developed. Limited data occur because of limited recordings or the existence of rare phrases. In this paper, classification of up to 81 phrase classes of Cassin's Vireo is performed using one to five training samples per class. The algorithm involves dynamic time warping (DTW) and two passes of sparse representation (SR) classification. DTW improves the similarity between training and test phrases from the same class in the presence of individual bird differences and phrase segmentation inconsistencies. The SR classifier works by finding a sparse linear combination of training feature vectors from all classes that best approximates the test feature vector. When the class decisions from DTW and the first pass SR classification are different, SR classification is repeated using training samples from these two conflicting classes. Compared to DTW, support vector machines, and an SR classifier without DTW, the proposed classifier achieves the highest classification accuracies of 94% and 89% on manually segmented and automatically segmented phrases, respectively, from unseen Cassin's Vireo individuals, using five training samples per class.

Robust score normalization for DTW-based on-line signature verification

Abstract: In the field of automatic signature verification, a major challenge for statistical analysis and pattern recognition is the small number of reference signatures per user. Score normalization, in particular, is challenged by the lack of information about intra-user variability. In this paper, we analyze several approaches to score normalization for dynamic time warping and propose a new two-stage normalization which detects simple forgeries in a first stage and copes with more skilled forgeries in a second stage. An experimental evaluation is conducted on two data sets with different characteristics, namely the MCYT online signature corpus, which contains over three hundred users, and the SUSIG visual sub-corpus, which contains highly skilled forgeries. The results demonstrate that score normalization is a key component for signature verification and that the proposed two-stage normalization achieves some of the best results on these difficult data sets both for random and for skilled forgeries.

Support vector-based algorithms with weighted dynamic time warping kernel function for time series classification

Abstract: In this paper, we propose support vector-based supervised learning algorithms, called multiclass support vector data description with weighted dynamic time warping kernel function (MSVDD-WDTWK) and multiclass support vector machines with weighted dynamic time warping kernel function (MSVM-WDTWK), which provides a flexible and robust kernel function for time series classification between non-aligned time series data resulting in improved accuracy. The proposed WDTW kernel function provides an optimal match between two time series data by not only allowing a non-linear mapping between two data sequences, but also considering relative significance depending on the phase difference between points on time series data. We validate the proposed approaches using extensive numerical experiments on a number of multiclass UCR time series data mining archive, and demonstrate that our proposed methods provide lower classification error rates compared with existing techniques.

Recognition of Hand Gestures Observed by Depth Cameras

Abstract: We focus on gesture recognition based on 3D information in the form of a point cloud of the observed scene. A descriptor of the scene is built on the basis of a Viewpoint Feature Histogram (VFH). To increase the distinctiveness of the descriptor the scene is divided into smaller 3D cells and VFH is calculated for each of them. A verification of the method on publicly available Polish and American sign language datasets containing dynamic gestures as well as hand postures acquired by a time-of-flight (ToF) camera or Kinect is presented. Results of cross-validation test are given. Hand postures are recognized using a nearest neighbour classifier with city-block distance. For dynamic gestures two types of classifiers are applied: (i) the nearest neighbour technique with dynamic time warping and (ii) hidden Markov models. The results confirm the usefulness of our approach.