Showing papers on "Dynamic time warping published in 2021"

Multisensor-Based Navigation and Control of a Mobile Service Robot

Abstract: Service robot navigation must take the humans into account explicitly so as to produce motion behaviors that reflect its social awareness. Generally, the navigation problems of mobile service robot can be summarized to three aspects: 1) human detection; 2) robot real-time localization; and 3) robot motion planning. The purpose of this paper is to provide a feasible strategy to integrate these three aspects to achieve a conscious, safe, accurate, robust, and efficient navigation. We first introduce the human detection system for recognition of human gesture using a weighted dynamic time warping (DTW) with kinematic constraints. Thus, by interpreting the human body language through gesture recognition, robot motion behaviors like heading to the assigned position or following people can be activated. Then, for the robot localization, a simultaneous localization and mapping (SLAM) method based on artificial and natural landmark recognition is employed to provide absolute position feedback in real time. For the motion planning, a novel quadrupole potential field (QPF) method is proposed to plan collision-free trajectories, adequately considering the nonholomic constraint of the mobile robot system. Then, a robust kinematic controller is designed for trajectory tracking to account for slip disturbances. Such a design automatically merges path finding, trajectory generation, and trajectory tracking in a closed-loop fashion, achieving simultaneous motion planning for obstacle avoidance and feedback stabilization to a desired position and orientation even in the presence of slippage. Finally, experiments prove the effectiveness and feasibility of the proposed strategy, showing a good navigation performance on mobile service robot.

DeepSign: Deep On-Line Signature Verification

Abstract: Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database, 1 iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs) 2 . for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user. 1 https://github.com/BiDAlab/DeepSignDB 2 Spanish Patent Application (P202030060).

Mapping crop types in complex farming areas using SAR imagery with dynamic time warping

Abstract: Crop type related information is very essential for various planning and decision support activities in everyday life especially for early forecast and monitoring of food production. Though smallholder farming areas are profound food producers, mapping crop types is mainly constrained by their inherent characteristics like fragmentation (small farm size), rugged terrain, and presence of thick clouds in the growing season. More importantly, crops mixed dominance of the landscape coupled with fragmented holdings, crops behave different phenological characteristics which mostly constrains conventional mapping techniques for crop type mapping. Therefore, the main objective of this study was mapping crop types using all-weather time-series Synthetic Aperture Radar (SAR) with time-weighted Dynamic Time Warping that accounts for phenological development of crops. The study has used Sentinel-1 dual polarimetry (VV, VH) and TerraSAR-X single polarimetry (HH) images. Basic Registration of Crop Plots (BRP) dataset was used as a reference for training and validation. Obtained imagery was passed through a series of pre-processing operations. As Sentinel-1 imagery has dual polarimetry bands, derived features (Ratio, Modified Radar Vegetation Index, and Dual Polarimetric Soil Vegetation Index) were computed. Additionally, polarimetric decomposition was also undertaken. Within stated broader objective, a detailed analysis was done to know crop-specific responses for incident radar signal, to understand the capability of Time-Weighted Dynamic Time Warping for crop type mapping, implications of using either only backscatter bands and inclusion of derived features and decomposed polarimetric features on mapping accuracy of crops. In addition to these, under broader dynamic time warping, two further model improvement strategies (Variable Time Weight Dynamic Time Warping and Angular Metric for Shape Similarity) were also tested for performance. More importantly, the study has investigated an ensemble classifier that integrates TerraSAR-X and Sentinel-1 classification outputs for synergistic use of both sensing systems for crop type mapping. From these analyses, the study has come up with promising findings that show potentials of SAR imagery with time-weighted Dynamic Time Warping for crop type mapping. It has also clearly demonstrated predictive capabilities of either using dual polarimetry or single polarimetry SAR datasets for mapping crops in smallholder farming areas. Finally, by considering achieved outputs and existing caveats on this study, to refine the findings, further works were also recommended.

Trimmed fuzzy clustering of financial time series based on dynamic time warping

Abstract: In finance, cluster analysis is a tool particularly useful for classifying stock market multivariate time series data related to daily returns, volatility daily stocks returns, commodity prices, volume trading, index, enhanced index tracking portfolio, and so on. In the literature, following different methodological approaches, several clustering methods have been proposed for clustering multivariate time series. In this paper by adopting a fuzzy approach and using the Partitioning Around Medoids strategy, we suggest to cluster multivariate financial time series by considering the dynamic time warping distance. In particular, we proposed a robust clustering method capable to neutralize the negative effects of possible outliers in the clustering process. The clustering method achieves its robustness by adopting a suitable trimming procedure to identify multivariate financial time series more distant from the bulk of data. The proposed clustering method is applied to the stocks composing the FTSE MIB index to identify common time patterns and possible outliers.

Representation Learning via Global Temporal Alignment and Cycle-Consistency

Abstract: We introduce a weakly supervised method for representation learning based on aligning temporal sequences (e.g., videos) of the same process (e.g., human action). The main idea is to use the global temporal ordering of latent correspondences across sequence pairs as a supervisory signal. In particular, we propose a loss based on scoring the optimal sequence alignment to train an embedding network. Our loss is based on a novel probabilistic path finding view of dynamic time warping (DTW) that contains the following three key features: (i) the local path routing decisions are contrastive and differentiable, (ii) pairwise distances are cast as probabilities that are contrastive as well, and (iii) our formulation naturally admits a global cycle-consistency loss that verifies correspondences. For evaluation, we consider the tasks of fine-grained action classification, few shot learning, and video synchronization. We report significant performance increases over previous methods. In addition, we report two applications of our temporal alignment framework, namely 3D pose reconstruction and fine-grained audio/visual retrieval.

A fast and accurate similarity measure for long time series classification based on local extrema and dynamic time warping

Abstract: The problem of similarity measures is a major area of interest within the field of time series classification (TSC). With the ubiquitous of long time series and the increasing demand for analyzing them on limited resource devices, there is a crucial need for efficient and accurate measures to deal with such kind of data. In fact, there are a plethora of good time series similarity measures in the literature. However, most existing methods achieve good performance for short time series, but their effectiveness decreases quickly as time series are longer. In this paper, we develop a new parameter-free measure for the specific purpose of quickly and accurately assessing the similarity between two given long time series. The proposed “Local Extrema Dynamic Time Warping” (LE-DTW) consists of two steps. The first is a time series representation technique that starts by reducing the dimensionality of a given time series using its local extrema. Next, it physically separates the minima and maxima points for more intuitiveness and consistency of the so-obtained time series representation. The second step consists in adapting the Dynamic Time Warping (DTW) measure so as to evaluate the score of similarity between the generated representations. We test the performance of LE-DTW on a wide range of real-world problems from the UCR time series archive for TSC. Experimental results indicate that for short time series, the proposed method achieves reasonable classification accuracy as compared to DTW. However, for long time series, LE-DTW performs much better. Indeed, it outperforms DTW while providing competitive results against popular distance-based classifiers. Moreover, in terms of efficiency, LE-DTW is orders of magnitude faster than DTW.

Adversarial Attacks on Time Series

Abstract: Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples.

Evaluation of the Time Stability and Uniqueness in PPG-Based Biometric System

Abstract: In this work, we demonstrates the feasibility of employing the biometric photoplethysmography (PPG) signal for human verification applications. The PPG signal has dominance in terms of accessibility and portability which makes its usage in many applications such as user access control very appealing. Therefore, we developed robust time-stable features using signal analysis and deep learning models to increase the robustness and performance of the verification system with the PPG signal. The proposed system focuses on utilizing different stretching mechanisms namely Dynamic Time Warping, zero padding and interpolation with Fourier transform, and fuses them at the data level to be then deployed with different deep learning models. The designed deep models consist of Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) which are considered to build a user specific model for the verification task. We collected a dataset consisting of 100 participants and recorded at two different time sessions using Plux pulse sensor. This dataset along with another two public databases are deployed to evaluate the performance of the proposed verification system in terms of uniqueness and time stability. The final result demonstrates the superiority of our proposed system tested on the built dataset and compared with other two public databases. The best performance achieved from our collected two-sessions database in terms of accuracy is 98% for the single-session and 87.1% for the two-sessions scenarios.

Time Series Data Augmentation for Neural Networks by Time Warping with a Discriminative Teacher

Abstract: Neural networks have become a powerful tool in pattern recognition and part of their success is due to generalization from using large datasets. However, unlike other domains, time series classification datasets are often small. In order to address this problem, we propose a novel time series data augmentation called guided warping. While many data augmentation methods are based on random transformations, guided warping exploits the element alignment properties of Dynamic Time Warping (DTW) and shapeDTW, a high-level DTW method based on shape descriptors, to deterministically warp sample patterns. In this way, the time series are mixed by warping the features of a sample pattern to match the time steps of a reference pattern. Furthermore, we introduce a discriminative teacher in order to serve as a directed reference for the guided warping. We evaluate the method on all 85 datasets in the 2015 UCR Time Series Archive with a deep convolutional neural network (CNN) and a recurrent neural network (RNN). The code with an easy to use implementation can be found at https://github.com/uchidalab/time_series_augmentation.

Input data selection for daily traffic flow forecasting through contextual mining and intra-day pattern recognition

Abstract: There is a large amount of literature about the traffic flow forecasting and most existing studies focus on prediction algorithm itself. However, how to select the appropriate historical data as input is also vital for the prediction task, while such studies are limited. This paper aims to cover this gap and proposes a method to select the appropriate historical data for daily traffic flow forecasting. The main idea is that some contextual factors including season, day of the week, weather, and holiday, influence the daily traffic flow pattern, and we select historical days with the similar pattern to the target day as the training data for prediction algorithm. The method consists of three steps: first, the similarities for traffic flow series between any two days are measured by Dynamic Time Warping, and then historical days are divided into different groups using a density-peak clustering algorithm; Second, the contextual factors are sorted by Elitist Non-dominated Sorting Genetic Algorithm (NSGA-II) using the clustering results, and their degrees of importance are transformed into weights in order to better measure the degrees of similarity between the clustered groups of days and the target day; third, one clustered group of historical data is selected based on the weighted degree of similarity and this group is used as the input for the prediction algorithm. At last, the benefits of the new method are discussed based on a Seattle case study, which illustrates that the proposed approach has higher prediction accuracy and stability across various prediction algorithms.

Time series motifs discovery under DTW allows more robust discovery of conserved structure

Abstract: In recent years, time series motif discovery has emerged as perhaps the most important primitive for many analytical tasks, including clustering, classification, rule discovery, segmentation, and summarization. In parallel, it has long been known that Dynamic Time Warping (DTW) is superior to other similarity measures such as Euclidean Distance under most settings. However, due to the computational complexity of both DTW and motif discovery, virtually no research efforts have been directed at combining these two ideas. The current best mechanisms to address their lethargy appear to be mutually incompatible. In this work, we present the first efficient, scalable and exact method to find time series motifs under DTW. Our method automatically performs the best trade-off of time-to-compute versus tightness-of-lower-bounds for a novel hierarchy of lower bounds that we introduce. As we shall show through extensive experiments, our algorithm prunes up to 99.99% of the DTW computations under realistic settings and is up to three to four orders of magnitude faster than the brute force search, and two orders of magnitude faster than the only other competitor algorithm. This allows us to discover DTW motifs in massive datasets for the first time. As we will show, in many domains, DTW-based motifs represent semantically meaningful conserved behavior that would escape our attention using all existing Euclidean distance-based methods.

Disentangling the city traffic rhythms: A longitudinal analysis of MFD patterns over a year

Abstract: Urban road transportation performance is the result of a complex interplay between the network supply and the travel demand. Fortunately, the framework around the macroscopic fundamental diagram (MFD) provides an efficient description of network-wide traffic performance. In this paper, we show how temporal patterns of vehicle traffic define the performance of urban road networks. We present two high-resolution traffic datasets covering a year each. We introduce a methodology to quantify the similarity of macroscopic traffic patterns. We do so by using the concepts of the MFD and a dynamic time warping (DTW) based algorithm for time series. This allows us to derive a few representative MFD clusters that capture the essential macroscopic traffic patterns. We then provide an in-depth analysis of traffic heterogeneity in the network which is indicative of the previously found clusters. Thereupon, we define a parsimonious classification approach to predict the expected MFD clusters early in the morning with high accuracy.

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

Abstract: This article contributes to the discourse on how contemporary computer and information technology may help in improving foreign language learning not only by supporting better and more flexible workflow and digitizing study materials but also through creating completely new use cases made possible by technological improvements in signal processing algorithms. We discuss an approach and propose a holistic solution to teaching the phonological phenomena which are crucial for correct pronunciation, such as the phonemes; the energy and duration of syllables and pauses, which construct the phrasal rhythm; and the tone movement within an utterance, i.e., the phrasal intonation. The working prototype of StudyIntonation Computer-Assisted Pronunciation Training (CAPT) system is a tool for mobile devices, which offers a set of tasks based on a “listen and repeat” approach and gives the audio-visual feedback in real time. The present work summarizes the efforts taken to enrich the current version of this CAPT tool with two new functions: the phonetic transcription and rhythmic patterns of model and learner speech. Both are designed on a base of a third-party automatic speech recognition (ASR) library Kaldi, which was incorporated inside StudyIntonation signal processing software core. We also examine the scope of automatic speech recognition applicability within the CAPT system workflow and evaluate the Levenstein distance between the transcription made by human experts and that obtained automatically in our code. We developed an algorithm of rhythm reconstruction using acoustic and language ASR models. It is also shown that even having sufficiently correct production of phonemes, the learners do not produce a correct phrasal rhythm and intonation, and therefore, the joint training of sounds, rhythm and intonation within a single learning environment is beneficial. To mitigate the recording imperfections voice activity detection (VAD) is applied to all the speech records processed. The try-outs showed that StudyIntonation can create transcriptions and process rhythmic patterns, but some specific problems with connected speech transcription were detected. The learners feedback in the sense of pronunciation assessment was also updated and a conventional mechanism based on dynamic time warping (DTW) was combined with cross-recurrence quantification analysis (CRQA) approach, which resulted in a better discriminating ability. The CRQA metrics combined with those of DTW were shown to add to the accuracy of learner performance estimation. The major implications for computer-assisted English pronunciation teaching are discussed.

Learning Discriminative Prototypes with Dynamic Time Warping

Abstract: Dynamic Time Warping (DTW) is widely used for temporal data processing. However, existing methods can neither learn the discriminative prototypes of different classes nor exploit such prototypes for further analysis. We propose Discriminative Prototype DTW (DP-DTW), a novel method to learn class-specific discriminative prototypes for temporal recognition tasks. DP-DTW shows superior performance compared to conventional DTWs on time series classification benchmarks1. Combined with end-to-end deep learning, DP-DTW can handle challenging weakly supervised action segmentation problems and achieves state of the art results on standard benchmarks. Moreover, detailed reasoning on the input video is enabled by the learned action prototypes. Specifically, an action-based video summarization can be obtained by aligning the input sequence with action prototypes.

Combining Machine Learning and Dynamic Time Wrapping for Vehicle Driving Event Detection Using Smartphones

Abstract: The detection of driving events could be useful for reducing accidents, fleet management and insurance premiums etc. Currently, top of the range vehicles and large fleets employ expensive driver monitoring systems. However, most drivers do not have access to such systems. The required monitoring platform would have to deliver the required performance while also being affordable and accessible. A candidate with considerable promise is the smartphone with sensors built-in that could be exploited for the detection of driving events. However, to date it has not been possible to achieve the required correct, missed and false detection rates in addition to the computational efficiency for real-time operations. This paper proposes a novel bagging tree and dynamic time warping (DTW) integrated algorithm for the detection of driving events employing acceleration and orientation data from a smartphone’s low cost three-axis accelerometers and gyroscopes. The bagging tree-based machine learning algorithm provides the initial maneuver detection results, as well as the location of the event start and end points. Event detection is then achieved by calculating the similarity of the results predicted through the bagging tree algorithm with the corresponding templates extracted from the experience datasets, while also applying a number of constraints to verify the calculated results. Field test results show that the proposed integrated algorithm is superior to the state-of-the-art, achieving a high correct detection accuracy of 97.5%, a low missed detection of 2.5% and a false detection rate of 2.9%. The corresponding results for the best alternative candidate method are 90.2%, 9.8% and 11.7%. Furthermore, the improvement in computational efficiency offered by our proposed approach is three to more than ten times greater than that of the other state-of-the-art algorithms.

Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

Abstract: This paper introduces Parallel Tacotron 2, a non-autoregressive neural text-to-speech model with a fully differentiable duration model which does not require supervised duration signals. The duration model is based on a novel attention mechanism and an iterative reconstruction loss based on Soft Dynamic Time Warping, this model can learn token-frame alignments as well as token durations automatically. Experimental results show that Parallel Tacotron 2 outperforms baselines in subjective naturalness in several diverse multi speaker evaluations. Its duration control capability is also demonstrated.

Vision Based Hand Gesture Recognition Using 3D Shape Context

Abstract: Hand gesture recognition is a popular topic in computer vision and makes human-computer interaction more flexible and convenient. The representation of hand gestures is critical for recognition. In this paper, we propose a new method to measure the similarity between hand gestures and exploit it for hand gesture recognition. The depth maps of hand gestures captured via the Kinect sensors are used in our method, where the 3D hand shapes can be segmented from the cluttered backgrounds. To extract the pattern of salient 3D shape features, we propose a new descriptor-3D Shape Context, for 3D hand gesture representation. The 3D Shape Context information of each 3D point is obtained in multiple scales because both local shape context and global shape distribution are necessary for recognition. The description of all the 3D points constructs the hand gesture representation, and hand gesture recognition is explored via dynamic time warping algorithm. Extensive experiments are conducted on multiple benchmark datasets. The experimental results verify that the proposed method is robust to noise, articulated variations, and rigid transformations. Our method outperforms state-of-the-art methods in the comparisons of accuracy and efficiency.

Information Theoretic Model to Simulate Agent-Signage Interaction for Wayfinding

Abstract: Signage systems are critical for communicating spatial information during wayfinding among a plethora of noise in the environment. A proper signage system can improve wayfinding performance and user experience by reducing the perceived complexity of the environment. However, previous models of sign-based wayfinding do not incorporate realistic noise or quantify the reduction in perceived complexity from the use of signage. Drawing upon concepts from information theory, we propose and validate a new agent-signage interaction model that quantifies available wayfinding information from signs for wayfinding. We conducted two online crowd-sourcing experiments to compute the distribution of a sign’s visibility and an agent’s decision-making confidence as a function of observation angle and viewing distance. We then validated this model using a virtual reality (VR) experiment with trajectories from human participants. The crowd-sourcing experiments provided a distribution of decision-making entropy (conditioned on visibility) that can be applied to any sign/environment. From the VR experiment, a training dataset of 30 trajectories was used to refine our model, and the remaining test dataset of 10 trajectories was compared with agent behavior using dynamic time warping (DTW) distance. The results revealed a reduction of 38.76% in DTW distance between the average trajectories before and after refinement. Our refined agent-signage interaction model provides realistic predictions of human wayfinding behavior using signs. These findings represent a first step towards modeling human wayfinding behavior in complex real environments in a manner that can incorporate several additional random variables (e.g., environment layout).

TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

Abstract: Teeth gestures become an alternative input modality for different situations and accessibility purposes. In this paper, we present TeethTap, a novel eyes-free and hands-free input technique, which can recognize up to 13 discrete teeth tapping gestures. TeethTap adopts a wearable 3D printed earpiece with an IMU sensor and a contact microphone behind both ears, which works in tandem to detect jaw movement and sound data, respectively. TeethTap uses a support vector machine to classify gestures from noise by fusing acoustic and motion data, and implements K-Nearest-Neighbor (KNN) with a Dynamic Time Warping (DTW) distance measurement using motion data for gesture classification. A user study with 11 participants demonstrated that TeethTap could recognize 13 gestures with a real-time classification accuracy of 90.9% in a laboratory environment. We further uncovered the accuracy differences on different teeth gestures when having sensors on single vs. both sides. Moreover, we explored the activation gesture under real-world environments, including eating, speaking, walking and jumping. Based on our findings, we further discussed potential applications and practical challenges of integrating TeethTap into future devices.

TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

Abstract: Teeth gestures become an alternative input modality for different situations and accessibility purposes. In this paper, we present TeethTap, a novel eyes-free and hands-free input technique, which can recognize up to 13 discrete teeth tapping gestures. TeethTap adopts a wearable 3D printed earpiece with an IMU sensor and a contact microphone behind both ears, which works in tandem to detect jaw movement and sound data, respectively. TeethTap uses a support vector machine to classify gestures from noise by fusing acoustic and motion data, and implements K-Nearest-Neighbor (KNN) with a Dynamic Time Warping (DTW) distance measurement using motion data for gesture classification. A user study with 11 participants demonstrated that TeethTap could recognize 13 gestures with a real-time classification accuracy of 90.9% in a laboratory environment. We further uncovered the accuracy differences on different teeth gestures when having sensors on single vs. both sides. Moreover, we explored the activation gesture under real-world environments, including eating, speaking, walking and jumping. Based on our findings, we further discussed potential applications and practical challenges of integrating TeethTap into future devices.