http://www.cs.cas.cz/mp/papers07/causality-IT-final.pdf

Causality detection based on information-theoretic approaches in time series analysis

Statistical measures coming from information theory represent interesting bases for image and video processing tasks such as image retrieval and video object tracking. For example, let us mention the entropy and the Kullback-Leibler divergence. Accurate estimation of these measures requires to adapt to the local sample density, especially if the data are high-dimensional. The k nearest neighbor (kNN) framework has been used to define efficient variable-bandwidth kernel-based estimators with such a locally adaptive property. Unfortunately, these estimators are computationally intensive since they rely on searching neighbors among large sets of d-dimensional vectors. This computational burden can be reduced by pre-structuring the data, e.g. using binary trees as proposed by the approximated nearest neighbor (ANN) library. Yet, the recent opening of graphics processing units (GPU) to general-purpose computation by means of the NVIDIA CUDA API offers the image and video processing community a powerful platform with parallel calculation capabilities. In this paper, we propose a CUDA implementation of the ldquobrute forcerdquo kNN search and we compare its performances to several CPU-based implementations including an equivalent brute force algorithm and ANN. We show a speed increase on synthetic and real data by up to one or two orders of magnitude depending on the data, with a quasi-linear behavior with respect to the data size in a given, practical range.

Fast k nearest neighbor search using GPU

A new universal estimator of divergence is presented for multidimensional continuous densities based on k-nearest-neighbor (k-NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k-NN method can be further improved by an adaptive choice of k.

Divergence Estimation for Multidimensional Densities Via $k$ -Nearest-Neighbor Distances

A class of estimators of the Renyi and Tsallis entropies of an unknown distribution f in R^m is presented. These estimators are based on the k-th nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon's entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.

/pdf/a-class-of-renyi-information-estimators-for-multidimensional-2snwur3ozs.pdf

A class of Rényi information estimators for multidimensional densities

Uncovering the directionality of coupling is a significant step in understanding drive-response relationships in complex systems. In this paper, we discuss a nonparametric method for detecting the directionality of coupling based on the estimation of information theoretic functionals. We consider several different methods for estimating conditional mutual information. The behavior of each estimator with respect to its free parameter is shown using a linear model where an analytical estimate of conditional mutual information is available. Numerical experiments in detecting coupling directionality are performed using chaotic oscillators, where the influence of the phase extraction method and relative frequency ratio is investigated.

Inferring the directionality of coupling with conditional mutual information

This paper proposes a new class of estimators of an unknown entropy of random vector Its asymptotic unbiasedness and consistency are proved Further, this class of estimators is used to build both goodness-of-fit and independence tests based on sample entropy A simulation study indicates that the test involving the proposed entropy estimate has higher power than other well-known competitors under heavy tailed alternatives which are frequently used in many financial applications

V. V. Mergel

Papers

A new class of random vector entropy estimators and its applications in testing statistical hypotheses