We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

High dimension, low sample size data are emerging in various areas of science. We find a common structure underlying many such data sets by using a non-standard type of asymptotics: the dimension tends to ∞ while the sample size is fixed. Our analysis shows a tendency for the data to lie deterministically at the vertices of a regular simplex. Essentially all the randomness in the data appears only as a random rotation of this simplex. This geometric representation is used to obtain several new statistical insights. Copyright 2005 Royal Statistical Society.

Geometric representation of high dimension, low sample size data

The book is meant to be useful for practitioners in applied data analysis in life sciences, engineering, physics and chemistry; it will also be valuable to PhD ...

Principal Manifolds for Data Visualization and Dimension Reduction

Results obtained using continuous and discrete wavelet transforms as applied to problems in neurodynamics are reviewed, with the emphasis on the potential of wavelet analysis for decoding signal information from neural systems and networks. The following areas of application are considered: (1) the microscopic dynamics of single cells and intracellular processes, (2) sensory data processing, (3) the group dynamics of neuronal ensembles, and (4) the macrodynamics of rhythmical brain activity (using multichannel EEG recordings). The detection and classification of various oscillatory patterns of brain electrical activity and the development of continuous wavelet-based brain activity monitoring systems are also discussed as possibilities.

/pdf/wavelet-analysis-in-neurodynamics-2iabij3yca.pdf

Wavelet analysis in neurodynamics

The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional lp quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm l2, but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for lp at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data.

Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality

High-dimensional data and high-dimensional representations of reality are inherent features of modern Artificial Intelligence systems and applications of machine learning. The well-known phenomenon of the “curse of dimensionality” states: many problems become exponentially difficult in high dimensions. Recently, the other side of the coin, the “blessing of dimensionality”, has attracted much attention. It turns out that generic high-dimensional datasets exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high dimensional spaces. Here we present a brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience.

https://www.mdpi.com/1099-4300/22/1/82/pdf

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality.

High-dimensional data and high-dimensional representations of reality are inherent features of modern Artificial Intelligence systems and applications of machine learning. The well-known phenomenon of the "curse of dimensionality" states: many problems become exponentially difficult in high dimensions. Recently, the other side of the coin, the "blessing of dimensionality", has attracted much attention. It turns out that generic high-dimensional datasets exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high dimensional spaces. Here we present a brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience.

High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

We study mechanisms of information processing in the principalis (Pr5), oralis (Sp5o) and interpolaris (Sp5i) nuclei of the trigeminal sensory complex of the rat under whisker stimulation by short air puffs. After the standard electrophysiological description of the neural spiking activity we apply a novel wavelet based method quantifying the structural stability of Bring patterns evoked by a periodic whisker stimulation. We show that the response stability depends on the puff duration delivered to the vibrissae and differs among the analyzed nuclei. Pr5 and Sp5i exhibit the maximal stability to an intermediate stimulus duration, whereas Sp5o shows "preference" for short stimuli.

Tactile information processing in the trigeminal complex of the rat

The article discusses the algorithm and methods of statistical processing of one type of grouped data – interval-censored samples with overlapping observation intervals. The samples are censored by the observation time interval of the measured value. The samples consist entirely of such interval observations. The initial information was the data of one-time and multiple inspections of the structure in operation in order to detect damage in the form of cracks. The stage of appearance of the initial crack (macrocrack) in the zone of stress concentration is determined. A macrocrack, for objective reasons, cannot be directly detected during inspections. Examples of describing the time of formation of initial cracks in the form of interval-censored samples with intersecting observation intervals are presented. The main focus was on the development of methods for statistical processing of the studied samples. The maximum likelihood method was applied for parametric estimation of the time of appearance of macrocracks. Basic maximum likelihood equations are obtained for distributions with location and scale parameters. The computational procedure of the accumulation method with the use of Fisher’s information is applied. Expressions for calculating the variances of parameter estimates are obtained. An equation for checking the existence of a solution to the system of likelihood equations is presented. The appearance of macrocracks can have a different physical nature, for example, the accumulation of fatigue or instantaneous damage due to an overload exceeding the calculated one. Different physical nature implies different types of preventive work to prevent them and is described by different probabilistic models. A likelihood ratio criterion is proposed to test the statistical hypothesis about the situation of sudden appearance of macrocracks at a load exceeding the calculated one. The results obtained show the features of the structure of the studied samples and possible methods of statistical processing. Bibliography 11 titles Table 2. 

Valery A. Makarov

Papers

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality.

High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

Tactile information processing in the trigeminal complex of the rat

Estimation of maximum likelihood in the conditions of inspections of the current state