${k}$ nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed ${k}$ value (even though set by experts) to all test samples. Previous solutions assign different $k$ values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal $k$ values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal $k$ values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal $k$ values. In the test stage, the kTree fast outputs the optimal $k$ value for each test sample, and then, the kNN classification can be conducted using the learned optimal $k$ value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed ${k}$ value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different ${k}$ values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks.

Efficient kNN Classification With Different Numbers of Nearest Neighbors

In the present study k-Nearest Neighbor classification method, have been studied for economic forecasting. Due to the effects of companies’ financial distress on stakeholders, financial distress prediction models have been one of the most attractive areas in financial research. In recent years, after the global financial crisis, the number of bankrupt companies has risen. Since companies' financial distress is the first stage of bankruptcy, using financial ratios for predicting financial distress have attracted too much attention of the academics as well as economic and financial institutions. Although in recent years studies on predicting companies’ financial distress in Iran have been increased, most efforts have exploited traditional statistical methods; and just a few studies have used nonparametric methods. Recent studies demonstrate this method is more capable than other methods.

Application of K-Nearest Neighbor (KNN) Approach for Predicting Economic Events: Theoretical Background

K-nearest neighbor (KNN) rule is a well-known non-parametric classifier that is widely used in pattern recognition. However, the sensitivity of the neighborhood size k always seriously degrades the KNN-based classification performance, especially in the case of the small sample size with the existing outliers. To overcome this issue, in this article we propose a generalized mean distance-based k-nearest neighbor classifier (GMDKNN) by introducing multi-generalized mean distances and the nested generalized mean distance that are based on the characteristic of the generalized mean. In the proposed method, multi-local mean vectors of the given query sample in each class are calculated by adopting its class-specific k nearest neighbors. Using the achieved k local mean vectors per class, the corresponding k generalized mean distances are calculated and then used to design the categorical nested generalized mean distance. In the classification phase, the categorical nested generalized mean distance is used as the classification decision rule and the query sample is classified into the class with the minimum nested generalized mean distance among all the classes. Extensive experiments on the UCI and KEEL data sets, synthetic data sets, the KEEL noise data sets and the UCR time series data sets are conducted by comparing the proposed method to the state-of-art KNN-based methods. The experimental results demonstrate that the proposed GMDKNN performs better and has the less sensitiveness to k. Thus, our proposed GMDKNN with the robust and effective classification performance could be a promising method for pattern recognition in some expert and intelligence systems.

A generalized mean distance-based k-nearest neighbor classifier

Recent years have seen significant progress in improving both the efficiency and effectiveness of time series classification. However, because the best solution is typically the Nearest Neighbor algorithm with the relatively expensive Dynamic Time Warping as the distance measure, successful deployments on resource constrained devices remain elusive. Moreover, the recent explosion of interest in wearable devices, which typically have limited computational resources, has created a growing need for very efficient classification algorithms. A commonly used technique to glean the benefits of the Nearest Neighbor algorithm, without inheriting its undesirable time complexity, is to use the Nearest Centroid algorithm. However, because of the unique properties of (most) time series data, the centroid typically does not resemble any of the instances, an unintuitive and underappreciated fact. In this work we show that we can exploit a recent result to allow meaningful averaging of 'warped' times series, and that this result allows us to create ultra-efficient Nearest 'Centroid' classifiers that are at least as accurate as their more lethargic Nearest Neighbor cousins.

/pdf/dynamic-time-warping-averaging-of-time-series-allows-faster-583y5gh7xe.pdf

Dynamic Time Warping Averaging of Time Series Allows Faster and More Accurate Classification

https://hal.archives-ouvertes.fr/hal-01998052/document

Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach.

In this paper, we develop a novel Distance-weighted k -nearest Neighbor rule (DWKNN), using the dual distance-weighted function. The proposed DWKNN is motivated by the sensitivity problem of the selection of the neighborhood size k that exists in k -nearest Neighbor rule (KNN), with the aim of improving classification performance. The experiment results on twelve real data sets demonstrate that our proposed classifier is robust to different choices of k to some degree, and yields good performance with a larger optimal k, compared to the other state-of-art KNN-based methods.

A new distance-weighted k-nearest neighbor classifier

K-nearest neighbor rule (KNN) is the well-known non-parametric technique in the statistical pattern classification, owing to its simplicity, intuitiveness and effectiveness. In this paper, we firstly review the related works in brief and detailedly analyze the sensitivity issue on the choice of the neighborhood size k , existed in the KNN rule. Motivated by the problem, a novel dual weighted voting scheme for KNN is developed. With the goal of overcoming the sensitivity of the choice of the neighborhood size k and improving the classification performance, the proposed classifier mainly employs the dual weighted voting function to reduce the effect of the outliers in the k nearest neighbors of each query object. To verify the superiority of the proposed classifier, the experiments are conducted on one artificial data set and twelve real data sets, in comparison with the other classifiers. Experimental results suggest that our proposed classifier is an effective algorithm for the classification tasks in many practical situations, owing to its satisfactory classification performance and robustness over a wide range of k .

A Novel Weighted Voting for K-Nearest Neighbor Rule

K-nearest neighbor (KNN) rule is a simple and effective algorithm in pattern classification. In this article, we propose a local mean-based k-nearest centroid neighbor classifier that assigns to each query pattern a class label with nearest local centroid mean vector so as to improve the classification performance. The proposed scheme not only takes into account the proximity and spatial distribution of k neighbors, but also utilizes the local mean vector of k neighbors from each class in making classification decision. In the proposed classifier, a local mean vector of k nearest centroid neighbors from each class for a query pattern is well positioned to sufficiently capture the class distribution information. In order to investigate the classification behavior of the proposed classifier, we conduct extensive experiments on the real and synthetic data sets in terms of the classification error. Experimental results demonstrate that our proposed method performs significantly well, particularly in the small sample size cases, compared with the state-of-the-art KNN-based algorithms.

A Local Mean-Based k-Nearest Centroid Neighbor Classifier

K-nearest neighbor rule (KNN) has been regarded as one of the top 10 methods in the field of data mining. Due to its simplicity and effectiveness, it has been widely studied and applied to various classification tasks. In this article, we develop a novel representation coefficient-based k-nearest centroid neighbor method (RCKNCN), which aims to further improve the classification performance and reduce the method’s sensitivity to the neighborhood size k, especially in the cases of small sample size. Different from existing KNN-based methods, RCKNCN is able to capture both the proximity and the geometry of k-nearest neighbors, and learn to differentiate the contribution of each neighbor to the classification of a testing sample through a linear representation method. Moreover, under the RCKNCN framework, we also propose a novel weighted majority voting algorithm using the representation coefficients associated with individual nearest centroid neighbors, which are deemed to hold more discriminative information of the neighbors. To fully study the classification performance of RCKNCN, we compare it with the state-of-the-art KNN-based methods on many data sets that are widely used in the literature. The extensive experiments demonstrate the effectiveness and robustness of our method in various classification tasks. 

Taisong Xiong

Papers

A new distance-weighted k-nearest neighbor classifier

A Novel Weighted Voting for K-Nearest Neighbor Rule

A Local Mean-Based k-Nearest Centroid Neighbor Classifier

A representation coefficient-based k-nearest centroid neighbor classifier

Weighted K-nearest centroid neighbor classification