We consider the problem of partitioning a set of m points in the n-dimensional Euclidean space into k clusters (usually m and n are variable, while k is fixed), so as to minimize the sum of squared distances between each point and its cluster center. This formulation is usually the objective of the k-means clustering algorithm (Kanungo et al. (2000)). We prove that this problem in NP-hard even for k e 2, and we consider a continuous relaxation of this discrete problem: find the k-dimensional subspace V that minimizes the sum of squared distances to V of the m points. This relaxation can be solved by computing the Singular Value Decomposition (SVD) of the m × n matrix A that represents the m pointss this solution can be used to get a 2-approximation algorithm for the original problem. We then argue that in fact the relaxation provides a generalized clustering which is useful in its own right.

Finally, we show that the SVD of a random submatrix—chosen according to a suitable probability distribution—of a given matrix provides an approximation to the SVD of the whole matrix, thus yielding a very fast randomized algorithm. We expect this algorithm to be the main contribution of this paper, since it can be applied to problems of very large size which typically arise in modern applications.

/pdf/clustering-large-graphs-via-the-singular-value-decomposition-4ju4j5vu6d.pdf

Clustering Large Graphs via the Singular Value Decomposition

Data Clustering: Theory, Algorithms, and Applications (ASA-SIAM Series on Statistics and Applied Probability)

In this paper, we propose a very simple and elegant patch-based, machine learning technique for image denoising using the higher order singular value decomposition (HOSVD). The technique simply groups together similar patches from a noisy image (with similarity defined by a statistically motivated criterion) into a 3D stack, computes the HOSVD coefficients of this stack, manipulates these coefficients by hard thresholding, and inverts the HOSVD transform to produce the final filtered image. Our technique chooses all required parameters in a principled way, relating them to the noise model. We also discuss our motivation for adopting the HOSVD as an appropriate transform for image denoising. We experimentally demonstrate the excellent performance of the technique on grayscale as well as color images. On color images, our method produces state-of-the-art results, outperforming other color image denoising algorithms at moderately high noise levels. A criterion for optimal patch-size selection and noise variance estimation from the residual images (after denoising) is also presented.

Image Denoising Using the Higher Order Singular Value Decomposition

On the relationships between SVD, KLT and PCA

In this paper, we introduce a novel technique to automatically detect salient regions of an image via high-dimensional color transform. Our main idea is to represent a saliency map of an image as a linear combination of high-dimensional color space where salient regions and backgrounds can be distinctively separated. This is based on an observation that salient regions often have distinctive colors compared to the background in human perception, but human perception is often complicated and highly nonlinear. By mapping a low dimensional RGB color to a feature vector in a high-dimensional color space, we show that we can linearly separate the salient regions from the background by finding an optimal linear combination of color coefficients in the high-dimensional color space. Our high dimensional color space incorporates multiple color representations including RGB, CIELab, HSV and with gamma corrections to enrich its representative power. Our experimental results on three benchmark datasets show that our technique is effective, and it is computationally efficient in comparison to previous state-of-the-art techniques.

/pdf/salient-region-detection-via-high-dimensional-color-e0jidsb8sh.pdf

Salient Region Detection via High-Dimensional Color Transform

The use of singular value decomposition (SVD) techniques in digital image processing is of considerable interest for those facilities with large computing power and stringent imaging requirements. The SVD methods are useful for image as well as quite general point spread function (impulse response) representations. The methods represent simple extensions of the theory of linear filtering. Image enhancement examples will be developed illustrating these principles. The most interesting cases of image restoration are those which involve space variant imaging systems. The SVD, combined with pseudoinverse techniques, provides insight into these types of restorations. Illustrations of large scale N2× N2point spread function matrix representations are discussed along with separable space variant N2× N2point spread function matrix examples. Finally, analysis and methods for obtaining a pseudoinverse of separable space variant point spread functions (SVPSF's) are presented with a variety of object and imaging system dagradations.

H. Andrews

Papers

Singular value decompositions and digital image processing