A Least-Squares Framework for Component Analysis
Reads0
Chats0
TLDR
The LS-WKRRR formulation of CA methods has several benefits: it provides a clean connection between many CA techniques and an intuitive framework to understand normalization factors, overcomes the small sample size problem, and provides a framework to easily extend CA methods.Abstract:
Over the last century, Component Analysis (CA) methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Canonical Correlation Analysis (CCA), Locality Preserving Projections (LPP), and Spectral Clustering (SC) have been extensively used as a feature extraction step for modeling, classification, visualization, and clustering. CA techniques are appealing because many can be formulated as eigen-problems, offering great potential for learning linear and nonlinear representations of data in closed-form. However, the eigen-formulation often conceals important analytic and computational drawbacks of CA techniques, such as solving generalized eigen-problems with rank deficient matrices (e.g., small sample size problem), lacking intuitive interpretation of normalization factors, and understanding commonalities and differences between CA methods. This paper proposes a unified least-squares framework to formulate many CA methods. We show how PCA, LDA, CCA, LPP, SC, and its kernel and regularized extensions correspond to a particular instance of least-squares weighted kernel reduced rank regression (LS--WKRRR). The LS-WKRRR formulation of CA methods has several benefits: 1) provides a clean connection between many CA techniques and an intuitive framework to understand normalization factors; 2) yields efficient numerical schemes to solve CA techniques; 3) overcomes the small sample size problem; 4) provides a framework to easily extend CA methods. We derive weighted generalizations of PCA, LDA, SC, and CCA, and several new CA techniques.read more
Citations
More filters
Journal ArticleDOI
Matrix Differential Calculus with Applications in Statistics and Econometrics
Journal ArticleDOI
Demixed principal component analysis of neural population data
Dmitry Kobak,Wieland Brendel,Wieland Brendel,Christos Constantinidis,Claudia E. Feierstein,Adam Kepecs,Zachary F. Mainen,Xue-Lian Qi,Ranulfo Romo,Naoshige Uchida,Christian K. Machens +10 more
TL;DR: A new dimensionality reduction technique, demixed principal component analysis (dPCA), that decomposes population activity into a few components and exposes the dependence of the neural representation on task parameters such as stimuli, decisions, or rewards is demonstrated.
Journal Article
Linear dimensionality reduction: survey, insights, and generalizations
TL;DR: This survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.
Journal ArticleDOI
Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1.
TL;DR: In this paper, the authors provide mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations.
Journal ArticleDOI
Hierarchical Aligned Cluster Analysis for Temporal Clustering of Human Motion
TL;DR: This work poses the problem of learning motion primitives as one of temporal clustering, and derives an unsupervised hierarchical bottom-up framework called hierarchical aligned cluster analysis (HACA), which finds a partition of a given multidimensional time series into m disjoint segments such that each segment belongs to one of k clusters.
References
More filters
Journal ArticleDOI
Distinctive Image Features from Scale-Invariant Keypoints
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Some methods for classification and analysis of multivariate observations
TL;DR: The k-means algorithm as mentioned in this paper partitions an N-dimensional population into k sets on the basis of a sample, which is a generalization of the ordinary sample mean, and it is shown to give partitions which are reasonably efficient in the sense of within-class variance.
Book
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
TL;DR: In this paper, the authors describe the important ideas in these areas in a common conceptual framework, and the emphasis is on concepts rather than mathematics, with a liberal use of color graphics.
Journal ArticleDOI
Pattern Recognition and Machine Learning
TL;DR: This book covers a broad range of topics for regular factorial designs and presents all of the material in very mathematical fashion and will surely become an invaluable resource for researchers and graduate students doing research in the design of factorial experiments.
Related Papers (5)
Nonlinear dimensionality reduction by locally linear embedding.
Sam T. Roweis,Lawrence K. Saul +1 more