Sparse approximation

About: Sparse approximation is a research topic. Over the lifetime, 18037 publications have been published within this topic receiving 497739 citations. The topic is also known as: Sparse approximation.

Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets

Abstract: A sparse representation of Support Vector Machines (SVMs) with respect to input features is desirable for many applications. In this paper, by introducing a 0-1 control variable to each input feature, l0-norm Sparse SVM (SSVM) is converted to a mixed integer programming (MIP) problem. Rather than directly solving this MIP, we propose an efficient cutting plane algorithm combining with multiple kernel learning to solve its convex relaxation. A global convergence proof for our method is also presented. Comprehensive experimental results on one synthetic and 10 real world datasets show that our proposed method can obtain better or competitive performance compared with existing SVM-based feature selection methods in term of sparsity and generalization performance. Moreover, our proposed method can effectively handle large-scale and extremely high dimensional problems.

Forward sequential algorithms for best basis selection

Abstract: The problem of signal representation in terms of basis vectors from a large, over-complete, spanning dictionary has been the focus of much research. Achieving a succinct, or 'sparse', representation is known as the problem of best basis representation. Methods are considered which seek to solve this problem by sequentially building up a basis set for the signal. Three distinct algorithm types have appeared in the literature which are here termed basic matching pursuit (BMP), order recursive matching pursuit (ORMP) and modified matching pursuit (MMP). The algorithms are first described and then their computation is closely examined. Modifications are made to each of the procedures which improve their computational efficiency. The complexity of each algorithm is considered in two contexts; one where the dictionary is variable (time-dependent) and the other where the dictionary is fixed (time-independent). Experimental results are presented which demonstrate that the ORMP method is the best procedure in terms of its ability to give the most compact signal representation, followed by MMP and then BMP which gives the poorest results. Finally, weighing the performance of each algorithm, its computational complexity and the type of dictionary available, recommendations are made as to which algorithm should be used for a given problem.