Approximation and learning by greedy algorithms
TLDR
In this article, the authors consider the problem of approximating a given element f from a Hilbert space by means of greedy algorithms and the application of such procedures to the regression problem in statistical learning theory.Abstract:
We consider the problem of approximating a given element f from a Hilbert space $\mathcal{H}$ by means of greedy algorithms and the application of such procedures to the regression problem in statistical learning theory. We improve on the existing theory of convergence rates for both the orthogonal greedy algorithm and the relaxed greedy algorithm, as well as for the forward stepwise projection algorithm. For all these algorithms, we prove convergence results for a variety of function classes and not simply those that are related to the convex hull of the dictionary. We then show how these bounds for convergence rates lead to a new theory for the performance of greedy algorithms in learning. In particular, we build upon the results in [IEEE Trans. Inform. Theory 42 (1996) 2118–2132] to construct learning algorithms based on greedy approximations which are universally consistent and provide provable convergence rates for large classes of functions. The use of greedy algorithms in the context of learning is very appealing since it greatly reduces the computational burden when compared with standard model selection using general dictionaries.read more
Citations
More filters
Journal ArticleDOI
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
TL;DR: The aim of this paper is to introduce a few key notions and applications connected to sparsity, targeting newcomers interested in either the mathematical aspects of this area or its applications.
Journal ArticleDOI
Sure independence screening for ultrahigh dimensional feature space
Jianqing Fan,Jinchi Lv +1 more
TL;DR: In this article, the authors introduce the concept of sure screening and propose a sure screening method that is based on correlation learning, called sure independence screening, to reduce dimensionality from high to a moderate scale that is below the sample size.
Posted Content
Sure Independence Screening for Ultra-High Dimensional Feature Space
Jianqing Fan,Jinchi Lv +1 more
TL;DR: The concept of sure screening is introduced and a sure screening method that is based on correlation learning, called sure independence screening, is proposed to reduce dimensionality from high to a moderate scale that is below the sample size.
Journal ArticleDOI
Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise
T. Tony Cai,Lie Wang +1 more
TL;DR: It is shown that under conditions on the mutual incoherence and the minimum magnitude of the nonzero components of the signal, the support of the signals can be recovered exactly by the OMP algorithm with high probability.
Journal ArticleDOI
Computational Methods for Sparse Solution of Linear Inverse Problems
Joel A. Tropp,Stephen J. Wright +1 more
TL;DR: This paper surveys the major practical algorithms for sparse approximation with specific attention to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available.
References
More filters
Book
Interpolation of operators
C. Bennett,M Sharpley +1 more
TL;DR: In this article, the classical interpolation theorem is extended to the Banach Function Spaces, and the K-Method is used to find a Banach function space with a constant number of operators.
Book
A Distribution-Free Theory of Nonparametric Regression
TL;DR: How to Construct Nonparametric Regression Estimates * Lower Bounds * Partitioning Estimates * Kernel Estimates * k-NN Estimates * Splitting the Sample * Cross Validation * Uniform Laws of Large Numbers
Journal ArticleDOI
Wavelet Shrinkage: Asymptopia?
TL;DR: A method for curve estimation based on n noisy data: translate the empirical wavelet coefficients towards the origin by an amount √(2 log n) /√n and draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves.
Journal ArticleDOI
Adaptive greedy approximations
TL;DR: A notion of the coherence of a signal with respect to a dictionary is derived from the characterization of the approximation errors of a pursuit from their statistical properties, which can be obtained from the invariant measure of the pursuit.
Related Papers (5)
Least angle regression
Bradley Efron,Trevor Hastie,Iain M. Johnstone,Robert Tibshirani,Hemant Ishwaran,Keith Knight,Jean-Michel Loubes,Jean-Michel Loubes,Pascal Massart,Pascal Massart,David Madigan,David Madigan,Greg Ridgeway,Greg Ridgeway,Saharon Rosset,Saharon Rosset,Ji Zhu,Robert A. Stine,Berwin A. Turlach,Sanford Weisberg +19 more