SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.

/pdf/regression-shrinkage-and-selection-via-the-lasso-6mx35txfr5.pdf

Regression Shrinkage and Selection via the Lasso

We present a new technique called “t-SNE” that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images of objects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large datasets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of datasets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the datasets.

/pdf/visualizing-data-using-t-sne-2w1n304iq0.pdf

Visualizing Data using t-SNE

다중혈관 관상동맥 환자에서 y-문합을 이용하여 양쪽 내흉동맥만을 사용한 우회술의 조기 성적

Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0<ples1. The N most important coefficients in that expansion allow reconstruction with lscr2 error O(N1/2-1p/). It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients. Moreover, a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing. The nonadaptive measurements have the character of "random" linear combinations of basis/frame elements. Our results use the notions of optimal recovery, of n-widths, and information-based complexity. We estimate the Gel'fand n-widths of lscrp balls in high-dimensional Euclidean space in the case 0<ples1, and give a criterion identifying near- optimal subspaces for Gel'fand n-widths. We show that "most" subspaces are near-optimal, and show that convex optimization (Basis Pursuit) is a near-optimal way to extract information derived from these near-optimal subspaces

Compressed sensing

Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

http://ahvazdownload.persiangig.com/document/article/39.pdf

A wavelet tour of signal processing

Removal of noise from 2D and 3D datasets is a task frequently needed in applications typical examples including in magnetic resonance imaging, in seismic exploration, and in video processing. We are currently interested in visualizing macromolecular structures of biological specimen, in which slices of very noisy electron microscopy (EM) images are volume rendered. Volume rendering of those datasets without any denoising normally gives very 'foggy' results that are not very informative. The wavelet and image processing communities have proposed in the past decade various multiscale image representations, many of which are of potential use for image de-noising. One of our goals here is to explore the importance of translation and direction invariance to the quality of reconstruction, which leads us to study the use of various tight frames for image reconstruction. We have developed 2D translation invariant transforms for both the isotropic and anisotropic wavelet bases. These allow us to develop a 2D analog of the 1D translation invariant de-noising algorithm proposed by Coifman and Donoho. We have also developed algorithms for implementing directionally-invariant de-noising for digital images. We have experiments to measure the relative importance of translation- and direction-invariance for both isotropic and anisotropic transforms. We also are exploring how to apply tight frames for linear inversion of noisy indirect data, which is what ultimately is needed in EM tomography.

Translation- and direction-invariant denoising of 2D and 3D images: experience and algorithms

The problem of recovering an input signal form noisy and linearly distorted data arises in many different areas of scientific investigation; e.g., noisy Radon inversion is a problem of special interest and considerable practical relevance in medical imaging. We will argue that traditional methods for solving inverse problems - damping of the singular value decomposition or cognate methods - behave poorly when the object to recover has edges. We apply a new system of representation, namely the curvelets in this setting. Curvelets provide near-optimal representations of otherwise smooth objects with discontinuities along smooth C2 edges. Inspired by some recent work on nonlinear estimation, we construct a curvelet-based biorthogonal decomposition of the Radon operator and build a reconstruction based on the shrinkage of the noisy curvelet coefficients. This novel approach is shown to give a new theoretical understanding of the problem of edges in the Radon inversion problem.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Curvelets and reconstruction of images from noisy radon data

We consider the compressed sensing problem where the object x 0 ∈ ℝN is to be recovered from incomplete measurements y = Ax 0 +z. Here the sensing matrix A is an n×N random matrix with Gaussian entries and n &#60; N. A popular method of sparsity-promoting reconstruction is l1-penalized least-squares reconstruction (aka LASSO, Basis Pursuit).

/pdf/compressed-sensing-over-l-p-balls-minimax-mean-square-error-5aqthop3s3.pdf

Compressed Sensing over ℓ p -balls: Minimax mean square error

We discuss a method for curve estimation based on n noisy data; one translates the empirical wavelet coefficients towards the origin by an amount \( \sqrt {{2\log \left( n \right)}} \cdot \sigma /\sqrt {n}\) The method is nearly minimax for a wide variety of loss functions-e.g. pointwise error, global error measured in LP norms, pointwise and global error in estimation of derivatives—and for a wide range of smoothness classes, including standard Holder classes, Sobolev classes, and Bounded Variation. This is a broader near-optimality than anything previously proposed in the minimax literature. The theory underlying the method exploits a correspondence between statistical questions and questions of optimal recovery and information-based complexity. This paper contains a detailed proof of the result announced in Donoho, Johnstone, Kerkyacharian & Picard (1995).

/pdf/universal-near-minimaxity-of-wavelet-shrinkage-44jwrxvfa1.pdf

Universal Near Minimaxity of Wavelet Shrinkage

Scientific Computation is emerging as absolutely central to the scientific method. Unfortunately, it is error-prone and currently immature: traditional scientific publication is incapable of finding and rooting out errors in scientific computation; this must be recognized as a crisis. Reproducible computational research, in which the full computational environment that produces a result is published along with the article, is an important recent development, and a necessary response to this crisis. We have been practicing reproducible computational research for 15 years and integrated it with our scientific research, and with doctoral and postdoctoral education. In this article, we review our approach, how the approach has spread over time, and then discuss some of the arguments for and against working reproducibly.

David L. Donoho

Papers

Translation- and direction-invariant denoising of 2D and 3D images: experience and algorithms

Curvelets and reconstruction of images from noisy radon data

Compressed Sensing over ℓ p -balls: Minimax mean square error

Universal Near Minimaxity of Wavelet Shrinkage

15 Years of Reproducible Research in Computational Harmonic Analysis