Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0<ples1. The N most important coefficients in that expansion allow reconstruction with lscr2 error O(N1/2-1p/). It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients. Moreover, a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing. The nonadaptive measurements have the character of "random" linear combinations of basis/frame elements. Our results use the notions of optimal recovery, of n-widths, and information-based complexity. We estimate the Gel'fand n-widths of lscrp balls in high-dimensional Euclidean space in the case 0<ples1, and give a criterion identifying near- optimal subspaces for Gel'fand n-widths. We show that "most" subspaces are near-optimal, and show that convex optimization (Basis Pursuit) is a near-optimal way to extract information derived from these near-optimal subspaces

Compressed sensing

TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. Tensor-Flow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state. It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices, including multicore CPUs, general-purpose GPUs, and custom-designed ASICs known as Tensor Processing Units (TPUs). This architecture gives flexibility to the application developer: whereas in previous "parameter server" designs the management of shared state is built into the system, TensorFlow enables developers to experiment with novel optimizations and training algorithms. TensorFlow supports a variety of applications, with a focus on training and inference on deep neural networks. Several Google services use TensorFlow in production, we have released it as an open-source project, and it has become widely used for machine learning research. In this paper, we describe the TensorFlow dataflow model and demonstrate the compelling performance that TensorFlow achieves for several real-world applications.

/pdf/tensorflow-a-system-for-large-scale-machine-learning-10wgdjhl6k.pdf

TensorFlow: a system for large-scale machine learning

TensorFlow is an interface for expressing machine learning algorithms, and an
implementation for executing such algorithms. A computation expressed using
TensorFlow can be executed with little or no change on a wide variety of
heterogeneous systems, ranging from mobile devices such as phones and tablets
up to large-scale distributed systems of hundreds of machines and thousands of
computational devices such as GPU cards. The system is flexible and can be used
to express a wide variety of algorithms, including training and inference
algorithms for deep neural network models, and it has been used for conducting
research and for deploying machine learning systems into production across more
than a dozen areas of computer science and other fields, including speech
recognition, computer vision, robotics, information retrieval, natural language
processing, geographic information extraction, and computational drug
discovery. This paper describes the TensorFlow interface and an implementation
of that interface that we have built at Google. The TensorFlow API and a
reference implementation were released as an open-source package under the
Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.

/pdf/tensorflow-large-scale-machine-learning-on-heterogeneous-2yk4fgchw6.pdf

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Donoho and Johnstone (1994) proposed a method for reconstructing an unknown function f on [0,1] from noisy data d/sub i/=f(t/sub i/)+/spl sigma/z/sub i/, i=0, ..., n-1,t/sub i/=i/n, where the z/sub i/ are independent and identically distributed standard Gaussian random variables. The reconstruction f/spl circ/*/sub n/ is defined in the wavelet domain by translating all the empirical wavelet coefficients of d toward 0 by an amount /spl sigma//spl middot//spl radic/(2log (n)/n). The authors prove two results about this type of estimator. [Smooth]: with high probability f/spl circ/*/sub n/ is at least as smooth as f, in any of a wide variety of smoothness measures. [Adapt]: the estimator comes nearly as close in mean square to f as any measurable estimator can come, uniformly over balls in each of two broad scales of smoothness classes. These two properties are unprecedented in several ways. The present proof of these results develops new facts about abstract statistical inference and its connection with an optimal recovery model. >

De-noising by soft-thresholding

The concept of an agent has become important in both Artificial Intelligence (AI) and mainstream computer science. Our aim in this paper is to point the reader at what we perceive to be the most important theoretical and practical issues associated with the design and construction of intelligent agents. For convenience, we divide these issues into three areas (though as the reader will see, the divisions are at times somewhat arbitrary). Agent theory is concerned with the question of what an agent is, and the use of mathematical formalisms for representing and reasoning about the properties of agents. Agent architectures can be thought of as software engineering models of agents;researchers in this area are primarily concerned with the problem of designing software or hardware systems that will satisfy the properties specified by agent theorists. Finally, agent languages are software systems for programming and experimenting with agents; these languages may embody principles proposed by theorists. The paper is not intended to serve as a tutorial introduction to all the issues mentioned; we hope instead simply to identify the most important issues, and point to work that elaborates on them. The article includes a short review of current and potential applications of agent technology.

/pdf/intelligent-agents-theory-and-practice-5dchj7p8pc.pdf

Intelligent Agents: Theory and Practice

General Preliminaries: 1.1 Introduction 1.2 Basic concepts and notations General Theorems on Iteration Functions: 2.1 The solution of a fixed-point problem 2.2 Linear and superlinear convergence 2.3 The iteration calculus The Mathematics of Difference Relations: 3.1 Convergence of difference inequalities 3.2 A theorem on the solutions of certain inhomogeneous difference equations 3.3 On the roots of certain indicial equations 3.4 The asymptotic behavior of the solutions of certain difference equations Interpolatory Iteration Functions: 4.1 Interpolation and the solution of equations 4.2 The order of interpolatory iteration functions 4.3 Examples One-Point Iteration Functions: 5.1 The basic sequence $E_s$ 5.2 Rational approximations to $E_s$ 5.3 A basic sequence of iteration functions generated by direct interpolation 5.4 The fundamental theorem of one-point iteration functions 5.5 The coefficients of the error series of $E_s$ One-Point Iteration Functions With Memory: 6.1 Interpolatory iteration functions 6.2 Derivative-estimated one-point iteration functions with memory 6.3 Discussion of one-point iteration functions with memory Multiple Roots: 7.1 Introduction 7.2 The order of $E_s$ 7.3 The basic sequence $\scr{E}_s$ 7.4 The coefficients of the error series of $\scr{E}_s$ 7.5 Iteration functions generated by direct interpolation 7.6 One-point iteration functions with memory 7.7 Some general results 7.8 An iteration function of incommensurate order Multipoint Iteration Functions: 8.1 The advantages of multipoint iteration functions 8.2 A new interpolation problem 8.3 Recursively formed iteration functions 8.4 Multipoint iteration functions generated by derivative estimation 8.5 Multipoint iteration functions generated by composition 8.6 Multipoint iteration functions with memory Multipoint Iteration Functions: Continuation: 9.1 Introduction 9.2 Multipoint iteration functions of type 1 9.3 Multipoint iteration functions of type 2 9.4 Discussion of criteria for the selection of an iteration function Iteration Functions Which Require No Evaluation of Derivatives: 10.1 Introduction 10.2 Interpolatory iteration functions 10.3 Some additional iteration functions Systems of Equations: 11.1 Introduction 11.2 The generation of vector-valued iteration functions by inverse interpolation 11.3 Error estimates for some vector-valued iteration functions 11.4 Vector-valued iteration functions which require no derivative evaluations A Compilation of Iteration Functions: 12.1 Introduction 12.2 One-point iteration functions 12.3 One-point iteration functions with memory 12.4 Multiple roots 12.5 Multipoint iteration functions 12.6 Multipoint iteration functions with memory 12.7 Systems of equations Appendices: A. Interpolation B. On the $j$th derivative of the inverse function C. Significant figures and computational efficiency D. Acceleration of convergence E. Numerical examples F. Areas for future research Bibliography Index.

Iterative Methods for the Solution of Equations

The problem is to calculate a simple zero of a nonlinear function ƒ by iteration. There is exhibited a family of iterations of order 2n-1 which use n evaluations of ƒ and no derivative evaluations, as well as a second family of iterations of order 2n-1 based on n — 1 evaluations of ƒ and one of ƒ′. In particular, with four evaluations an iteration of eighth order is constructed. The best previous result for four evaluations was fifth order.It is proved that the optimal order of one general class of multipoint iterations is 2n-1 and that an upper bound on the order of a multipoint iteration based on n evaluations of ƒ (no derivatives) is 2n.It is conjectured that a multipoint iteration without memory based on n evaluations has optimal order 2n-1.

Optimal Order of One-Point and Multipoint Iteration

Information-based complexity seeks to develop general results about the intrinsic difficulty of solving problems where available information is partial or approximate and to apply these results to specific problems. This allows one to determine what is meant by an optimal algorithm in many practical situations, and offers a variety of interesting and sometimes surprising theoretical results.

/pdf/information-based-complexity-1ikpws7nfc.pdf

Information-based complexity

By reading, you can know the knowledge and things more, not only about what you get from people to people. Book will be more trusted. As this a general theory of optimal algorithms, it will really give you the good idea to be successful. It is not only for you to be success in certain life you can be successful in everything. The success can be started by knowing the basic knowledge and do actions.

A General Theory of Optimal Algorithms

High-dimensional integrals are usually solved with Monte Carlo algorithms although theory suggests that low-discrepancy algorithms are sometimes superior. We report on numerical testing which compares low-discrepancy and Monte Carlo algorithms on the evaluation of financial derivatives. The testing is performed on a Collateralized Mortgage Obligation (CMO) which is formulated as the computation of ten integrals of dimension up to 360. We tested two low-discrepancy algorithms (Sobol and Halton) and two randomized algorithms (classical Monte Carlo and Monte Carlo combined with antithetic variables). We conclude that for this CMO the Sobol algorithm is always superior to the other algorithms. We believe that it will be advantageous to use the Sobol algorithm for many other types of financial derivatives. Our conclusion regarding the superiority of the Sobol algorithm also holds when a rather small number of sample points are used, an important case in practice. We have built a software system called FINDER for computing high-dimensional integrals. FINDER runs on a heterogeneous network of workstations under PVM 3.2 (Parallel Virtual Machine). Since workstations are ubiquitous, this is a cost-effect way to do large computations fast. The measured speedup is at least .9N for $N$ workstations, $N$ less than or equal to 25. The software can also be used to compute high-dimensional integrals on a single workstation. A routine for generating Sobol points may be found, for example, in "Numerical Recipes in C" by Press et al. However, we incorporated major improvements in FINDER and we stress that the results reported in this paper were obtained using FINDER. One of the improvements was developing the table of primitive polynomials and initial direction numbers for dimensions up to 360.

Joseph F. Traub

Papers

Iterative Methods for the Solution of Equations

Optimal Order of One-Point and Multipoint Iteration

Information-based complexity

A General Theory of Optimal Algorithms

Faster valuation of financial derivatives