Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

A digital computer is generally believed to be an efficient universal computing device; that is, it is believed able to simulate any physical computing device with an increase in computation time by at most a polynomial factor. This may not be true when quantum mechanics is taken into consideration. This paper considers factoring integers and finding discrete logarithms, two problems which are generally thought to be hard on a classical computer and which have been used as the basis of several proposed cryptosystems. Efficient randomized algorithms are given for these two problems on a hypothetical quantum computer. These algorithms take a number of steps polynomial in the input size, e.g., the number of digits of the integer to be factored.

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer

We present a novel approach to measuring similarity between shapes and exploit it for object recognition. In our framework, the measurement of similarity is preceded by: (1) solving for correspondences between points on the two shapes; (2) using the correspondences to estimate an aligning transform. In order to solve the correspondence problem, we attach a descriptor, the shape context, to each point. The shape context at a reference point captures the distribution of the remaining points relative to it, thus offering a globally discriminative characterization. Corresponding points on two similar shapes will have similar shape contexts, enabling us to solve for correspondences as an optimal assignment problem. Given the point correspondences, we estimate the transformation that best aligns the two shapes; regularized thin-plate splines provide a flexible class of transformation maps for this purpose. The dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points, together with a term measuring the magnitude of the aligning transform. We treat recognition in a nearest-neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image. Results are presented for silhouettes, trademarks, handwritten digits, and the COIL data set.

Shape matching and object recognition using shape contexts

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

: This research project involved the development of mathematical models for analysis, synthesis, and simulation of large systems of interacting devices. The work was motivated by problems that may become important in high density VLSI chips with characteristic feature sizes less than 1 micron: it is anticipated that interactions of neighboring devices will play an important role in the determination of circuit properties. It is hoped that the combination of high device densities and such local interactions can somehow be exploited to increase circuit speed and to reduce power consumption. To address these issues from the point of view of system theory, research was pursued in the areas of nonlinear and stochastic systems and into neural network models. Statistical models were developed to characterize various features of the dynamic behavior of interacting systems. Random process models for studying the resulting asynchronous modes of operation were investigated. The local interactions themselves may be modeled as stochastic effects. The resulting behavior has been investigated through the use of various scaling limits, and by a combination of other analytical and simulation techniques. Techniques arising in a variety of disciplines where models of interaction have been formulated and explored were considered and adapted for use.

System Theoretic Models for High Density VLSI Structures

A linear programming algorithm is described for designing circulant Markov chains which generate random waveforms whose spectral densities have specified poles. These chains can be implemented by random jumping in a fixed table, and have the advantages of speed and simplicity. Possible applications to speech synthesis are discussed.

http://ieeexplore.ieee.org/iel6/29/26121/01162874.pdf

The design of Markov chains for waveform generation

We describe the implementation of a practical system for pole-zero analysis-synthesis which produces synthesized speech that sounds qualitatively different from linear prediction at points of mouth closure. We also describe an apparently new synthesis method, windowed synthesis, which circumvents the problem of choosing initial conditions in synthesis using a filter model, and is also applicable to cepstral vocoding.

Implementation of a pole-zero analysis - synthesis system for speech

The paper compares two parallel architectures in terms of throughput versus cost. Throughput is estimated using machine parameters extracted from detailed models of each architecture. Cost models are used to express total resource use in terms of a common unit. Performance of an architecture is then evaluated by optimizing throughput for each possible cost. Finally, these throughput-versus-cost results for the two architectures are compared for a particular class of problems: iterative computations on a 2-dimensional grid. The results show that there is a cost below which one architecture is an order of magnitude faster than the other, and above which this relationship is reversed. This approach may prove useful as a general comparison methodology. >

Comparing architectures using throughput-versus-cost modeling

We consider transmission of data over multiple coupled channels, such as bundles of twisted-pair cables in the local subscriber loop, and between central offices in the public switched telephone network. Transceiver designs for such channels typically treat the crosstalk between adjacent cables as random noise uncorrelated with the transmitted signal. We propose a transmitter/receiver pair that compensates for crosstalk by treating an entire bundle of cables as a single multi-input/multi-output channel with a (slowly varying) matriz transfer function. One attribute of the proposed transceiver is the use of a multi-channel adaptive FIR filter to cancel near-end crosstalk. Results of numerical simulations, including plots of mean squared error vs. time and eye diagrams, are presented assuming a standard transmission line model for the channel. These results indicate that data rates over coupled channels can be significantly increased by exploiting the multi-dimensional character of the channel.

https://ieeexplore.ieee.org/iel2/723/5035/00196985.pdf

Kenneth Steiglitz

Papers

System Theoretic Models for High Density VLSI Structures

The design of Markov chains for waveform generation

Implementation of a pole-zero analysis - synthesis system for speech

Comparing architectures using throughput-versus-cost modeling

D11.2 multi-channel signal processing for dat communications in the presence of crosstalk p