G. David Forney

Bio: G. David Forney is an academic researcher from Massachusetts Institute of Technology. The author has contributed to research in topics: Linear code & Convolutional code. The author has an hindex of 17, co-authored 40 publications receiving 1539 citations. Previous affiliations of G. David Forney include Codex Corporation.

Convolutional codes II. Maximum-likelihood decoding

Abstract: Convolutional codes are characterized by a trellis structure. Maximum-likelihood decoding is characterized as the finding of the shortest path through the code trellis, an efficient solution for which is the Viterbi algorithm. A universal asymptotic bounding technique is developed and used to bound error probability, free distance, list-of-2 error probability, and other subsidiary quantities. The bounds are dominated by what happens at a certain critical length N erit . Termination of a convolutional code to length N erit or shorter results in an optimum block code. In general, block code exponents can be related to convolutional code exponents and vice versa by a graphical construction, called the concatenation construction. It is shown that termination is unnecessary with the Viterbi algorithm.

On the role of MMSE estimation in approaching the information-theoretic limits of linear Gaussian channels: Shannon meets Wiener

Abstract: We discuss why MMSE estimation arises in lattice-based schemes for approaching the capacity of linear Gaussian channels, and comment on its properties

Lattice and trellis-coded quantization

Abstract: Vectors are quantized by representing the vectors by quantized values using a codebook having granular regions which are based on a coset code, and which lie inside a boundary region which is based on another coset code, the boundary region being other than an N-cube and other than the Voronoi region of a sublattice Λ b =MΛ g of a lattic Λ g upon which the granular regions may have been based.

On the Effective Weights of Pseudocodewords for Codes Defined on Graphs with Cycles

Abstract: The behavior of an iterative decoding algorithm for a code defined on a graph with cycles and a given decoding schedule is characterized by a cycle-free computation tree. The pseudocodewords of such a tree are the words that satisfy all tree constraintsj pseudocodewords govern decoding performance. Wiberg [12] determined the effective weight of pseudocodewords for binary codewords on an AWGN channel. This paper extends Wiberg’s formula for AWGN channels to nonbinary codes, develops similar results for BSC and BEC channels, and gives upper and lower bounds on the effective weight. The 16-state tail-biting trellis of the Golay code [2] is used for exampIes. Although in this case no pseudocodeword is found with effective weight less than the minimum Hamming weight of the Golay code on an AWGN channel, it is shown by example that the minimum effective pseudocodeword weight can be less than the minimum codeword weight.

Convolutional codes III. Sequential decoding

Abstract: Sequential decoding is characterized as a sequential search for the shortest path through a trellis. An easily analyzed algorithm closely related to the stack and Fano algorithms is described. Martingale techniques are used to find the distribution of computation on totally symmetric channels. For general channels, our universal bounding technique yields the well-known Pareto distribution of computation, as well as a bound on error probability that is asymptotically optimum in the high-rate range. The performance-complexity relation is shown to be asymptotically Pareto for both sequential and maximum-likelihood (Viterbi) decoding, with the same exponent in either case. A semisequential list-of-L Viterbi algorithm is introduced to extend the analogies below Rcomp.

Graphical Models, Exponential Families, and Variational Inference

Abstract: The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide variety of algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

Linear systems

Abstract: Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

Intelligent Automated Assistant

Abstract: An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.