This tutorial provides an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and gives practical details on methods of implementation of the theory along with a description of selected applications of the theory to distinct problems in speech recognition. Results from a number of original sources are combined to provide a single source of acquiring the background required to pursue further this area of research. The author first reviews the theory of discrete Markov chains and shows how the concept of hidden states, where the observation is a probabilistic function of the state, can be used effectively. The theory is illustrated with two simple examples, namely coin-tossing, and the classic balls-in-urns system. Three fundamental problems of HMMs are noted and several practical techniques for solving these problems are given. The various types of HMMs that have been studied, including ergodic as well as left-right models, are described. >

/pdf/a-tutorial-on-hidden-markov-models-and-selected-applications-1vlzbxbk68.pdf

A tutorial on hidden Markov models and selected applications in speech recognition

Increasingly, for many application areas, it is becoming important to include elements of nonlinearity and non-Gaussianity in order to model accurately the underlying dynamics of a physical system. Moreover, it is typically crucial to process data on-line as it arrives, both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics. In this paper, we review both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model and which generalize the traditional Kalman filtering methods. Several variants of the particle filter such as SIR, ASIR, and RPF are introduced within a generic framework of the sequential importance sampling (SIS) algorithm. These are discussed and compared with the standard EKF through an illustrative example.

/pdf/a-tutorial-on-particle-filters-for-online-nonlinear-non-1rzkwgk2pp.pdf

A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking

Wireless Communications

Alhussein Abouzeid Rensselaer Polytechnic Institute Raviraj Adve University of Toronto Dharma Agrawal University of Cincinnati Walid Ahmed Tyco M/A-COM Sonia Aissa University of Quebec, INRSEMT Huseyin Arslan University of South Florida Nallanathan Arumugam National University of Singapore Saewoong Bahk Seoul National University Claus Bauer Dolby Laboratories Brahim Bensaou Hong Kong University of Science and Technology Rick Blum Lehigh University Michael Buehrer Virginia Tech Antonio Capone Politecnico di Milano Javier Gómez Castellanos National University of Mexico Claude Castelluccia INRIA Henry Chan The Hong Kong Polytechnic University Ajit Chaturvedi Indian Institute of Technology Kanpur Jyh-Cheng Chen National Tsing Hua University Yong Huat Chew Institute for Infocomm Research Tricia Chigan Michigan Tech Dong-Ho Cho Korea Advanced Institute of Science and Tech. Jinho Choi University of New South Wales Carlos Cordeiro Philips Research USA Laurie Cuthbert Queen Mary University of London Arek Dadej University of South Australia Sajal Das University of Texas at Arlington Franco Davoli DIST University of Genoa Xiaodai Dong, University of Alberta Hassan El-sallabi Helsinki University of Technology Ozgur Ercetin Sabanci University Elza Erkip Polytechnic University Romano Fantacci University of Florence Frank Fitzek Aalborg University Mario Freire University of Beira Interior Vincent Gaudet University of Alberta Jairo Gutierrez University of Auckland Michael Hadjitheodosiou University of Maryland Zhu Han University of Maryland College Park Christian Hartmann Technische Universitat Munchen Hossam Hassanein Queen's University Soong Boon Hee Nanyang Technological University Paul Ho Simon Fraser University Antonio Iera University "Mediterranea" of Reggio Calabria Markku Juntti University of Oulu Stefan Kaiser DoCoMo Euro-Labs Nei Kato Tohoku University Dongkyun Kim Kyungpook National University Ryuji Kohno Yokohama National University Bhaskar Krishnamachari University of Southern California Giridhar Krishnamurthy Indian Institute of Technology Madras Lutz Lampe University of British Columbia Bjorn Landfeldt The University of Sydney Peter Langendoerfer IHP Microelectronics Technologies Eddie Law Ryerson University in Toronto

Wireless communications

Optimal decoding of linear codes for minimizing symbol error rate

The Viterbi algorithm (VA) is a recursive optimal solution to the problem of estimating the state sequence of a discrete-time finite-state Markov process observed in memoryless noise. Many problems in areas such as digital communications can be cast in this form. This paper gives a tutorial exposition of the algorithm and of how it is implemented and analyzed. Applications to date are reviewed. Increasing use of the algorithm in a widening variety of areas is foreseen.

The viterbi algorithm

A convolutional encoder is defined as any constant linear sequential circuit. The associated code is the set of all output sequences resulting from any set of input sequences beginning at any time. Encoders are called equivalent if they generate the same code. The invariant factor theorem is used to determine when a convolutional encoder has a feedback-free inverse, and the minimum delay of any inverse. All encoders are shown to be equivalent to minimal encoders, which are feedback-free encoders with feedback-free delay-free inverses, and which can be realized in the conventional manner with as few memory elements as any equivalent encoder, Minimal encoders are shown to be immune to catastrophic error propagation and, in fact, to lead in a certain sense to the shortest decoded error sequences possible per error event. In two appendices, we introduce dual codes and syndromes, and show that a minimal encoder for a dual code has exactly the complexity of the original encoder; we show that systematic encoders with feedback form a canonical class, and compare this class to the minimal class.

Convolutional codes I: Algebraic structure

For pt.I see ibid., vol.34, no.5, p.1123-51 (1988). The family of Barnes-Wall lattices (including D/sub 4/ and E/sub 8/) of lengths N=2/sup n/ and their principal sublattices, which are useful in constructing coset codes, are generated by iteration of a simple construction called the squaring construction. The closely related Reed-Muller codes are generated by the same construction. The principal properties of these codes and lattices are consequences of the general properties of iterated squaring constructions, which also exhibit the interrelationships between codes and lattices of different lengths. An extension called the cubing construction generates good codes and lattices of lengths N=3*2/sup n/, including the Golay code and Leech lattice, with the use of special bases for 8-space. Another related construction generates the Nordstrom-Robinson code and an analogous 16-dimensional nonlattice packing. These constructions are represented by trellis diagrams that display their structure and interrelationships and that lead to efficient maximum-likelihood decoding algorithms. >

Coset codes. II. Binary lattices and related codes

Practically all known good constructive coding techniques for bandlimited channels, including lattice codes and various trellis-coded modulation schemes, can be characterized as coset codes. A coset code is defined by a lattice partition Lambda / Lambda ' and by a binary encoder C that selects a sequence of cosets of the lattice Lambda '. The fundamental coding gain of a coset code, as well as other important parameters such as the error coefficient, the decoding complexity, and the constellation expansion factor, are purely geometric parameters determined by C Lambda / Lambda '. The known types of coset codes, as well as a number of new classes that systematize and generalize known codes, are classified and compared in terms of these parameters. >

/pdf/coset-codes-i-introduction-and-geometrical-classification-4gyi3ztt4l.pdf

Coset codes. I. Introduction and geometrical classification

The purpose of this paper is to organize and clarify the work of the past decade on burst-correcting codes. Our method is, first, to define an idealized model, called the classic bursty channel, toward which most burst-correcting schemes are explicitly or implicitly aimed; next, to bound the best possible performance on this channel; and, finally, to exhibit classes of schemes which are asymptotically optimum and serve as archetypes of the burstcorrecting codes actually in use. In this light we survey and categorize previous work on burst-correcting codes. Finally, we discuss qualitatively the ways in which real channels fail to satisfy the assumptions of the classic bursty channel, and the effects of such failures on the various types of burst-correcting schemes. We conclude by comparing forward-error-correction to the popular alternative of automatic repeat-request (ARQ).

Jr. G.D. Forney

Papers

The viterbi algorithm

Convolutional codes I: Algebraic structure

Coset codes. II. Binary lattices and related codes

Coset codes. I. Introduction and geometrical classification

Burst-Correcting Codes for the Classic Bursty Channel