Competitive learning algorithms for vector quantization

An iterative descent algorithm based on a Lagrangian formulation for designing vector quantizers having minimum distortion subject to an entropy constraint is discussed. These entropy-constrained vector quantizers (ECVQs) can be used in tandem with variable-rate noiseless coding systems to provide locally optimal variable-rate block source coding with respect to a fidelity criterion. Experiments on sampled speech and on synthetic sources with memory indicate that for waveform coding at low rates (about 1 bit/sample) under the squared error distortion measure, about 1.6 dB improvement in the signal-to-noise ratio can be expected over the best scalar and lattice quantizers when block entropy-coded with block length 4. Even greater gains are made over other forms of entropy-coded vector quantizers. For pattern recognition, it is shown that the ECVQ algorithm is a generalization of the k-means and related algorithms for estimating cluster means, in that the ECVQ algorithm estimates the prior cluster probabilities as well. Experiments on multivariate Gaussian distributions show that for clustering problems involving classes with widely different priors, the ECVQ outperforms the k-means algorithm in both likelihood and probability of error. >

/pdf/entropy-constrained-vector-quantization-2djba6vhy4.pdf

Entropy-constrained vector quantization

An algorithm introduced by L. Breiman et al. (1984) in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved. These include variable-rate and minimum-entropy tree-structured vector quantization, minimum expected cost decision trees, variable-order Markov modeling, optimum bit allocation, and computer graphics and image processing using quadtrees. A concentration on the first of these and a detailed analysis of variable-rate tree-structured vector quantization are provided. It is found that variable-rate tree-structured vector quantization outperforms not only the fixed-rate variety but also full-search vector quantization. The successive approximation character of variable-rate tree-structured vector quantization permits it to degrade gracefully if the rate is reduced at the encoder. This has applications to the problem of buffer overflow. >

http://www.angelfire.com/electronic2/ricardo_neri/00032124.pdf

Optimal pruning with applications to tree-structured source coding and modeling

The past decade has witnessed substantial progress towards the application of low-rate speech coders to civilian and military communications as well as computer-related voice applications. Central to this progress has been the development of new speech coders capable of producing high-quality speech at low data rates. Most of these coders incorporate mechanisms to: represent the spectral properties of speech, provide for speech waveform matching, and "optimize" the coder's performance for the human ear. A number of these coders have already been adopted in national and international cellular telephony standards. The objective of this paper is to provide a tutorial overview of speech coding methodologies with emphasis on those algorithms that are part of the recent low-rate standards for cellular communications. Although the emphasis is on the new low-rate coders, we attempt to provide a comprehensive survey by covering some of the traditional methodologies as well. We feel that this approach will not only point out key references but will also provide valuable background to the beginner. The paper starts with a historical perspective and continues with a brief discussion on the speech properties and performance measures. We then proceed with descriptions of waveform coders, sinusoidal transform coders, linear predictive vocoders, and analysis-by-synthesis linear predictive coders. Finally, we present concluding remarks followed by a discussion of opportunities for future research. >

Speech coding: a tutorial review

The vector sum excited linear prediction speech coder is presented. It utilizes a codebook with a structure that allows for a very efficient search procedure. Other advantages of the VSELP codebook structure are discussed, and a detailed description of an 8-kb/s VSELP coder is given. This coder was selected by the Telecommunications Industry Association (TIA) as the standard for use in North American digital cellular telephone systems. The coder uses two VSELP excitation codebooks, a gain quantizer which is robust to channel errors, and a novel adaptive pre/postfilter arrangement. >

/pdf/vector-sum-excited-linear-prediction-vselp-speech-coding-at-11u76d3hnr.pdf

Vector sum excited linear prediction (VSELP) speech coding at 8 kbps

Vector Excitation Coding (VXC) is based on a new and general source-filter modeling technique in which the excitation signal for a speech production model is encoded at very low bit-rates using vector quantization. Various speech coder structures which fall into this class have recently been shown to reproduce speech with very high perceptual quality. The primary drawback of VXC is the large amount of computation required in the process of selecting an optimal excitation signal. We present several schemes in this paper which substantially reduce search computation in VXC coders while retaining their remarkably high reconstructed speech quality.

Complexity reduction methods for vector excitation coding

A family of architectural techniques are proposed which offer efficient computation of weighted Euclidean distance measures for nearest-neighbor codebook searching. The general approach uses a single metric comparator chip in conjunction with a linear array of inner product processor chips. Very high vector-quantization (VQ) throughout can be achieved for many speech and image-processing applications. Several alternative configurations allow reasonable tradeoffs between speed and VLSI chip area required. >

Systolic architectures for vector quantization

In Vector Excitation Coding (VXC), speech is represented by applying a sequence of excitation vectors to a time-varying speech production filter with each vector chosen from a codebook using a perceptually-based performance measure. Although VXC is a powerful technique for achieving natural and high quality speech compression at low bit-rates, it suffers as other excitation coders do from a very high computational complexity. Recent research has shown that codebook search computation can be reduced to approximately 40 MFlops without compromising speech quality. However, this operation count still prohibits a practical real-time implementation of the coder using today's DSP chips. We present a real-time 4.8 kb/s Pulse Excitation VXC coder (PVXC) which achieves high reconstructed speech quality and incorporates new techniques which reduce the codebook search complexity to only 0.55 MFlops. The coder utilizes an optimized excitation codebook and a promising new interframe vector predictive LPC parameter quantization scheme. A preliminary implementation using a single floating-point signal processor is described.

Real-time vector excitation coding of speech at 4800 bps

An efficient, low-complexity method called switched-adaptive interframe vector prediction (SIVP) has been developed for linear predictive coding (LPC) of spectral parameters in the development of low-bit-rate speech coding systems. SIVP utilizes vector linear prediction to exploit the high frame-to-frame redundancy present in the successive frames of LPC parameters. When SIVP is combined with scalar quantization, it has been found that the LPC parameter bit rate required to achieve high-quality synthetic speech is only 1300 b/s. With vector quantization, the bit-rate can be reduced even further (to 1000 b/s) without introducing any perceivable quantization noise in the reconstructed speech. >

Encoding of LPC spectral parameters using switched-adaptive interframe vector prediction (speech coding)

An approach to vector-excitation-coding (VXC) speech compression utilizing multiple-stage vector quantization (VQ) is considered. Called multiple-stage VXC (MSVXC), this technique facilitates the use of high-dimensional excitation vectors at medium-band rates without substantially increasing computation. The basic approach consists of successively approximating the input speech vector in several cascaded VQ stages, where the input vector for each stage is the quantization error vector from the preceding stage. It is shown that if a number of VQ stages is increased sufficiently, MSVXC can be expressed as a form of transform coding, in which the computationally intensive excitation codebook search is completely eliminated. >

G. Davidson

Papers

Complexity reduction methods for vector excitation coding

Systolic architectures for vector quantization

Real-time vector excitation coding of speech at 4800 bps

Encoding of LPC spectral parameters using switched-adaptive interframe vector prediction (speech coding)

Multiple-stage vector excitation coding of speech waveforms