Showing papers on "Codebook published in 1994"

A new initialization technique for generalized Lloyd iteration

Abstract: The generalized Lloyd algorithm plays an important role in the design of vector quantizers (VQ) and in feature clustering for pattern recognition. In the VQ context, this algorithm provides a procedure to iteratively improve a codebook and results in a local minimum that minimizes the average distortion function. We propose an efficient method to obtain a good initial codebook that can accelerate the convergence of the generalized Lloyd algorithm and achieve a better local minimum as well. >

Handwriting signal processing front-end for handwriting recognizers

Abstract: A handwriting signal processing front-end method and apparatus for a handwriting training and recognition system which includes non-uniform segmentation and feature extraction in combination with multiple vector quantization. In a training phase, digitized handwriting samples are partitioned into segments of unequal length. Features are extracted from the segments and are grouped to form feature vectors for each segment. Groups of adjacent from feature vectors are then combined to form input frames. Feature-specific vectors are formed by grouping features of the same type from each of the feature vectors within a frame. Multiple vector quantization is then performed on each feature-specific vector to statistically model the distributions of the vectors for each feature by identifying clusters of the vectors and determining the mean locations of the vectors in the clusters. Each mean location is represented by a codebook symbol and this information is stored in a codebook for each feature. These codebooks are then used to train a recognition system. In the testing phase, where the recognition system is to identify handwriting, digitized test handwriting is first processed as in the training phase to generate feature-specific vectors from input frames. Multiple vector quantization is then performed on each feature-specific vector to represent the feature-specific vector using the codebook symbols that were generated for that feature during training. The resulting series of codebook symbols effects a reduced representation of the sampled handwriting data and is used for subsequent handwriting recognition.

Technique for modifying reference vector quantized speech feature signals

Abstract: A signal processing arrangement uses a codebook of first vector quantized speech feature signals formed responsive to a large collection of speech feature signals. The codebook is altered by combining the first speech feature signals of the codebook with second speech feature signals generated responsive to later input speech patterns during normal speech processing. A speaker recognition template can be updated in this fashion to take account of change which may occur in the voice and speaking characteristics of a known speaker.

Signal processing for robust speech recognition

Abstract: This paper describes a series of cepstral-based compensation procedures that render the SPHINX-II system more robust with respect to acoustical environment. The first algorithm, phone-dependent cepstral compensation, is similar in concept to the previously-described MFCDCN method, except that cepstral compensation vectors are selected according to the current phonetic hypothesis, rather than on the basis of SNR or VQ codeword identity. We also describe two procedures to accomplish adaptation of the VQ codebook for new environments, as well as the use of reduced-bandwidth frequency analysis to process telephone-bandwidth speech. Use of the various compensation algorithms in consort produces a reduction of error rates for SPHINX-II by as much as 40 percent relative to the rate achieved with cepstral mean normalization alone, in both development test sets and in the context of the 1993 ARPA CSR evaluations.

Low rate multi-mode CELP codec that encodes line SPECTRAL frequencies utilizing an offset

Abstract: The present invention provides a multi-mode CELP encoding and decoding method and device for digitized speech signals providing improvements over prior art codecs and coding methods by selectively utilizes backward prediction for the short-term predictor parameters and fixed codebook gain of a speech signal. In order to achieve these improvements, the present invention provides a coding method comprising the steps of classifying a segment of the digitized speech signal as one of a plurality of predetermined modes, determining a set of unquantized line spectral frequencies to represent the short term predictor parameters for that segment, and quantizing the determined set of unquantized line spectral frequencies using a mode-specific combination of scalar quantization and vector quantization, which utilizes backward prediction for modes with voiced speech signals. Furthermore, backward prediction is selectively applied to the fixed codebook gain in the modes that are free of transients so that it may be used in the fixed codebook search and fixed codebook gain quantization in those modes.

Fast closest codeword search algorithms for vector quantisation

Abstract: One of the most serious problems for vector quantisation is the high computational complexity of searching for the closest codeword in the codebook design and encoding phases. The authors present a fast algorithm to search for the closest codeword. The proposed algorithm uses two significant features of a vector, mean value and variance, to reject many unlikely codewords and saves a great deal of computation time. Since the proposed algorithm rejects those codewords that are impossible to be the closest codeword, this algorithm introduces no extra distortion than conventional full search method. The results obtained confirm the effectiveness of the proposed algorithm.

An improvement on codebook search for vector quantization

Abstract: Presents a simple but effective algorithm to speed up the codebook search in a vector quantization scheme when a MSE criterion is used. A considerable reduction in the number of operations is achieved. This algorithm was originally designed for image vector quantization in which the samples of the image signal (pixels) are positive, although it can be used with any positive-negative signal with only minor modifications. >

Dynamic finite-state vector quantization of digital images

Abstract: A vector quantization (VQ) scheme with finite memory called dynamic finite-state vector quantization (DFSVQ) is presented The encoder consists of a large codebook, so called super-codebook, where for each input vector a fixed number of its codevectors are chosen to generate a much smaller codebook (sub-codebook) This sub-codebook represents the best matching codevectors that could be found in the super-codebook for encoding the current input vector The choice for the codevectors in the sub-codebook is based on the information obtained from the previously encoded blocks where directional conditional block probability (histogram) matrices are used in the selection of the codevectors The index of the best matching codevector in the sub-codebook is transmitted to the receiver An adaptive DFSVQ scheme is also proposed in which, when encoding an input vector, first the sub-codebook is searched for a matching codevector to satisfy a pre-specified waveform distortion If such a codevector is not found in tile current sub-codebook then the whole super-codebook is checked for a better match If a better match is found then a signaling flag along with the corresponding index of the codevector is transmitted to the receiver Both the DFSVQ encoder and its adaptive version are implemented Experimental results for several monochrome images with a super-codebook size of 256 or 512 and different sub-codebook sizes are presented >

Combined source-channel vector quantization using deterministic annealing

Abstract: The authors present a new approach to combined source-channel vector quantization. The method, derived within information theory and probability theory, utilizes deterministic annealing to avoid some local minima that trap conventional descent algorithms such as the generalized Lloyd algorithm. The resulting vector quantizers satisfy the necessary conditions for local optimality for the noisy channel case. They tested the method against several versions of the noisy channel generalized Lloyd algorithm on stationary, first order Gauss-Markov sources with a binary symmetric channel. The method outperformed other methods under all test conditions, with the gains over noisy channel GLA growing with the codebook size. The quantizers designed using deterministic annealing are also shown to behave robustly under channel mismatch conditions. As a comparison with a separate source-channel system, over a large range of test channel conditions, the method outperformed a bandwidth-equivalent system incorporating a Hamming code. Also, for severe channel conditions, the method produces solutions with explicit error control coding. >

Adaptive speech coder having code excited linear prediction with multiple codebook searches

Abstract: Methods and apparatus for determining codevectors in response to a speech signal including an adaptive-stochastic codebook search combination. Each stochastic codebook search is made up of BPC and SHC search components (124). The speech signal is used as the input to each of the two possible codebook searches, LTP-CB1 and CB0-CB1. The codebook target vector is computed at (120). The present invention determines when it is desirable to dispense with the adaptive LTP analysis (122) of the target vector and instead use the bits freed up by foregoing the LTP to add another codevector obtained from a second stochastic codebook to the modeling process. A first synthesized speech signal can be determined from the first and second codevectors and a second synthesized speech can be determined from the first and second codewords. The error between the synthesized and the input speech signals is computed (126), concurrently the SHC/BPC search for codebook is performed (128). The resultant vector is searched in the SHC/BPC (124) search for codebook 1 (130).

Method and apparatus for speech recognition using optimized partial mixture tying of HMM state functions

Abstract: In accordance with the invention, a speech recognizer is provided which uses a computationally-feasible method for constructing a set of Hidden Markov Models (HMMs) for speech recognition that utilize a partial and optimal degree of mixture tying. With partially-tied HMMs, improved recognition accuracy of a large vocabulary word corpus as compared to systems that use fully-tied HMMs is achieved with less computational overhead than with a fully untied system. The computationally-feasible technique comprises the steps of determining a cluster of HMM states that share Gaussian components which are close together, developing a subset codebook for those clusters, and recalculating the Gaussians in the codebook to best estimate the clustered states.

Improved techniques for single-pass adaptive vector quantization

Abstract: Constantinescu and Storer presented a new single-pass adaptive vector quantization algorithm that learns a codebook of variable size and shape entries; they presented experiments on a set of test images showing that with no training or prior knowledge of the data, for a given fidelity, the compression achieved typically equals or exceeds that of the JPEG standard. This paper presents improvements in speed (by employing K-D trees), simplicity of codebook entries, and visual quality with no loss in either the amount of compression or the SNR as compared to the original full-search version. >

Two-stage vector quantization-lattice vector quantization

Abstract: A two-stage vector quantizer is introduced that uses an unstructured first-stage codebook and a second-stage lattice codebook. Joint optimum two-stage encoding is accomplished by exhaustive search of the parent codebook of the two-stage product code. Due to the relative ease of lattice vector quantization, optimum encoding is feasible for moderate-to-large encoding rates and vector dimensions, provided the first-stage codebook size is kept reasonable. For memoryless Gaussian and Laplacian sources, encoding rates of 2 to 3 b/sample, and vector dimensions of 8 to 35 the signal-to-noise ratio performance is comparable or superior to equivalent-delay encoding results previously reported. For Gaussian sources with memory, the effectiveness of the encoding method is dependent on the feasibility of using a large enough first-stage vector quantizer codebook to exploit most of the source memory. >

A fast method of designing better codebooks for image vector quantization

Abstract: A maximum descent (MD) method is proposed as an alternative to the generalized Lloyd (LBG) algorithm for vector quantization codebook generation. Compared with the LBG algorithm, the MD algorithm requires far less computation and generates codebooks that are superior in quality. This paper investigates and compares the performances of various partition techniques in the MD algorithm. In image vector quantization, the MD algorithm can produce codebooks with about 1 dB improvement in peak signal to noise ratio in less than 1/100 of the time required by the LBG algorithm. >

Index assignment for progressive transmission of full-search vector quantization

Abstract: The authors study codeword index assignment to allow for progressive image transmission of fixed rate full-search vector quantization (VQ). They develop three new methods of assigning indices to a vector quantization codebook and formulate these assignments as labels of nodes of a full-search progressive transmission tree. The tree is used to design intermediate codewords for the decoder so that full-search VQ has a successive approximation character. The binary representation for the path through the tree represents the progressive transmission code. The methods of designing the tree that they apply are the generalized Lloyd algorithm, minimum cost perfect matching from optimization theory, and a method of principal component partitioning. Their empirical results show that the final method gives intermediate signal-to-noise ratios (SNRs) that are close to those obtained with tree-structured vector quantization, yet they have higher final SNRs. >

Vector quantization over a noisy channel using soft decision decoding

Abstract: A soft decision decoder is presented. The soft decision decoder is optimal in the mean square sense, if the encoder entropy is full. A source vector estimate is obtained as a linear mapping of a soft Hadamard column. The soft Hadamard column is formed as a generally nonlinear mapping of soft information bits. It is shown that the best index assignment, on the encoder, is obtained in the special case of a linear mapping from the soft information bits. Simulations indicate that the jointly trained system performs better than channel optimized VQ with hard decisions. The interesting case, for applications, of using an ordinary VQ codebook as encoder, together with our soft decision decoder, is also investigated. In our examples this approach gives comparable performance to channel optimized VQ with hard decisions. >

Variable-dimension vector quantization of speech spectra for low-rate vocoders

Abstract: Optimal vector quantization of variable-dimension vectors in principle is feasible by using a set of fixed dimension VQ codebooks. However, for typical applications, such a multi-codebook approach demands a grossly excessive and impractical storage and computational complexity. Efficient quantization of such variable-dimension spectral shape vectors is the most challenging and difficult encoding task required in an important family of low bit-rate vocoders. The authors introduce a simple and effective formulation of variable-dimension vector quantization (VDVQ) which quantizes variable-dimension vectors using a single universal codebook having fixed dimension yet covering the entire range of input vector dimensions under consideration. This VDVQ technique is applied to quantize variable-dimension spectral shape vectors leading to a high quality speech coder at the low bit-rate of 2.5 kb/s. The combination of a universal spectral codebook and structured VQ reduces storage and computational complexity, yet delivers a high quantization efficiency and enhanced perceptual quality of the coded speech. >

Universal source coding with codebook transmission

Abstract: A universal source coding system with vector quantizer codebook transmissions is studied using high resolution quantization theory. Conditions are derived for the optimal tradeoff between quantizer resolution and the information rate used to transmit codebooks. A formula that tightly bounds the mean squared error of the universal coding system as a function of the time between codebook transmissions is experimentally verified and found to be tight, and a new and simpler derivation is given. Other research in the literature has proposed vector quantizing the transmitted codebooks; one conclusion we prove here is that under some reasonable conditions uniform scalar quantization of the transmitted codebooks performs as well as vector quantizing them. Experimental results are given that support the analytic derivations. >

Vector quantization using tree-structured self-organizing feature maps

Abstract: In this paper, we propose a binary-tree structure neural network model suitable for structured clustering. During and after training, the centroids of the clusters in this model always form a binary tree in the input pattern space. This model is used to design tree search vector quantization codebooks for image coding. Simulation results show that the acquired codebook not only produces better-quality images but also achieves a higher compression ratio than conventional tree search vector quantization. When source coding is applied after VQ, the new model performs better than the generalized Lloyd algorithm in terms of distortion, bits per pixel, and encoding complexity for low-detail and medium-detail images. >

The capacity for a discrete-state code division multiple-access channel

Abstract: One of the proposed techniques for the third generation of mobile communications is code division multiple access (CDMA). Some attributes of CDMA which are important to indoor systems include desirable anti-multipath properties, the lack of a need for frequency management or assignment, coexistence with other systems and suitability for micro-cell and in-building systems. The anti-multipath feature turns out to be a key issue for indoor systems and derives from the property that a signal with path delay greater than one chip interval is treated much like an interfering user. The number of users that share the same spectrum, and still maintain an acceptable performance, is determined by the interference generated by the set of remaining users and leads to soft degradation as the number of users increases. This work describes and analyzes a model for the discrete-input continuous-output Gaussian multiple-access channel, that uses spread spectrum techniques. The notion of sum capacity is used to characterize the system performance. Bounds on the sum capacity of this essentially binary input, continuous output channel, where the variance of the output noise is dependent on the number of users present, are obtained. The users are noncooperative in that no common codebook or synchronization is assumed. A novel feature of the work is the use of the modeling of the activity of the user community as a birth-death process. An information theoretic approach is used and the sum capacity is then considered in light of the various regimes to determine the effect of this modeling. >

Variable dimension weighted universal vector quantization and noiseless coding

Abstract: A new algorithm for variable dimension weighted universal coding is introduced. Combining the multi-codebook system of weighted universal vector quantization (WUVQ), the partitioning technique of variable dimension vector quantization, and the optimal design strategy common to both, variable dimension WUVQ allows mixture sources to be effectively carved into their component subsources, each of which can then be encoded with the codebook best matched to that source. Application of variable dimension WUVQ to a sequence of medical images provides up to 4.8 dB improvement in signal to quantization noise ratio over WUVQ and up to 11 dB improvement over a standard full-search vector quantizer followed by an entropy code. The optimal partitioning technique can likewise be applied with a collection of noiseless codes, as found in weighted universal noiseless coding (WUNC). The resulting algorithm for variable dimension WUNC is also described. >

Multi-segment vector quantizer for a speech coder suitable for use in a radiotelephone

Abstract: A Vector-Sum Excited Linear Predictive Coding (VSELP) speech coder provides improved quality and reduced complexity over a typical speech coder. VSELP uses a codebook which has a predefined structure such that the computations required for the codebook search process can be significantly reduced. This VSELP speech coder uses single or multi-segment vector quantizer of the reflection coefficients based on a Fixed-Point-Lattice-Technique (FLAT). Additionally, this speech coder uses a pre-quantizer to reduce the vector codebook search complexity and a high-resolution scalar quantizer to reduce the amount of memory needed to store the reflection coefficient vector codebooks. Resulting in a high quality speech coder with reduced computations and storage requirements.

Single stage spectral quantization at 20 bits

Abstract: Efficient coding of spectral parameters constitutes a major concern for speech compression. It is generally agreed that a single-stage vector quantizer, operating in the log-spectral domain would give superior performance, as compared to any other block coding procedure. Such a coder has, however, been considered impractical due to its complexity. The authors demonstrate that high quality encoding of speech spectra is feasible around 20 bits/spectrum with a single-stage approach. The experimental results are accurately predicted by a theoretical performance analysis. The author suggests a straightforward tree-based procedure that can be employed for the codebook search such that the computational complexity is acceptable. >

Theory and practice of vector quantizers trained on small training sets

Abstract: Examines how the performance of a memoryless vector quantizer changes as a function of its training set size. Specifically, the authors study how well the training set distortion predicts test distortion when the training set is a randomly drawn subset of blocks from the test or training image(s). Using the Vapnik-Chervonenkis (VC) dimension, the authors derive formal bounds for the difference of test and training distortion of vector quantizer codebooks. The authors then describe extensive empirical simulations that test these bounds for a variety of codebook sizes and vector dimensions, and give practical suggestions for determining the training set size necessary to achieve good generalization from a codebook. The authors conclude that, by using training sets comprising only a small fraction of the available data, one can produce results that are close to the results obtainable when all available data are used. >

An adaptive vector quantizer based on the Gold-Washing method for image compression

Abstract: The VLSI architecture for an adaptive vector quantizer is presented. The adaptive vector quantization method does not require a-priori knowledge of the source statistics and the pre-trained codebook. The codebook is generated on the fly and is constantly updated to capture local textual features of data. The source data are directly compressed without requiring the generation of codebook in a separate pass. The adaptive method is based on backward adaption without any side information. The speed of data compression by using the proposed adaptive method is much faster than that by using the conventional vector quantization methods. The algorithm is shown to reach the rate distortion function for memoryless sources. In image processing, most smooth regions are matched by the code vectors and most edge data are preserved by using the block-data interpolation scheme. The VLSI architecture consists of two move-to-front vector quantizers and an index generator. It explores parallelism in the direction of the codebook size and pipelining in the direction of the vector dimension. According to the circuit simulations using the popular SPICE program, the computation power of the move-to-front vector quantizer can reach 40 billion operations per second at a system clock of 100 MHz by using 0.8 /spl mu/m CMOS technology. It can provide a computing capability of 50 Mpixels per second for high-speed image compression. The proposed algorithm and architecture can lead to the development of a high-speed image compressor with great local adaptivity, minimized complexity, and fairly good compression ratio. >

The MacGuffin block cipher algorithm

Abstract: This paper introduces MacGuffin, a 64 bit “codebook” block cipher. Many of its characteristics (block size, application domain, performance and implementation structure) are similar to those of the U.S. Data Encryption Standard (DES). It is based on a Feistel network, in which the cleartext is split into two sides with one side repeatedly modified according to a keyed function of the other. Previous block ciphers of this design, such as DES, operate on equal length sides. MacGuffin is unusual in that it is based on a generalized unbalanced Feistel network (GUFN) in which each round of the cipher modifies only 16 bits according to a function of the other 48. We describe the general characteristics of MacGuffin architecture and implementation and give a complete specification for the 32-round, 128-bit key version of the cipher.