Preface 1 Introduction 1.1 Compression Techniques 1.1.1 Lossless Compression 1.1.2 Lossy Compression 1.1.3 Measures of Performance 1.2 Modeling and Coding 1.3 Organization of This Book 1.4 Summary 1.5 Projects and Problems 2 Mathematical Preliminaries 2.1 Overview 2.2 A Brief Introduction to Information Theory 2.3 Models 2.3.1 Physical Models 2.3.2 Probability Models 2.3.3. Markov Models 2.3.4 Summary 2.5 Projects and Problems 3 Huffman Coding 3.1 Overview 3.2 "Good" Codes 3.3. The Huffman Coding Algorithm 3.3.1 Minimum Variance Huffman Codes 3.3.2 Length of Huffman Codes 3.3.3 Extended Huffman Codes 3.4 Nonbinary Huffman Codes 3.5 Adaptive Huffman Coding 3.5.1 Update Procedure 3.5.2 Encoding Procedure 3.5.3 Decoding Procedure 3.6 Applications of Huffman Coding 3.6.1 Lossless Image Compression 3.6.2 Text Compression 3.6.3 Audio Compression 3.7 Summary 3.8 Projects and Problems 4 Arithmetic Coding 4.1 Overview 4.2 Introduction 4.3 Coding a Sequence 4.3.1 Generating a Tag 4.3.2 Deciphering the Tag 4.4 Generating a Binary Code 4.4.1 Uniqueness and Efficiency of the Arithmetic Code 4.4.2 Algorithm Implementation 4.4.3 Integer Implementation 4.5 Comparison of Huffman and Arithmetic Coding 4.6 Applications 4.6.1 Bi-Level Image Compression-The JBIG Standard 4.6.2 Image Compression 4.7 Summary 4.8 Projects and Problems 5 Dictionary Techniques 5.1 Overview 5.2 Introduction 5.3 Static Dictionary 5.3.1 Diagram Coding 5.4 Adaptive Dictionary 5.4.1 The LZ77 Approach 5.4.2 The LZ78 Approach 5.5 Applications 5.5.1 File Compression-UNIX COMPRESS 5.5.2 Image Compression-the Graphics Interchange Format (GIF) 5.5.3 Compression over Modems-V.42 bis 5.6 Summary 5.7 Projects and Problems 6 Lossless Image Compression 6.1 Overview 6.2 Introduction 6.3 Facsimile Encoding 6.3.1 Run-Length Coding 6.3.2 CCITT Group 3 and 4-Recommendations T.4 and T.6 6.3.3 Comparison of MH, MR, MMR, and JBIG 6.4 Progressive Image Transmission 6.5 Other Image Compression Approaches 6.5.1 Linear Prediction Models 6.5.2 Context Models 6.5.3 Multiresolution Models 6.5.4 Modeling Prediction Errors 6.6 Summary 6.7 Projects and Problems 7 Mathematical Preliminaries 7.1 Overview 7.2 Introduction 7.3 Distortion Criteria 7.3.1 The Human Visual System 7.3.2 Auditory Perception 7.4 Information Theory Revisted 7.4.1 Conditional Entropy 7.4.2 Average Mutual Information 7.4.3 Differential Entropy 7.5 Rate Distortion Theory 7.6 Models 7.6.1 Probability Models 7.6.2 Linear System Models 7.6.3 Physical Models 7.7 Summary 7.8 Projects and Problems 8 Scalar Quantization 8.1 Overview 8.2 Introduction 8.3 The Quantization Problem 8.4 Uniform Quantizer 8.5 Adaptive Quantization 8.5.1 Forward Adaptive Quantization 8.5.2 Backward Adaptive Quantization 8.6 Nonuniform Quantization 8.6.1 pdf-Optimized Quantization 8.6.2 Companded Quantization 8.7 Entropy-Coded Quantization 8.7.1 Entropy Coding of Lloyd-Max Quantizer Outputs 8.7.2 Entropy-Constrained Quantization 8.7.3 High-Rate Optimum Quantization 8.8 Summary 8.9 Projects and Problems 9 Vector Quantization 9.1 Overview 9.2 Introduction 9.3 Advantages of Vector Quantization over Scalar Quantization 9.4 The Linde-Buzo-Gray Algorithm 9.4.1 Initializing the LBG Algorithm 9.4.2 The Empty Cell Problem 9.4.3 Use of LBG for Image Compression 9.5 Tree-Structured Vector Quantizers 9.5.1 Design of Tree-Structured Vector Quantizers 9.6 Structured Vector Quantizers 9.6.1 Pyramid Vector Quantization 9.6.2 Polar and Spherical Vector Quantizers 9.6.3 Lattice Vector Quantizers 9.7 Variations on the Theme 9.7.1 Gain-Shape Vector Quantization 9.7.2 Mean-Removed Vector Quantization 9.7.3 Classified Vector Quantization 9.7.4 Multistage Vector Quantization 9.7.5 Adaptive Vector Quantization 9.8 Summary 9.9 Projects and Problems 10 Differential Encoding 10.1 Overview 10.2 Introduction 10.3 The Basic Algorithm 10.4 Prediction in DPCM 10.5 Adaptive DPCM (ADPCM) 10.5.1 Adaptive Quantization in DPCM 10.5.2 Adaptive Prediction in DPCM 10.6 Delta Modulation 10.6.1 Constant Factor Adaptive Delta Modulation (CFDM) 10.6.2 Continuously Variable Slope Delta Modulation 10.7 Speech Coding 10.7.1 G.726 10.8 Summary 10.9 Projects and Problems 11 Subband Coding 11.1 Overview 11.2 Introduction 11.3 The Frequency Domain and Filtering 11.3.1 Filters 11.4 The Basic Subband Coding Algorithm 11.4.1 Bit Allocation 11.5 Application to Speech Coding-G.722 11.6 Application to Audio Coding-MPEG Audio 11.7 Application to Image Compression 11.7.1 Decomposing an Image 11.7.2 Coding the Subbands 11.8 Wavelets 11.8.1 Families of Wavelets 11.8.2 Wavelets and Image Compression 11.9 Summary 11.10 Projects and Problems 12 Transform Coding 12.1 Overview 12.2 Introduction 12.3 The Transform 12.4 Transforms of Interest 12.4.1 Karhunen-Loeve Transform 12.4.2 Discrete Cosine Transform 12.4.3 Discrete Sine Transform 12.4.4 Discrete Walsh-Hadamard Transform 12.5 Quantization and Coding of Transform Coefficients 12.6 Application to Image Compression-JPEG 12.6.1 The Transform 12.6.2 Quantization 12.6.3 Coding 12.7 Application to Audio Compression 12.8 Summary 12.9 Projects and Problems 13 Analysis/Synthesis Schemes 13.1 Overview 13.2 Introduction 13.3 Speech Compression 13.3.1 The Channel Vocoder 13.3.2 The Linear Predictive Coder (Gov.Std.LPC-10) 13.3.3 Code Excited Linear Prediction (CELP) 13.3.4 Sinusoidal Coders 13.4 Image Compression 13.4.1 Fractal Compression 13.5 Summary 13.6 Projects and Problems 14 Video Compression 14.1 Overview 14.2 Introduction 14.3 Motion Compensation 14.4 Video Signal Representation 14.5 Algorithms for Videoconferencing and Videophones 14.5.1 ITU_T Recommendation H.261 14.5.2 Model-Based Coding 14.6 Asymmetric Applications 14.6.1 The MPEG Video Standard 14.7 Packet Video 14.7.1 ATM Networks 14.7.2 Compression Issues in ATM Networks 14.7.3 Compression Algorithms for Packet Video 14.8 Summary 14.9 Projects and Problems A Probability and Random Processes A.1 Probability A.2 Random Variables A.3 Distribution Functions A.4 Expectation A.5 Types of Distribution A.6 Stochastic Process A.7 Projects and Problems B A Brief Review of Matrix Concepts B.1 A Matrix B.2 Matrix Operations C Codes for Facsimile Encoding D The Root Lattices Bibliography Index

Introduction to data compression

The development of the Internet in recent years has made it possible and useful to access many different information systems anywhere in the world to obtain information. While there is much research on the integration of heterogeneous information systems, most commercial systems stop short of the actual integration of available data. Data fusion is the process of fusing multiple records representing the same real-world object into a single, consistent, and clean representation.This article places data fusion into the greater context of data integration, precisely defines the goals of data fusion, namely, complete, concise, and consistent data, and highlights the challenges of data fusion, namely, uncertain and conflicting data values. We give an overview and classification of different ways of fusing data and present several techniques based on standard and advanced operators of the relational algebra and SQL. Finally, the article features a comprehensive survey of data integration systems from academia and industry, showing if and how data fusion is performed in each.

Data fusion

Quantization is a process that maps a continous or discrete set of values into approximations that belong to a smaller set. Quantization is a lossy: some information about the original data is lost in the process. The key to a successful quantization is therefore the selection of an error criterion – such as entropy and signal-to-noise ratio – and the development of optimal quantizers for this criterion.

Quantization

In this paper basic results on distributed detection are reviewed. In particular we consider the parallel and the serial architectures in some detail and discuss the decision rules obtained from their optimization based an the Neyman-Pearson (NP) criterion and the Bayes formulation. For conditionally independent sensor observations, the optimality of the likelihood ratio test (LRT) at the sensors is established. General comments on several important issues are made including the computational complexity of obtaining the optimal solutions the design of detection networks with more general topologies, and applications to different areas.

/pdf/distributed-detection-with-multiple-sensors-part-i-1v2kzhupbi.pdf

Distributed detection with multiple sensors Part I. Fundamentals

In this paper, we investigate a binary decentralized detection problem in which a network of wireless sensors provides relevant information about the state of nature to a fusion center. Each sensor transmits its data over a multiple access channel. Upon reception of the information, the fusion center attempts to accurately reconstruct the state of nature. We consider the scenario where the sensor network is constrained by the capacity of the wireless channel over which the sensors are transmitting, and we study the structure of an optimal sensor configuration. For the problem of detecting deterministic signals in additive Gaussian noise, we show that having a set of identical binary sensors is asymptotically optimal, as the number of observations per sensor goes to infinity. Thus, the gain offered by having more sensors exceeds the benefits of getting detailed information from each sensor. A thorough analysis of the Gaussian case is presented along with some extensions to other observation distributions.

http://www.ifp.illinois.edu/~vvv/papers/journal/dd_wsn.pdf

Decentralized detection in sensor networks

Most results about quantized detection rely strongly on an assumption of independence among random variables. With this assumption removed, little is known. Thus, in this paper, Bayes-optimal binary quantization for the detection of a shift in mean in a pair of dependent Gaussian random variables is studied. This is arguably the simplest meaningful problem one could consider. If results and rules are to be found, they ought to make themselves plain in this problem. For certain problem parametrizations (meaning the signals and correlation coefficient), optimal quantization is achievable via a single threshold applied to each observation-the same as under independence. In other cases, one observation is best ignored or is quantized with two thresholds; neither behavior is seen under independence. Further, and again in distinction from the case of independence, it is seen that in certain situations, an XOR fusion rule is optimal, and in these cases, the implied decision rule is bizarre. The analysis is extended to the multivariate Gaussian problem.

The good, bad and ugly: distributed detection of a known signal in dependent Gaussian noise

Decentralized detection networks implement local likelihood ratio tests at each sensor and combine the local decisions into a global decision. In general this is a "one-pass" procedure, with data transmission from the sensors (or subordinate decision makers) to the fusion center. There have been treatments of multipass schemes, but usually the approach has been to update the local decisions based on the previous fused decision. We explore the use of feedback (of all sensor decisions to all sensors) and successive retesting and rebroadcasting of the updated decisions until a common decision or consensus is reached, an operation we call parley due to its similarity to the process of discussion and negotiation by a team of human decision makers. We consider two modes of operation of such a network. In the first, all sensors are as correct as possible at all times. We see that this network is fast in reaching consensus, but not particularly as "correct" as it could be. Under the second scheme, we observe that via feedback it is possible for the network to make an optimum decision-this may, of course, require several rounds of parley. Of particular interest is that under both schemes a consensus always occurs-the sensors never "agree to disagree". >

Parley as an approach to distributed detection

Unrestricted polar quantizers (UPQ's) were previously introduced to overcome the weak mean-squared error performance of polar coordinate quantizers for the independent bivariate Gaussian source when the number of levels was small ( bits /dimension). An asymptotic ntean-squared error analysis of the UPQ's with a general circularly symmetric source is presented. The performance of the UPQ's is shown to be asymptotically within 0.17 dB of that of the optimum bivariate quantizer. The results are extended to dimensions greater than two. Examples for the independent Gaussian source are included.

Asymptotic performance of unrestricted polar quantizers (Corresp.)

Distributed detection on the serial (or tandem) topology is considered with the probability of error performance criterion. Previously published efforts, while presenting probability of detection versus false alarm results, limited the number of array elements to two or three. For the detection of known, equally likely signals in additive, symmetric noise, the author presents simple recursive expressions for the threshold values and the performance of the system. Examples for known signals in Gaussian and Laplacian noise show the degradation in performance due to the array structure. >

On the performance of serial networks in distributed detection

The detection of a vanishingly small, known signal in multi-variate noise is considered. Efficacy is used as a criterion of detector performance, and the locally optimal detector (LOD) for multivariate noise is derived. It is shown that this is a generalization of the well-known LOD for independent, identically distributed (i.i.d.) noise. Several characterizations of multivariate noise are used as examples; these include specific examples and some general methods of density generation. In particular, the class of multivariate densities generated by a zero-memory nonlinear transformation of a correlated Gaussian source is discussed in some detail. The detector structure is derived and practical aspects of obtaining detector subsystems are considered. Through the use of Monte Carlo simulations, the performance of this system if compared to that of the matched filter and of the i.i.d. LOD. Finally, the class of multivariate densities generated by a linear transformation of an i.i.d, noise source is described, and its LOD is shown to be a form frequently suggested to deal with multivariate, non-Gaussian noise: a linear filter followed by a memoryless nonlinearity and a correlator.

Peter F. Swaszek

Papers

The good, bad and ugly: distributed detection of a known signal in dependent Gaussian noise

Parley as an approach to distributed detection

Asymptotic performance of unrestricted polar quantizers (Corresp.)

On the performance of serial networks in distributed detection

Locally optimal detection in multivariate non-Gaussian noise