Digital Coding of Waveforms

Consider a discrete source producing a sequence of message letters from a finite alphabet. A single-letter distortion measure is given by a non-negative matrix (d ij ). The entryd ij measures the ?cost? or ?distortion? if letteriis reproduced at the receiver as letterj. The average distortion of a communications system (source-coder-noisy channel-decoder) is taken to bed= ? i.j P ij d ij whereP ij is the probability ofibeing reproduced asj. It is shown that there is a functionR(d) that measures the ?equivalent rate? of the source for a given level of distortion. For coding purposes where a leveldof distortion can be tolerated, the source acts like one with information rateR(d). Methods are given for calculatingR(d), and various properties discussed. Finally, generalizations to ergodic sources, to continuous sources, and to distortion measures involving blocks of letters are developed. In this paper a study is made of the problem of coding a discrete source of information, given afidelity criterionor ameasure of the distortionof the final recovered message at the receiving point relative to the actual transmitted message. In a particular case there might be a certain tolerable level of distortion as determined by this measure. It is desired to so encode the information that the maximum possible signaling rate is obtained without exceeding the tolerable distortion level. This work is an expansion and detailed elaboration of ideas presented earlier [1], with particular reference to the discrete case. We shall show that for a wide class of distortion measures and discrete sources of information there exists a functionR(d) (depending on the particular distortion measure and source) which measures, in a sense, the equivalent rateRof the source (in bits per letter produced) whendis the allowed distortion level. Methods will be given for evaluatingR(d) explicitly in certain simple cases and for evaluatingR(d) by a limiting process in more complex cases. The basic results are roughly that it is impossible to signal at a rate faster thanC/R(d) (source letters per second) over a memoryless channel of capacityC(bits per second) with a distortion measure less than or equal tod. On the other hand, by sufficiently long block codes it is possible to approach as closely as desired the rateC/R(d) with distortion leveld. Finally, some particular examples, using error probability per letter of message and other simple distortion measures, are worked out in detail.

Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959.

This paper proposes a technique for jointly quantizing continuous features and the posterior distributions of their class labels based on minimizing empirical information loss such that the quantizer index of a given feature vector approximates a sufficient statistic for its class label. Informally, the quantized representation retains as much information as possible for classifying the feature vector correctly. We derive an alternating minimization procedure for simultaneously learning codebooks in the Euclidean feature space and in the simplex of posterior class distributions. The resulting quantizer can be used to encode unlabeled points outside the training set and to predict their posterior class distributions, and has an elegant interpretation in terms of lossless source coding. The proposed method is validated on synthetic and real data sets and is applied to two diverse problems: learning discriminative visual vocabularies for bag-of-features image classification and image segmentation.

/pdf/supervised-learning-of-quantizer-codebooks-by-information-evphocsaoq.pdf

Supervised Learning of Quantizer Codebooks by Information Loss Minimization

In this paper we present a system for online power prediction in vir-tualized environments. It is based on Gaussian mixture models that use architectural metrics of the physical and virtual machines (VM) collected dynamically by our system to predict both the physical machine and per VM level power consumption. A real implementation of our system shows that it can achieve average prediction error of less than 10%, outperforming state of the art regression based approaches at negligible runtime overhead.

A system for online power prediction in virtualized environments using Gaussian mixture models

Real-time communication (RTC) applications such as VoIP, video conferencing, and online gaming are flourishing. To adapt and deliver good performance, these applications require accurate estimations of short-term network performance metrics, e.g., loss rate, one-way delay, and throughput. However, the wide variation in mobile cellular network performance makes running RTC applications on these networks problematic. To address this issue, various performance adaptation techniques have been proposed, but one common problem of such techniques is that they only adjust application behavior reactively after performance degradation is visible. Thus, proactive adaptation based on accurate short-term, fine-grained network performance prediction can be a preferred alternative that benefits RTC applications. In this study, we show that forecasting the short-term performance in cellular networks is possible in part due to the channel estimation scheme on the device and the radio resource scheduling algorithm at the base station. We develop a system interface called PROTEUS, which passively collects current network performance, such as throughput, loss, and one-way delay, and then uses regression trees to forecast future network performance. PROTEUS successfully predicts the occurrence of packet loss within a 0.5s time window for 98% of the time windows and the occurrence of long one-way delay for 97% of the time windows. We also demonstrate how PROTEUS can be integrated with RTC applications to significantly improve the perceptual quality. In particular, we increase the peak signal-to-noise ratio of a video conferencing application by up to 15dB and reduce the perceptual delay in a gaming application by up to 4s.

/pdf/proteus-network-performance-forecast-for-real-time-4ngce0e50b.pdf

PROTEUS: network performance forecast for real-time, interactive mobile applications

Gauss mixtures have gained popularity in statistics and statistical signal processing applications for a variety of reasons, including their ability to well approximate a large class of interesting densities and the availability of algorithms such as the Baum–Welch or expectation-maximization (EM) algorithm for constructing the models based on observed data. We here consider a quantization approach to Gauss mixture design based on the information theoretic view of Gaussian sources as a “worst case” for robust signal compression. Results in high-rate quantization theory suggest distortion measures suitable for Lloyd clustering of Gaussian components based on a training set of data. The approach provides a Gauss mixture model and an associated Gauss mixture vector quantizer which is locally robust. We describe the quantizer mismatch distortion and its relation to other distortion measures including the traditional squared error, the Kullback–Leibler (relative entropy) and minimum discrimination information, and the log-likehood distortions. The resulting Lloyd clustering algorithm is demonstrated by applications to image vector quantization, texture classification, and North Atlantic pipeline image classification.

Lloyd clustering of Gauss mixture models for image compression and classification

Interactive Internet Applications that rely on sequential streams for lossless data exchange often use retransmission protocols (e.g. TCP) for reliability and the guarantee of sequential data ordering. More so than for bulk file transfer or media delivery, lossless sequential streaming poses an even greater challenge for the common problem cases of retransmission protocols, such as lossy links or long network paths, manifesting as significant latency in the interactive user experience. We propose a hybrid FEC-ARQ protocol built on a packet streaming code that reduces to a simple strategy over sending or resending original data packets or check packets combining undecoded packets, based on actual network conditions. Experimental results show that our proposed protocol can significantly improve the total delay over retransmission and other schemes that use FEC, under a range of bandwidth and loss scenarios.

/pdf/a-hybrid-fec-arq-protocol-for-low-delay-lossless-sequential-2luw6ex8u5.pdf

A hybrid FEC-ARQ protocol for low-delay lossless sequential data streaming

As cloud computing is taking off, the presence of high-performance interactive Internet applications is exploding. By nature, these applications require responsive client-server data exchange and lossless, in-order delivery. Previous work has shown that by using forward error correction (FEC), it is possible to reduce the data streaming latency caused by retransmissions of lost packets. However, the prior schemes only send FEC packets when there are no original packets pending transmission. In this paper, we further expand the hybrid FEC-ARQ protocol and show that sometimes, the transmission latency can be further reduced by preempting original data packets with FEC packets. We have formulated the decision of whether to send new original data packets, FEC packets, or resend original data packets as a transmission policy. An optimal transmission policy is selected to minimize the delay experienced by the application subject to a constraint on the amount of overhead. By using this optimal policy, we significantly improve the delay performance over straightforward FEC schemes while controlling the amount of overhead due to FEC.

/pdf/optimizing-fec-transmission-strategy-for-minimizing-delay-in-29vdd8gk5z.pdf

Optimizing FEC Transmission Strategy for Minimizing Delay in Lossless Sequential Streaming

We propose a linear predictive quantization system for causally transmitting parallel sources with temporal memory (colored frames) over an erasure channel. By optimizing within this structure, we derive an achievability result in the high-rate limit and compare it to an upper bound on performance. The proposed system subsumes the well-known PCM and DPCM systems as special cases. While typically DPCM performs well without erasures and PCM suffers less with many erasures, we show that the proposed solution improves performance over both under all severities of erasures, with unbounded improvement in some cases.

/pdf/causal-transmission-of-colored-source-frames-over-a-packet-2703r4n3gk.pdf

Causal Transmission of Colored Source Frames over a Packet Erasure Channel

An encoder generates a compressed data sequence from an original data sequence using many-to-one mapping independently of a source model associated with the original data sequence and without extracting the source model. A decoder uses both the source model associated with the original data sequence and the mapping applied during compression that is devoid of, in substance, the source model, to regenerate, at least in part, the original uncompressed data sequence from the compressed data sequence that does not include a significant portion of the source model.

Ying-zong Huang

Papers

Lloyd clustering of Gauss mixture models for image compression and classification

A hybrid FEC-ARQ protocol for low-delay lossless sequential data streaming

Optimizing FEC Transmission Strategy for Minimizing Delay in Lossless Sequential Streaming

Causal Transmission of Colored Source Frames over a Packet Erasure Channel

Systems and methods for model-free compression and model-based decompression