Journal ArticleDOI
The context-tree weighting method: basic properties
Reads0
Chats0
TLDR
The authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences that shows that the proposed context-tree weighting procedure is optimal in the sense that it achieves the Rissanen (1984) lower bound.Abstract:
Describes a sequential universal data compression procedure for binary tree sources that performs the "double mixture." Using a context tree, this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources, and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters. Computational and storage complexity of the proposed procedure are both linear in the source sequence length. The authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences. The three terms in this bound can be identified as coding, parameter, and model redundancy, The bound holds for all source sequence lengths, not only for asymptotically large lengths. The analysis that leads to this bound is based on standard techniques and turns out to be extremely simple. The upper bound on the redundancy shows that the proposed context-tree weighting procedure is optimal in the sense that it achieves the Rissanen (1984) lower bound. >read more
Citations
More filters
Book
Pattern recognition and neural networks
Brian D. Ripley,N. L. Hjort +1 more
TL;DR: Professor Ripley brings together two crucial ideas in pattern recognition; statistical methods and machine learning via neural networks in this self-contained account.
Book
Prediction, learning, and games
Nicolò Cesa-Bianchi,Gábor Lugosi +1 more
TL;DR: In this paper, the authors provide a comprehensive treatment of the problem of predicting individual sequences using expert advice, a general framework within which many related problems can be cast and discussed, such as repeated game playing, adaptive data compression, sequential investment in the stock market, sequential pattern analysis, and several other problems.
Book ChapterDOI
FPGA Intrinsic PUFs and Their Use for IP Protection
TL;DR: New protocols for the IP protection problem on FPGAs are proposed and the first construction of a PUF intrinsic to current FPGA based on SRAM memory randomness present on current FFPAs is provided.
Journal ArticleDOI
The minimum description length principle in coding and modeling
TL;DR: The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms.
Book
Information Theory and Statistics: A Tutorial
I. Csiszar,Paul C. Shields +1 more
TL;DR: This tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting, and an introduction is provided to the theory of universal coding, and to statistical inference via the minimum description length principle motivated by that theory.
References
More filters
Journal ArticleDOI
A mathematical theory of communication
TL;DR: This final installment of the paper considers the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now.
Book
Elements of information theory
Thomas M. Cover,Joy A. Thomas +1 more
TL;DR: The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
Book
Information Theory and Reliable Communication
TL;DR: This chapter discusses Coding for Discrete Sources, Techniques for Coding and Decoding, and Source Coding with a Fidelity Criterion.
Journal ArticleDOI
A Method for the Construction of Minimum-Redundancy Codes
TL;DR: A minimum-redundancy code is one constructed in such a way that the average number of coding digits per message is minimized.
Journal ArticleDOI
Universal coding, information, prediction, and estimation
TL;DR: A connection between universal codes and the problems of prediction and statistical estimation is established, and a known lower bound for the mean length of universal codes is sharpened and generalized, and optimum universal codes constructed.