The minimum description length principle in coding and modeling
Reads0
Chats0
TLDR
The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms.Abstract:
We review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon's basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. We assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference. Context tree modeling, density estimation, and model selection in Gaussian linear regression serve as examples.read more
Citations
More filters
Journal ArticleDOI
Statistical pattern recognition: a review
TL;DR: The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field.
Book
Prediction, learning, and games
Nicolò Cesa-Bianchi,Gábor Lugosi +1 more
TL;DR: In this paper, the authors provide a comprehensive treatment of the problem of predicting individual sequences using expert advice, a general framework within which many related problems can be cast and discussed, such as repeated game playing, adaptive data compression, sequential investment in the stock market, sequential pattern analysis, and several other problems.
Journal ArticleDOI
A Completed Modeling of Local Binary Pattern Operator for Texture Classification
TL;DR: It is shown that CLBP_S preserves more information of the local structure thanCLBP_M, which explains why the simple LBP operator can extract the texture features reasonably well and can be made for rotation invariant texture classification.
Review on determining number of Cluster in K-Means Clustering
TL;DR: Six different approaches to determine the right number of clusters in a dataset are explored, including k-means method, a simple and fast clustering technique that addresses the problem of cluster number selection by using a k-Means approach.
Journal ArticleDOI
Hidden Markov processes
Yariv Ephraim,Neri Merhav +1 more
TL;DR: An overview of statistical and information-theoretic aspects of hidden Markov processes (HMPs) is presented and consistency and asymptotic normality of the maximum-likelihood parameter estimator were proved under some mild conditions.
References
More filters
Book
Elements of information theory
Thomas M. Cover,Joy A. Thomas +1 more
TL;DR: The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
Journal ArticleDOI
Estimating the Dimension of a Model
TL;DR: In this paper, the problem of selecting one of a number of models of different dimensions is treated by finding its Bayes solution, and evaluating the leading terms of its asymptotic expansion.
Estimating the dimension of a model
TL;DR: In this paper, the problem of selecting one of a number of models of different dimensions is treated by finding its Bayes solution, and evaluating the leading terms of its asymptotic expansion.
Proceedings Article
Information Theory and an Extention of the Maximum Likelihood Principle
TL;DR: The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.
Book ChapterDOI
Information Theory and an Extension of the Maximum Likelihood Principle
TL;DR: In this paper, it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion.