An Introduction to Kolmogorov Complexity and Its Applications
Citations
14,635 citations
Cites background from "An Introduction to Kolmogorov Compl..."
...Most FNN applications focused on FNNs with few hidden layers....
[...]
...In Supervised Learning (SL), certain NN output events xt may be associated with teacher-given, real-valued labels or targets dt yielding errors et , e.g., et = 1/2(xt − dt)2....
[...]
...…of a solution candidate by the length of the shortest program that computes it (e.g., Blumer, Ehrenfeucht, Haussler, & Warmuth, 1987; Chaitin, 1966; Grünwald,Myung, & Pitt, 2005; Kolmogorov, 1965b; Levin, 1973a; Li & Vitányi, 1997; Rissanen, 1986; Solomonoff, 1964, 1978;Wallace & Boulton, 1968)....
[...]
10,696 citations
Cites background from "An Introduction to Kolmogorov Compl..."
...…is important to keep in mind that there exist several fundamentally different approaches such as Minimum Description Length (cf. e.g. Rissanen 1978, Li and Vitányi 1993) which is based on the idea that the simplicity of an estimate, and therefore also its plausibility is based on the information…...
[...]
9,627 citations
7,767 citations
Cites background from "An Introduction to Kolmogorov Compl..."
...According to learning theory (Vapnik, 1995; Li & Vitanyi, 1997), to obtain good generalization it is enough that the total number of bits needed to encode thewhole training setbe small, compared to the size of the training set....
[...]
6,340 citations
Cites background or methods from "An Introduction to Kolmogorov Compl..."
...As long as the instance encoding is within a polynomial factor of the optimal encoding (this can be made precise using Kolmogorov complexity [633]), then this bizarre behavior is avoided....
[...]
...This brings it closer to the Kolmogorov complexity [250, 633] of the state transition graph, which is the shortest bit size to which it can possibly be compressed and then fully recovered by a Turing machine....
[...]
References
7,642 citations
7,202 citations
6,675 citations
3,939 citations
3,760 citations