The book is outstanding and admirable in many respects. ... is necessary reading for all kinds of readers from undergraduate students to top authorities in the field. Journal of Symbolic Logic Written by two experts in the field, this is the only comprehensive and unified treatment of the central ideas and their applications of Kolmogorov complexity. The book presents a thorough treatment of the subject with a wide range of illustrative applications. Such applications include the randomness of finite objects or infinite sequences, Martin-Loef tests for randomness, information theory, computational learning theory, the complexity of algorithms, and the thermodynamics of computing. It will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics. The book is self-contained in that it contains the basic requirements from mathematics and computer science. Included are also numerous problem sets, comments, source references, and hints to solutions of problems. New topics in this edition include Omega numbers, KolmogorovLoveland randomness, universal learning, communication complexity, Kolmogorov's random graphs, time-limited universal distribution, Shannon information and others.

An Introduction to Kolmogorov Complexity and Its Applications

This paper reviews algorithmic information theory, which is an attempt to apply information-theoretic and probabilistic ideas to recursive function theory. Typical concerns in this approach are, for example, the number of bits of information required to specify an algorithm, or the probability that a program whose bits are chosen by coin flipping produces a given output. During the past few years the definitions of algorithmic information theory have been reformulated. The basic features of the new formalism are presented here and certain results of R. M. Solovay are reported.

Algorithmic Information Theory

There has been a great deal of theoretical and experimental work in computer science on inductive inference systems, that is, systems that try to infer general rules from examples. However, a complete and applicable theory of such systems is still a distant goal. This survey highlights and explains the main ideas that have been developed in the study of inductive inference, with special emphasis on the relations between the general theory and the specific algorithms and implementations. 154 references.

Inductive Inference: Theory and Methods

A new definition of program-size complexity is made. H(A,B/C,D) is defined to be the size in bits of the shortest self-delimiting program for calculating strings A and B if one is given a minimal-size self-delimiting program for calculating strings C and D. This differs from previous definitions: (1) programs are required to be self-delimiting, i.e. no program is a prefix of another, and (2) instead of being given C and D directly, one is given a program for calculating them that is minimal in size. Unlike previous definitions, this one has precisely the formal properties of the entropy concept of information theory. For example, H(A,B) = H(A) + H(B/A) -~ 0(1). Also, if a program of length k is assigned measure 2 -k, then H(A) = -log2 (the probability that the standard universal computer will calculate A) -{- 0(1).

A Theory of Program Size Formally Identical to Information Theory

Intelligence tests occasionally require the extrapolation of an effective sequence (e.g. 1661, 2552, 3663, …) that is produced by some easily discernible algorithm. In this paper, we investigate the theoretical capabilities and limitations of a computer to infer such sequences. We design Turing machines that in principle are extremely powerful for this purpose and place upper bounds on the capabilities of machines that would do better.

Toward a mathematical theory of inductive inference

The notion of the complexity of performing an inductive inference is defined. Some examples of the tradeoffs between the complexity of performing an inference and the accuracy of the inferred result are presented. An axiomatization of the notion of the complexity of inductive inference is developed and several results are presented which both resemble and contrast with results obtainable for the axiomatic computational complexity of recursive functions.

On the complexity of inductive inference

On the error correcting power of pluralism in BC-type inductive inference

The extent and density of sequences within the minimal-program complexity hierarchies

Introduction. Throughout the history of the theory of recursive functions diverse hierarchies have been proposed in order to study and classify both constructive and nonconstructive objects. Recently, attempts to classify recursive functions according to their complexity of computation have exposed many important aspects of the relationship between these functions and the devices used to compute them. The objects under investigation in this work will be finite and infinite binary sequences. The infinite binary sequences which one may regard as the characteristic functions of sets provide a means of studying the limiting behavior of finite sequences as their length increases. Several minimalprogram complexity measures have been proposed (see Kolmogorov [9, 10], Chaitin [2, 3], Loveland [11, 12], Schnorr [18], Zvonkin and Levin [23]) which in a certain sense measure the information content of finite, and as a limit, infinite binary sequences. Recursive sequences are known to have the lowest minimal-program complexity and random sequences (e.g., in the sequential test sense of Martin-L6f [15]) the highest complexity. In this paper we investigate the minimal-program complexity of several formulations of pseudo-recursive sequence (a pseudo-recursive sequence is one which in some sense approximates a recursive sequence) and of pseudo-random sequence. Ideally, one would hope that the psuedo-recursive sequences would have relatively low minimal-program complexity and the pseudo-random sequences relatively high complexity. However, such is not the case. In fact, it is shown that there are pseudo-random sequences which have extremely low minimal-program complexity. Moreover, as a consequence of this result we will be unable to separate the pseudo-random sequences from the (nonrecursive) pseudo-recursive sequences by means of complexity classes. Thus we will be unable to characterize any reasonable notion of pseudo-recursive sequence in terms of the minimal-program complexity hierarchy. Therefore, in our investigation of pseudo-recursive sequences we will consider only upper bounds for the complexity of the various classes of sequences being considered.

Robert P. Daley

Papers

On the complexity of inductive inference

On the error correcting power of pluralism in BC-type inductive inference

The extent and density of sequences within the minimal-program complexity hierarchies

Minimal-program complexity of pseudo-recursive and pseudo-random sequences

On the inference of optimal descriptions