Effective Strong Dimension in Algorithmic Information
and Computational Complexity
Krishna B. Athreya
1
John M. Hitchcock
2
Jack H. Lutz
3
Elvira Mayordomo
4
1
Departments of Mathematics and Statistics, Iowa State University, Ames, IA 50011, USA.
kba@iastate.edu. This research was supported in part by Air Force Office of Scientific Research Grant
ITSI F 49620-01-1-0076 and National Science Foundation grant 0344187.
2
Department of Computer Science, University of Wyoming, Laramie, WY 82072, USA.
jhitchco@cs.uwyo.edu. This research was supported in part by National Science Foundation grants 9988483
and 0515313.
3
Department of Computer Science, Iowa State University, Ames, IA 50011, USA. lutz@cs.iastate.edu.
This research was supported in part by National Science Foundation grants 9988483 and 0344187, and by
Spanish Government MEC project T IN 2005-08832- C03-02.
4
Departamento de Inform´atica e Ingenier´ıa de Sistemas, Universidad de Zaragoza, 50015 Zaragoza,
SPAIN. elvira at unizar dot es. This research was supported in part by Spanish Gove rnment MEC
projects TIC2002-04019-C03-03 and TIN 2005-08832-C03-02, and by National Science Foundation grants
9988483 and 0344187. It was done while visiting Iowa State University.
Abstract
The two most important notions of fractal dimension are Hausdorff dimension, developed by Haus-
dorff (1919), and packing dimension, developed independently by Tricot (1982) and Sullivan (1984).
Both dimensions have the mathematical advantage of being defined from measures, and both have
yielded extensive applications in fractal geometry and dynamical systems .
Lutz (2000) has recently proven a simple characterization of Hausdorff dimension in terms of
gales, which are betting strategies that generalize m artingales. Imposing various computability
and complexity constraints on these gales produces a spectrum of effective versions of Hausdorff
dimension, including constructive, computable, polynomial-space, polynomial-time, and finite-state
dimensions. Work by several investigators has already used these effective dimensions to shed
significant new light on a variety of topics in theoretical computer science.
In this paper we show that packing dimension can also be characterized in terms of gales.
Moreover, even though the usual definition of packing dimension is considerably more complex
than that of Hausdorff dimension, our gale characterization of packing dimension is an exact dual
of – and every bit as simple as – the gale characterization of Hausdorff dimension.
Effectivizing our gale characterization of packing dimension produces a variety of effective strong
dimensions, which are exact duals of the effective dimensions mentioned above. In general (and in
analogy with the classical fractal dimensions), the effective strong dimension of a set or sequence is
at least as great as its effe ctive dimension, with equality for sets or sequences that are sufficiently
regular.
We develop the basic properties of eff ec tive strong dimensions and prove a number of results
relating them to fundamental aspects of randomness, Kolmogorov c omplexity, prediction, Boolean
circuit-size complexity, polynomial-time degrees, and data compression. Aside from the above
characterization of packing dimension, our two main theorems are the following.
1. If
~
β = (β
0
, β
1
, . . .) is a computable sequence of biases that are bounded away from 0 and R is
random with respect to
~
β, then the dimension and strong dimension of R are the lower and
upper average entropies, respectively, of
~
β.
2. For each pair of ∆
0
2
-computable real numbers 0 < α ≤ β ≤ 1, there exists A ∈ E such that
the polynomial-time many-one degree of A has dimension α in E and s trong dimension β in
E.
Our proofs of these theorems use a new large deviation theorem for self-information with respect
to a bias sequence
~
β that need not be convergent.
1 Introduction
Hausdorff dimension – a powerful tool of fractal geometry developed by Hausdorff [12] in 1919
– was effectivized in 2000 by Lutz [22, 23]. This has led to a spectrum of effective versions of
Hausdorff dimension, including constructive, com putable, polynomial-space, polynomial-time, and
finite-state dimensions. Work by several investigators has already used these effective dimensions
to illuminate a variety of topics in algorithmic information theory and computational complexity
[22, 23, 1, 7, 26, 13, 16, 11, 14, 15, 10]. (See [27] for a survey of some of these results.) This work has
also underscored and renewed the importance of earlier work by Ryabko [28, 29, 30, 31], Staiger
[37, 38, 39], and Cai and Hartmanis [5] relating Kolmogorov complexity to classical Hausdorff
dimension. (See Section 6 of [23] for a discussion of this work.)
The key to all these effective dimensions is a simple characterization of classical Hausdorff
dimension in terms of gales, which are betting strategies that generalize martingales. (Martingales,
introduced by L´evy [18] and Ville [45] have been used extensively by Schnorr [32, 33, 34] and others
in the investigation of randomness and by Lutz [20, 21] and others in the development of resource-
bounded measure.) Given this characterization, it is a simple matter to impose computability
and complexity constraints on the gales to produce the above-mentioned spectrum of effective
dimensions.
In the 1980s, a new concept of fractal dimension, c alled the packing dimension, was introduced
independently by Tricot [42] and Sullivan [40]. Packing dimension shares with Hausdorff dimension
the mathematical advantage of being based on a measure. Over the past two decades, despite its
greater complexity (requiring an extra optimization over all countable dec ompos itions of a set in
its definition), packing dimension has be com e, next to Hausdorff dimension, the most important
notion of fractal dimension, yielding extensive applications in fractal geometry and dynamical
systems [9, 8].
The main result of this paper is a proof that packing dimension can also be characterized in
terms of gales. Moreover, notwithstanding the greater complexity of packing dimension’s definition
(and the greater complexity of its behavior on compact sets, as established by Mattila and Mauldin
[25]), our gale characterization of packing dimension is an exact dual of – and every bit as simple
as – the gale characterization of Hausdorff dimension. (This duality and simplicity are in the
statement of our gale characterization; its proof is perforce more involved than its counterpart for
Hausdorff dimension.)
Effectivizing our gale characterization of packing dimension produces for each of the effective di-
mensions above an effective strong dimension that is its exact dual. Just as the Hausdorff dimension
of a set is bounded above by its packing dimension, the effective dimension of a set is bounded above
by its effective strong dimension. Moreover, just as in the classical case, the effective dimension
coincides with the strong effective dimension for sets that are sufficiently regular.
After proving our gale characterization and developing the effective strong dimensions and some
of their basic properties, we prove a number of results relating them to fundamental aspects of
randomness, Kolmogorov complexity, prediction, Boolean circuit-size complexity, polynomial-time
degrees, and data compression. Our two main theorems along these lines are the following.
1. If δ > 0 and
~
β = (β
0
, β
1
, . . .) is a computable sequence of biases with each β
i
∈ [δ,
1
2
], then
1
eve ry sequence R that is random with respect to
~
β has dimension
dim(R) = lim inf
n→∞
1
n
n−1
X
i=0
H(β
i
)
and strong dimension
Dim(R) = lim sup
n→∞
1
n
n−1
X
i=0
H(β
i
),
where H(β
i
) is the Shannon entropy of β
i
.
2. For every pair of ∆
0
2
-computable real numbers 0 < α ≤ β ≤ 1 there is a decision problem
A ∈ E such that the polynomial-time many-one degree of A has dimension α in E and strong
dimension β in E.
In order to prove these theorems, we prove a new large deviation theorem for the self-information
log
1
µ
~
β
(w)
, where
~
β is as in 1 above. Note that
~
β need not be convergent here.
A corollary of theorem 1 above is that, if the average entropies
1
n
P
n−1
i=0
H(β
i
) converge to a
limit H(
~
β) as n → ∞, then dim(R) = Dim(R) = H(
~
β). Since the convergence of these average
entropies is a much weaker condition than the convergence of the biases β
n
as n → ∞, this corollary
substantially strengthens Theorem 7.7 of [23].
Our remaining results are much easier to prove, but their breadth makes a strong prima facie
case for the utility of effective strong dimension. They in some cases explain dual concepts that had
been curiously negle cted in earlier work, and they are likely to be useful in future applications. It
is to be hoped that we are on the verge of seeing the full force of fractal geometry applied fruitfully
to difficult problems in the theory of computing.
2 Preliminaries
We use the set Z of integers, the set Z
+
of (strictly) positive integers, the set N of natural numbers
(i.e., nonnegative integers), the set Q of rational numbers, the set R of real numbers, and the set
[0, ∞) of nonnegative reals. All logarithms in this paper are base 2. We use the slow-growing
function log
∗
n = min{j ∈ N | t
j
≥ n}, where t
0
= 0 and t
j+1
= 2
t
j
, and Shannon’s binary entropy
function H : [0, 1] → [0, 1] defined by
H(β) = β log
1
β
+ (1 − β) log
1
1 − β
,
where 0 log
1
0
= 0.
A string is a finite, binary string w ∈ {0, 1}
∗
. We write |w| for the length of a string w and
λ for the empty string. For i, j ∈ {0, . . . , |w| − 1}, we write w[i..j] for the string consisting of the
i
th
through the j
th
bits of w and w [i] for w[i..i], the i
th
bit of w. Note that the 0
th
bit w[0] is the
leftmost bit of w and that w[i..j] = λ if i > j. A sequence is an infinite, binary sequence. If S is
a sequence and i, j ∈ N, then the notations S[i..j] and S[i] are defined exac tly as for strings. We
work in the Cantor space C consisting of all sequences. A string w ∈ {0, 1}
∗
is a prefix of a sequence
S ∈ C, and we write w v S, if S[0..|w| − 1] = w. The cylinder generated by a string w ∈ {0, 1}
∗
is
C
w
= {S ∈ C|w v S}. Note that C
λ
= C.
2
A language, or decision problem, is a set A ⊆ {0, 1}
∗
. We usually identify a language A
with its characteristic sequence χ
A
∈ C defined by χ
A
[n] = if s
n
∈ A then 1 else 0, where
s
0
= λ, s
1
= 0, s
2
= 1, s
3
= 00, . . . is the standard enumeration of {0, 1}
∗
. That is, we usually (but
not always) use A to denote both the set A ⊆ {0, 1}
∗
and the sequence A = χ
A
∈ C.
Given a set A ⊆ {0, 1}
∗
and n ∈ N, we use the abbreviations A
=n
= A ∩ {0, 1}
n
and A
≤n
=
A ∩ {0, 1}
≤n
. A prefix set is a set A ⊆ {0, 1}
∗
such that no element of A is a prefix of another
element of A.
For each i ∈ N we define a class G
i
of functions from N into N as follows.
G
0
= {f | (∃k)(∀
∞
n)f(n) ≤ kn}
G
i+1
= 2
G
i
(log n)
= {f | (∃g ∈ G
i
)(∀
∞
n)f(n) ≤ 2
g(log n)
}
We also define the functions ˆg
i
∈ G
i
by ˆg
0
(n) = 2n, ˆg
i+1
(n) = 2
ˆg
i
(log n)
. We regard the functions in
these classes as growth rates. In particular, G
0
contains the linearly bounded growth rates and G
1
contains the polynomially bounded growth rates. It is easy to show that each G
i
is closed under
composition, that each f ∈ G
i
is o(ˆg
i+1
), and that each ˆg
i
is o(2
n
). Thus G
i
contains superpolyno-
mial growth rates for all i > 1, but all growth rates in the G
i
-hierarchy are subexponential.
Let CE be the class of computably enumerable languages. Within the class DEC of all decid-
able languages, we are interested in the exponential complexity classes E
i
= DTIME(2
G
i−1
) and
E
i
SPACE = DSPACE(2
G
i−1
) for i ≥ 1. The much-studied classes E = E
1
= DTIME(2
linear
),
E
2
= DTIME(2
polynomial
), and ESPACE = E
1
SPACE = DSPACE(2
linear
) are of particular interest.
We use the following classes of functions.
all = {f | f : {0, 1}
∗
→ {0, 1}
∗
}
comp = {f ∈ all | f is computable}
p
i
= {f ∈ all | f is computable in G
i
time} (i ≥ 1)
p
i
space = {f ∈ all | f is computable in G
i
space} (i ≥ 1)
(The length of the output is included as part of the space used in computing f.) We write p for
p
1
and pspace for p
1
space.
A constructor is a function δ : {0, 1}
∗
→ {0, 1}
∗
that satisfies x
@
6=
δ(x) for all x. The result
of a constructor δ (i.e., the language constructed by δ) is the unique language R(δ) such that
δ
n
(λ) v R(δ) for all n ∈ N. Intuitive ly, δ constructs R(δ) by starting with λ and then iteratively
generating successively longer prefixes of R(δ). We write R(∆) for the set of languages R(δ) such
that δ is a constructor in ∆. The following facts are the reason for our interest in the above-defined
classes of functions.
R(all) = C.
R(comp) = DEC.
For i ≥ 1, R(p
i
)=E
i
.
For i ≥ 1, R(p
i
space) = E
i
SPACE.
If D is a discrete domain (such as N, {0, 1}
∗
, N × {0, 1}
∗
, etc.), then a function f : D −→ [0, ∞)
is ∆-computable if there is a function
ˆ
f : N × D −→ Q ∩ [0, ∞) such that |
ˆ
f(r, x) − f(x)| ≤ 2
−r
for all r ∈ N and x ∈ D and
ˆ
f ∈ ∆ (with r coded in unary and the output coded in binary). We
say that f is exactly ∆-computable if f : D −→ Q ∩ [0, ∞) and f ∈ ∆. We say that f is lower
semicomputable if there is a computable function
ˆ
f : D × N → Q such that
(a) for all (x, t) ∈ D × N,
ˆ
f(x, t) ≤
ˆ
f(x, t + 1) < f (x), and
3