Reprinl from
Topics
in Current Physics
Volume
20: Inverse Scattering Problems in Optics
Editer:
H. P. Baltes
©
by Springer-Verlag Berlin Heldelberg 1980
Printed in
Germany.
Not
(or Sale.
Springer-Verlag
Berlin
Heidelberg
New
York
Inverse
Scattering Problems in Optics
Editer:
H. P. Baltes
1.
Progress in Inverse Optical Problems. By H. P. Baltes
2.
The Inverse Scattering Problem in Structural
Déterminations
By
G. Ross, M. A. Fiddy, and M. Nieto-Vesperinas (With 9 Figures)
3. Ptioton-Counting
Statistics of Optical Scintillation
By
E. Jakeman and P. N. Pusey (With 9 Figures)
4. IVlicrosccpic Models
of
Photodetection.
By A. Selloni (With 3 Figures)
5.
The
Statîility
of Inverse Problems
By
M. Bertero, C. De Mol, and G. A. Viano (With 7 Figures)
6.
Combustion Diagnostics by
Multiangular
Absorption
By
R. Goulard and P. J. Emmerman (With 10 Figures)
7. Polarization Utilization
in Electromagnetic Inverse Scattering
By
W.-M. Boerner (With 11 Figures)
>
5.
The
Stability
of Inverse Problems
M.
Bertero, C. De Mol, and G. A, Viano
—
-
With
7 Figures
Hany
inverse problems arising in optics and other fields
like
geophysics,
médical
diagnostics
and remote sensing,
présent numerical
instability: the noise affecting
the
data
may
produce arbitrarily large errors in the solutions. In other
words,
thèse
problems are
ill-posed
in the
sensé
of Hadamard.
The
basic point, in the study of ill-posed problems, is that the development of
adéquate
computational methods, leading to stable results, requires prior
kncwledge
of
properties of the admissible solutions: global bounds, smoothness conditions,
positivity
constraints, statistical properties, etc. The problem is first to incor-
porate
the supplementary constraints in the computational algorithm, and secondly
to
estimate the accuracy of the solutions for a given prior
knowledge
and data accu-
racy.
General methods are available
only
for linear inverse problems.
This
chapter begins with an outline of the main features of ill-posed problems,
of
their connection with inverse problems and of the basic ideas enabling one to
solve
them. Next we discuss
regutarization tkeory
where the supplementary constraints
are
prescribed bounds on the
class
of admissible solutions. Then we analyze the
application
to ill-posed problems of the
method
of
linear
mean square estimation
(aptvmm filteriig),
when prior knowledge of statistical properties of the solu-
tions
is available. Finally, we review the applications of the previous methods to
some
linear inverse problems in optics and scattering theory.
5.1 II 1-Posedness
in Inverse Problems
The
concept of
ill-posedness
was introduced by HADAMARD [5.1] in the
field
of partial
differential équations.
For years, ill-posed problems have been considered as mere
mathematical
anomalies.
Indeed,
it was believed that physical situations were lead-
ing
only to
well-posed
problems like, for instance, the Dirichlet problem for
ellip-
tic équations
of potential theory, or the Cauchy problem for hyperbolic
équations
describing
wave motion. However, it appeared
later
that this attitude was erroneous
and
that many ill-posed problems, generally inverse problems, were arising from
practical
situations. Nowadays there is no doubt that a systematic study of
thèse
problems
is of great relevance in many fields of applied physics.
162
5.1.1
He11-Posed and
111-Posed Problems
It
is
rather
difficult to give a
précise
and exhaustive
définition
of an
ill-posed
problem.
Indeed this term covers a lot of various problems presenting
many common
features
but also
différences
so important that a global and unified theory is not
yet
available. The best characterization is perhaps a
négative
one: ill-posed prob-
lems
do not fulfill
ail
the required conditions for
wel1-posedness [5.1],
i.e.,
eù'istence, uniquemss
and
oontùmity
of the solution on the data (requirement of
stability).
As clearly
stated
by COURANT and HUBERT
[Réf.5.2, p.227], "tte
third
requirement, partïcularly
incisive, is
neaessary
if the
inathematiaal
formulation
is
to describe observable natural
phenonena.
Data in
nature
cannot
possibly
he con-
aeived
as rigidly fixed; the mere
proaess
of
measuring
them involves
small
errors.
Therefore
a
mathematicàl
problem cannot be
aonsidered
as
realistiaally
oorrer.ponding
to physiaal
phenomena unless a variation of the given data in a sufficiently
mail
range leads
to an arbitrary small change in the solution. This requirement of "sta-
bility"
is not
only essential
for
meaningful
problems in
mathematicàl
physias, but
also
for approximation
methods".
An
example of a
well-posed
problem is to find a solution u of the Laplace
équa-
tion
^.l^-O
(5.1)
SX ay
in
some domain D of the plane, with the condition u
=
g on the boundary of D (Diri-
chlet
problem). It is
well known
that there exists a unique solution which
dépends
continuously
on the data. Indeed, the maximum principle
[Réf.5.2, p.255)
guarantees
that
when g is slightly perturbed into g', the corresponding solution u' is in a
neighborhood
of u. More precisely,
|g-g'lis:
implies
|u-u'|<e.
Any
problem failing to satisfy one or more of the three requirements
quoted
above
might be
called
an ill-posed (or improperly posed) problem.
Nevertheless,
this
term is usually reserved to those problems for which the second requirement
(uniqueness)
is fulfilled, but not the first and the third ones. Indeed, as we
shall
see below,
existence and continuity are in
gênerai
closely related.
The
first
who
pointed out the concepts of well- and
ill-posedness
was J.
Hadamard.
Let
us recall
his famous
example showing the
lack
of continuity on the data in the
Cauchy
problem for elliptic partial differential
équations.
Consider (5.1) with the
boundary
conditions
u(x,
0)
=
0 ,
ly (X. 0) = ^ 5in(nx)
. (5.2)
3y ^
' ' n
It
is straightforward to verify that this problem has the
following
solution:
".(J*. y) =
sin(nx) sinh(ny) .
(5-3)
163
The
term
n"l
sin(nx)
départs
from
zéro
on the x axis in an imperceptible way for n
sufficiently
large. However, because of the hyperbolic sine, the solution (5.3) be-
comes
enormous at any given distance from the x axis, provided that n is sufficient-
ly
large.
Related
to the Cauchy problem for the Laplace
équation
is the analytic continu-
ation
of functions of a complex variable. In fact, let the values of the
harmonie
function
u, i.e., the solution of (5.1), and its normal derivative
5u/?n
be known
on
some curve r. We
dénote
by f(z), z
=
x + iy, the analytic function f
=
u + iv,
where
v is the function conjugate to u. Then, on the curve r, v is related to u as
follows
z
. .
-
. .
v(z) =
/
nj^
(z')ds + constant ,
'
' "' (5.4)
where Zg
is one of the endpoints of r. Hence, if u and
3u/8n
are known on r, one
may
consider that the values of the analytic function
f{z)
on r are known. This
shows
that the solution of the Cauchy problem for the Laplace
équation
gives the
analytic
continuation of f outside r, which is therefore also an ill-posed problem.
Moreover,
it is worth noting that the
détermination
of an analytic function from
its
values on a curve r, inside the domain of regularity, is a problem which can be
reduced
to the solution of a Fredholm
intégral équation
of the first kind, by means
of
the
well-known
Cauchy formula. Therefore, it is quite natural to
guess
that also
intégral équations
of the first kind give rise to ill-posed problems. This is indeed
true,
as we shall show in
Sect.5.1.2.
To
be convinced of the practical relevance of ill-posed problems, it is sufficient
to
have a glance at the enormous amount of literature devoted to this
field.
Many
références
may be found for instance in the books by LAVRENTIEV [5.31,
TIKHONOV
and
ARSENINE [5.4] and PAYNE [5.5].
5.1.2 II 1-Posedness
and Numerical
Instability
Let
us consider the following Fredholm
intégral équation
of the first kind:
b
/ K(x,
y)f(y)dy
= 5(x)
, c
<x <d
, (5.5)
where
the kernel
K(x,
y) is supposed to be
continuous.
Assuming that there exists a
unique
solution f corresponding to g, we might add to that solution a function
f*"'(x)
= C sin(nx) where C is an arbitrary constant. From the Riemann-Lebesgue
theorem
we know that
b
lim
/
K(x,
y) sin(ny)dy
=
0 .
• f6
61
n -•
+ a
*
•
'
I
u
164
Hence, taking
the constant C and the integer n sufficiently large, we see that
widely différent
functions f give
approximately
the
same
g. As in the case of the
Cauchy problem
for the Laplace
équation, small modifications
of g can
alter
radi-
cally
the solution of (5.5).
Without
being conscious of the
i11-posedness
of this problem, one
could
try to
solve
numerically (5.5) by discretizing it. By means of some N-point quadrature for-
mula,
the
intégral
in (5.5) may be approximated by a finite
sum.
Then, supposing
that
g is given in M points, the
intégral équation
becomes a
linear
algebraic System
[Kjf =
g
(5.7)
where (K]
is a
M
«H matrix of components
K^^
=
K(x^, y^)'^^
(the are the
weight
factors
depending upon the quadrature formula used)
while
f
= ^Hy^)'i
and g =
(9(X|j,))
are
vectors in
euclidean
spaces of dimension N and M, respectively. At this point,
let
us introduce the
usual
euclidean scalar product between two
M-dimensional
vectors
m=i
and
the corresponding euclidean norm
11 9 11 ^ =
(g.
g)|i^. Now,
when g is affected by
errors,
one could
always
add to a given solution f a spurious vector u such that
Il [Klull^ = ([K]u, [K]u)|^ <
, (5.9)
where
c is an estimate of data accuracy. Let us now investigate the shape of the
set
of those u satisfying (5.9). To this purpose let us put the quadratic form (5.9)
in
a somewhat
différent
form
([KinKIu, u)^ £,
, (5.10)
where (K)*
is a N
'M
matrix denoting the adjoint (or hermitian conjugate) matrix
of (KJ.
Even if [K] is not a square matrix, [K]* [K] is a
NxN
sytnmetric, nonnega-
tive
matrix, so that it can be
diagonalized.
Let us
dénote
by the eigenvalues of
[KJ*
[K]
(»^
is
also
called a singular value of [K]) and assume that they are
ail
strictly
positive. Of course this can happen
only
if N
<M.
Then inequality (5.10)
defines
the interior of a
N-dimensional
nondegenerate ellipsoid with center at the
origin
and axes directed
along
the eigenvectors of [K]* [K]. The length of each
axis
is given by
a^ = E/X^,
n =
1,
.... N, and when the eigenvalues are ordered
in
decreasing magnitude, the length of the greatest axis is
E/*^.
while the length
of
the shortest one is
e/Xj.
The ratio between the two lengths,
a = Xj/x^^
is the
so-called
condition
number
of the matrix [K]. When
a
is much greater than one, the
ellipsoid
(5.10) contains, along certain principal directions, vectors whose eu-
clidean
norm is very large. A small change in the data vector g may produce a large
errer
in the solution (or pseudo-solution) of (5.7). The algebraic System (5.7) is
then
said to be
ill-aonditioned. '
.
165
In gênerai
this actually arises when discretizing Fredholm
équations
of the first
kind.
Indeed, let us consider for simplicity an
intégral
operator whose kernel
K(x,
y) is symmetric, and let us assume that it does not have the eigenvalue
zéro
(of
course, we also assume a
=
c and b
=
d in (5.5)1. Then, as it is
well
known,
such
an operator admits an
infinité séquence
of
real
eigenvalues (with finite
multi-
plicity) accumulating
to
zéro (Réf.5.6,
Chap.2]. Hence it is easy to
understand
that
the finer the discretization of (5.5) is (i.e., the larger N and H), the worse
conditioned
the resulting System (5.7) is.
5.1.3
General Formulation of Linear Inverse
Problems
In
order to make
précise
the concepts illustrated in the previous sections concerning
instability,
we must specify the sets to which the data and the solutions belong.
Moreover,
we must define what is meant by "closeness" in each set. This can be done
by
introducing a norm and
defining
a distance between two functions of the set as
the
norm of their
différence.
Particularly important in many applications is a norm
induced
by a scalar product (or inner product)
like
the norm of a vector in Euclidean
space.
In that way one may speak about angles and perpendiculars and perform the
familiar
geometrical constructions even for
infinité
dimensional spaces. A typical
and
very important example is the space of square integrable functions on some inter-
val
(
duct
val
(a, b). This space, called
L^(a,
b), is equipped with the following scalar pro-
b
(f,
g)
=
/
f(x)g«(x)dx
(5.11)
a
and
the induced norm
is
l|f|l =
(f.
f)*" =
( /
If(x)l^dx)'*
.
(5.12)
a
The
space L (a, b) is not only a normed space, but also a
Hilbert epace [Réf.5.7,
Chap.l).
This means that it is
oornplete
with respect to the norm, i.e., that every
Cauchy séquence
converges to an
élément
of the space. Moreover, it is a separable
space:
there exists a countably
infinité
orthonormal
séquence (u^)
such that every
élément
of the space can be indefinitely approximated in norm by linear combinations
of
the vectors
u^.
Such a
séquence
is called a basis and every function f can thus
be
written as
f = r Vn • ' ' =
n-0
where f^ =
(f,
u^)
are the Fourier components of f with respect to the basis
In
the following we
shall
often use the
so-called Parseval
equality which expresses
the
scalar product of two functions in terms of their Fourier components
166
(f.
g)
= I f„g*
. (5.14)
n=0
•
'
Anotber
norm, which is often used in the case of continuous functions on the closed
interval
[a, b], is the
so-called
uniform norm defined as follows:
l|f|| = "lax
|f(x)| , (5.15)
a ^ X
b
i.e.,
the maximal value of the modulus of f on the interval [a, b). Convergence with
respect
to the norm (5.15) is uniform convergence and the space of continuous func-
tions
is
complète
with respect to this norm.
After thèse
few
preliminaries,
we can give a more
précise
meaning to the concept
of ilt-po3ed linear
inverse
problem.
First let
us define the direct
problem:
it is a mapping of a space F of functions,
called
by CHADAN and
SABATIER [5.8]
"parameters" and by BALTES
(Réf.5.9, p.l]
"source
functions",
into a space G of functions, called "results" or "data". In the analysis
of
imaging Systems a function of F is called an "object" and a fonction of G a "noise-
less
image". We assume that F, G are normed spaces and that the mapping is given by
a
linear operator A. We Write A:
F->G
and, in mathematical language, the space 5 is
called
the range of the operator A.
Usually
the operator A is continuous. This means that to any
séquence
of
éléments
of
F, say
{f'"h,
converging to the
null élément,
there corresponds a
séquence
(Af'"')
which converges to the null
élément
of G. This property ensures the stabili-
ty
of the direct problem: any perturbation of g vanishes when the inducing pertur-
bation
of f tends to
zéro.
Besides it is always possible to introduce a norm in S
such
that G becomes a
complète
normed space. Let us assume now that the inverse
mapping A"^
exists, which is
équivalent
to require that the
équation
Af
=
0
has
only
the trivial solution f = 0. Then a theorem of Banach
(Réf.5.10, p.83]
implies
that A"^
is
also
continuous. At this point one
could
try to define the inverse prob-
lem
as the problem of solving the functional
équation
Af =
g , (5.16)
where
g is a given function of G. The continuity of
A'^ would
ensure the
stability
of
the solution.
However,
this approach is
inadéquate
for the
following
reason. The operator A
has
usually a smoothing effect. Consider, for instance, the
intégral
operator of
(5.5):
if the kernel
K(x,
y) has continuous derivatives with respect to x up to a
certain
order, then the same property
holds
for g(x). In any case the operator
attén-
uâtes
the higher frequencies - see (5.6). Now, in
gênerai,
measurement errors or noise
destroy
the smoothness properties of g: the "measured resuit" g is no longer a func-
tion
of G (in the case of imaging Systems g is the "noisy image"). In other words,
167
as
remarked by SABATIER
[Réf.5.Il, p.5],
one has to extend the space G into a larger
space
G containing
ail
possible results of
measurements.
The space G must be equipped
2
2
with
a norm suitable for describing
expérimental
errors: a L - space with the L -
norm
for instance, when one considers
mean-squared
errors, or a space of continuous
functions
with the uniform norm (5.15), when one considers maximal absolute errors.
It
happens that G is no longer a
complète
space with respect to the norm of G and
the
operator
A"'
is no longer continuous. The
inversa
problem turns
oui ^r.
ill-posed problem.
Besides the
équation
Af
=
g might have no solution, because g
does
not necessarily belong to G. We see that the questions of existence and con-
tinuity
are closely connected.
We shall call
F the solution space and G the data space . If we assume a simple
additive model
for noise and measurements errors, then we have
Af
+ h
=
g
(5.17)
Since
both f (the solution) and h (the noise) are unknown and since the
équation
Af =
g might have no solution, it follows that:
I)
the best we can do is to search for some f reproducing the given g within a
tolerable
uncertainty. The problem is then reformulated as follows: find f such that
llAf
-
gllg
<
c
. (5.18)
where c
is the "size" of the noise, measured with the norm of G.
II)
the previous formulation is
adéquate
if the set H of
ail
the functions f satis-
fying
(5.18) is bounded and sufficiently
"small"
so that any
élément
of H might be
taken
as an approximation of the "true" solution. However, when
A"'
is not contin-
uous,
H is not bounded. In other words, given an arbitrary number A, one can find
Iwo
fonctions
fU),
f(2) satisfying (5.18) and such that
jl
f
( 1
) - f
(2) |i p-..
This
is
precisely the meaning of Hadamard's example discussed in Sect. 5.1.1. In such a
case,
as we shall see
below,
some
supplementary
constraints on the solution are
necessary.
The
situation illustrated above is quite similar to that of
i11-conditioned
Sys-
tems
as described in
Sect.5.1.2.
Evidently, in the finite dimensional case the set
H
1s always bounded, but it is very large
along
some directions.
Finally
we want to remark that, when the inverse operator does not exist, the
previous
analysis can be repeated considering for (5.16) only solutions of minimal
norm. Thèse
solutions can be expressed in tenus of the generalised
inverae
(or
pseudo-inverse)
of the operator A
[5.12].
The generalized inverse is an extension,
for operators
in functional spaces, of the
Moore-Penrose
inverse for matrices. When
the
operator A has a smoothing effect, it happens that its generalized inverse is
not
continuous with respect to the norm of the data space G and therefore we get
again
an ill-posed problem.