scispace - formally typeset
Search or ask a question
Journal ArticleDOI

A Newton Method for Shape-Preserving Spline Interpolation

01 Jun 2002-Siam Journal on Optimization (Society for Industrial and Applied Mathematics)-Vol. 13, Iss: 2, pp 588-602
TL;DR: This paper proves local quadratic convergence of their proposed Newton-type method for shape-preserving interpolation by viewing it as a semismooth Newton method and presents a modification of the method which has global quadRatic convergence.
Abstract: In 1986, Irvine, Marin, and Smith proposed a Newton-type method for shape-preserving interpolation and, based on numerical experience, conjectured its quadratic convergence. In this paper, we prove local quadratic convergence of their method by viewing it as a semismooth Newton method. We also present a modification of the method which has global quadratic convergence. Numerical examples illustrate the results.

Summary (1 min read)

If the integral over [a, b] of the piecewise linear function (

  • Furthermore, in this case quadratic convergence of the Newton method would follow directly from [13] .
  • The following example of dimension 2 shows that such an argument does not work.
  • Hence the function above is not piecewise smooth.
  • Then the positive semidefiniteness of the elements of ∂F − (λ) follows from the fact that any matrix in the generalized Jacobian of the gradient of a convex function must be symmetric and positive semidefinite.
  • By combining the above lemmas and applying Theorem 1.1, the authors obtain the main result of this paper which settles the question posed in [11] .

4. Numerical results.

  • In Figures 1-5 , the dashed line is for the resulting shape-preserving cubic spline (using the data obtained with the starting point λ 0 = sign(d)); the solid line is for the natural spline (using the MATLAB SPLINE function), and "o" stands for the original given data.
  • In Table 1 for results of the numerical experiments the authors use the following notation: 1 , they observe that Algorithm 3.2 converges rapidly to the solution from both starting points for all problems except Example 4.4 (y 9 = 4), to which the algorithm within 30 iterations failed to produce an approximate solution meeting the required accuracy.
  • These observations indicate that how far away from zero each divided difference is may make a big difference in the numerical performance of the algorithm.
  • The problem is, however, nonsmooth, and here the authors are entering a new territory.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

A NEWTON METHOD FOR SHAPE-PRESERVING SPLINE
INTERPOLATION
ASEN L. DONTCHEV
, HOU-DUO QI
, LIQUN QI
§
, AND HONGXIA YIN
SIAM J. O
PTIM.
c
2002 Society for Industrial and Applied Mathematics
Vol. 13, No. 2, pp. 588–602
This work is dedicated to Professor Jochem Zowe
Abstract. In 1986, Irvine, Marin, and Smith proposed a Newton-type method for shape-
preserving interpolation and, based on numerical experience, conjectured its quadratic convergence.
In this paper, we prove local quadratic convergence of their method by viewing it as a semismooth
Newton method. We also present a modification of the method which has global quadratic conver-
gence. Numerical examples illustrate the results.
Key words. shape-preserving interpolation, splines, semismooth equation, Newton’s method,
quadratic convergence
AMS subject classifications. 41A29, 65D15, 49J52, 90C25
PII. S1052623401393128
1. Introduction. Given nodes a = t
1
<t
2
< ··· <t
N+2
= b and values
y
i
= f(t
i
),i =1,...,N +2,N 3, of an unknown function f :[a, b] R, the
standard interpolation problem consists of finding a function s from a given set S of
interpolants such that s(t
i
)=y
i
, i =1,...,N+ 2. When S is the set of twice contin-
uously differentiable piecewise cubic polynomials across t
i
, we deal with cubic spline
interpolation. The problem of cubic spline interpolation can be viewed in various
ways; the closest to this paper is the classical Holladay variational characterization,
according to which the natural cubic interpolating spline can be defined as the unique
solution of the following optimization problem:
min f

2
subject to f(t
i
)=y
i
,i=1,...,N +2,(1)
where ·denotes the norm of L
2
[a, b]. With a simple transformation, this problem
can be written as a nearest point problem in L
2
[a, b]: find the projection of the origin
on the intersection of the hyperplanes
u L
2
[a, b] |
b
a
u(t)B
i
(t)dt = d
i
,i=1,...,N
,
where B
i
are the piecewise linear normalized B-splines with support [t
i
,t
i+2
] and d
i
are the second divided differences.
Received by the editors July 29, 2001; accepted for publication (in revised form) March 26, 2002;
published electronically October 1, 2002.
http://www.siam.org/journals/siopt/13-2/39312.html
Mathematical Reviews, Ann Arbor, MI 48107 (ald@ams.org).
School of Mathematics, University of New South Wales, Sydney 2052, NSW, Australia (hdqi@
maths.unsw.edu.au). The research of this author was supported by the Australian Research Council.
§
Department of Applied Mathematics, Hong Kong Polytechnic University, Hung Hom, Kowloon,
Hong Kong (maqilq@polyu.edu.hk). This author’s work is supported by the Hong Kong Research
Grant Council, grant PolyU 5296/02P.
Department of Mathematics, Graduate School, Chinese Academy of Sciences, P.O. Box 3908,
Beijing 100039, People’s Republic of China (hxyin@maths.unsw.edu.au). This author’s work was
done while the author was visiting the University of New South Wales, Australia, and was supported
by the Australian Research Council.
588

A NEWTON METHOD FOR CONSTRAINED INTERPOLATION 589
Since the mid ’80s, after the ground-breaking paper of Micchelli et al. [15], the
attention of a number of researchers has been attracted to spline interpolation prob-
lems with constraints. For example, if we add to problem (1) the additional constraint
f

0, we obtain a convex interpolation problem; provided that the data are “con-
vex,” then a convex interpolant “preserves the shape” of the data. If we add the
constraint f
0, we obtain a monotone interpolation problem. Central to our analy-
sis here is a subsequent paper by Irvine, Marin, and Smith [11], who rigorously defined
the problem of shape-preserving spline interpolation and laid the groundwork for its
numerical analysis. In particular, they proposed a Newton-type method and, based
on numerical examples, conjectured its fast (quadratic) theoretical convergence. In
the present paper we prove this conjecture.
We approach the problem of Irvine, Marin, and Smith [11] in a new way, by
using recent advances in optimization. It is now well understood that, in general,
the traditional methods based on standard calculus may not work for optimization
problems with constraints; however, such problems can be reformulated as nonsmooth
problems that need special treatment. The corresponding theory emerged already in
the ’70s, championed by the works of R. T. Rockafellar and his collaborators, and is
now becoming a standard tool for more and more theoretical and practical problems.
The present paper is an example of how nonsmooth analysis can be applied to solve
a problem from numerical analysis that hasn’t been solved for quite a while.
Before stating the problem of shape-preserving interpolation that we consider in
this paper, we briefly review the result of nonsmooth analysis which provides the basis
for this work.
For a locally Lipschitz continuous function G : R
n
R
n
, the generalized Jaco-
bian ∂G(x)ofG at x in the sense of Clarke [2] is the convex hull of all limits obtained
along sequences on which G is differentiable:
∂G(x)=co
lim
x
j
x
G(x
j
) | G is differentiable at x
j
R
n
.
The generalized Newton method for the (nonsmooth) equation G(x) = 0 has the
following form:
x
k+1
= x
k
V
1
k
G(x
k
),V
k
∂G(x
k
).(2)
A function G : R
n
R
m
is strongly semismooth at x if it is locally Lipschitz and
directionally differentiable at x, and for all h 0 and V ∂G(x + h) one has
G(x + h) G(x) Vh= O(h
2
).
The local convergence of the generalized Newton method for strongly semismooth
equations is summarized in the following fundamental result, which is a direct gener-
alization of the classical theorem of quadratic convergence of the Newton method.
Theorem 1.1 (see [16, Theorem 3.2]). Let G : R
n
R
n
be strongly semismooth
at x
and let G(x
)=0. Assume that all elements V of the generalized Jacobian
∂G(x
) are nonsingular matrices. Then every sequence generated by the method (2)
is q-quadratically convergent to x
, provided that the starting point x
0
is sufficiently
close to x
.
In the remaining part of the introduction we review the method of Irvine, Marin,
and Smith [11] for shape-preserving cubic spline interpolation and also briefly discuss
the contents of this paper. Let {(t
i
,y
i
)}
N+2
1
be given interpolation data and let
d
i
,i =1, 2,...,N, be the associated second divided differences. Throughout the

590 A. L. DONTCHEV, H.-D. QI, L. QI, AND H. YIN
paper we assume that d
i
= 0 for all i =1,...,N; we will discuss this assumption
later. Define the following subsets
i
,i=1, 2, 3, of [a, b]:
1
:= {[t
i
,t
i+1
]| d
i1
> 0 and d
i
> 0},
2
:= {[t
i
,t
i+1
]| d
i1
< 0 and d
i
< 0},
3
:= {[t
i
,t
i+1
]| d
i1
d
i
< 0}.
Also, let
[t
1
,t
2
]
1
if d
1
> 0,
2
if d
1
< 0,
[t
N+1
,t
N+2
]
1
if d
N
> 0,
2
if d
N
< 0.
The problem of shape-preserving interpolation as stated by Micchelli et al. [15] is as
follows:
minimize f

2
(3)
subject to f(t
i
)=y
i
,i=1, 2,...,N +2,
f

(t) 0,t
1
,f

(t) 0,t
2
,
f W
2,2
[a, b].
Here W
2,2
[a, b] denotes the Sobolev space of functions with absolutely continuous first
derivatives and second derivatives in L
2
[a, b]. The inequality constraint on the set
1
(resp.,
2
) means that the interpolant preserves the convexity (resp., concavity) of
the data; for more details, see [11, p. 137].
Micchelli et al. [15, Theorem 4.3] showed that the solution of the problem (3)
exists and is unique, and its second derivative has the following form:
f

(t)=
N
i=1
λ
i
B
i
(t)
+
X
1
(t)
N
i=1
λ
i
B
i
(t)
X
2
(t)(4)
+
N
i=1
λ
i
B
i
(t)
X
3
(t),
where λ =(λ
1
,...,λ
N
)
T
is a vector in R
N
, a
+
=max{0,a},(a)
=(a)
+
, and
X
is the characteristic function of the set Ω. This result can also be deduced, as
shown first in [4], from duality in optimization; specifically, here λ is the vector of
the Lagrange multipliers associated with the equality (interpolation) constraints. For
more on duality in this context, see the discussion in our previous paper [5]. In short,
the optimality condition of the problem dual to (3) has the form of the nonlinear
equation
F (λ)=d,(5)
where d =(d
1
,...,d
N
)
T
and the vector function F : R
N
R
N
has components
F
i
(λ)=
[t
i
,t
i+2
]
1
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt
[t
i
,t
i+2
]
2
N
l=1
λ
l
B
l
(t)
B
i
(t)dt

A NEWTON METHOD FOR CONSTRAINED INTERPOLATION 591
+
[t
i
,t
i+2
]
3
N
l=1
λ
l
B
l
(t)
B
i
(t)dt, i =1, 2,...,N.(6)
Irvine, Marin, and Smith [11] proposed the following method for solving equation
(5): Given λ
0
R
N
, λ
k+1
is a solution of the linear system
M(λ
k
)(λ
k+1
λ
k
)=F (λ
k
)+d,(7)
where M(λ) R
N×N
is the tridiagonal symmetric matrix with components
(M(λ))
ij
=
b
a
P (λ, t)B
i
(t)B
j
(t)dt.
Here
P (λ, t):=
N
l=1
λ
l
B
l
(t)
0
+
X
1
(t)+
N
l=1
λ
l
B
l
(t)
0
X
2
(t)+X
3
(t),(8)
where
(τ)
0
+
:=
1ifτ>0,
0 otherwise,
(τ)
0
:= (τ)
0
+
.
Since the matrix M resembles the Jacobian of F (which may not exist for some λ,
and then M is a kind of “directional Jacobian,” more precisely, as we will see later,
an element of the generalized Jacobian), the method (7) has been named the Newton
method. It was also observed in [11] that the Newton-type iteration (7) reduces to
M(λ
k
)λ
k+1
= d; that is, no evaluations of the function F are needed during iterations.
In our previous paper [5], we considered the problem of convex spline interpola-
tion, that is, with
1
=[a, b], and proved local superlinear convergence of the corre-
sponding version of the Newton method (7). In a subsequent paper [6], by a more
detailed analysis of the geometry of the dual problem, we obtained local quadratic
convergence of the Newton method, again for convex interpolation. In this paper, we
consider the shape-preserving interpolation problem originally stated in Irvine, Marin,
and Smith [11] and prove their conjecture that the method is locally quadratically
convergent. As a side result, we observe that the solution of the problem considered
is Lipschitz continuous with respect to the interpolation values. In section 3 we give
a modification of the method which has global quadratic convergence. Results of
extensive numerical experiments are presented in section 4.
As for related results, the conjecture of Irvine, Marin, and Smith [11] was proved
in [1] under an additional condition which turned out to be equivalent to smoothness
of the function F in (5). Also, a positive answer to this conjecture without additional
assumptions was announced in [10], but a proof was never made available to us.
2. Local quadratic convergence. For notational convenience, we introduce a
“dummy” node t
0
with corresponding λ
0
= 0 and B
0
(t) = 0; then, for every i, the
sum
N
l=1
λ
l
B
l
(t) restricted to [t
i
,t
i+1
] has the form λ
i1
B
i1
(t)+λ
i
B
i
(t). Our first
result concerns continuity and differentiability properties of the function F defined in
(6).
Lemma 2.1. The function F with components defined in (6) is strongly semi-
smooth.

592 A. L. DONTCHEV, H.-D. QI, L. QI, AND H. YIN
Proof. The claim is merely an extension of [6, Proposition 2.4], where it is proved
that the functions
t
i+1
t
i
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt,
t
i+2
t
i+1
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt,
and
t
i+2
t
i
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt
are strongly semismooth. Hence the function
[t
i
,t
i+2
]
1
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt
is strongly semismooth by noticing that
[t
i
,t
i+2
]
1
∈{[t
i
,t
i+1
], [t
i+1
,t
i+2
], [t
i
,t
i+2
], ∅} .
We note that the function
[t
i
,t
i+2
]
3
N
l=1
λ
l
B
l
(t)
B
i
(t)dt
is linear and therefore is strongly semismooth. Since either [t
i
,t
i+2
]
1
= or
[t
i
,t
i+2
]
2
= , F
i
is given either by
F
i
(λ)=
[t
i
,t
i+2
]
1
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt(9)
+
[t
i
,t
i+2
]
3
N
l=1
λ
l
B
l
(t)
B
i
(t)dt
or by
F
i
(λ)=
[t
i
,t
i+2
]
2
N
l=1
λ
l
B
l
(t)
B
i
(t)dt(10)
+
[t
i
,t
i+2
]
3
N
l=1
λ
l
B
l
(t)
B
i
(t)dt.
A composite of strongly semismooth functions is strongly semismooth [8, Theorem
19]. Hence the function F
i
by (9) is strongly semismooth. If F
i
is given by (10), then
F
i
(λ)=
[t
i
,t
i+2
]
2
N
l=1
λ
l
B
l
(t)
+
B
i
(t)dt +
[t
i
,t
i+2
]
3
N
l=1
λ
l
B
l
(t)
B
i
(t)dt.
Again from [8, Theorem 19], the first part of F
i
is strongly semismooth, which in turn
implies the strong semismoothness of F
i
. We conclude that F is strongly semismooth
since each component of F is strongly semismooth.

Citations
More filters
01 Jan 2004
TL;DR: The following sections are a start on organizing references on stable distributions by topic, and please provide all references in BibTeX form, especially if you have more than one or two additions.
Abstract: The following sections are a start on organizing references on stable distributions by topic. It is far from complete. Starting on page 18 there is an extensive list of papers on stable distributions, many of which are not included in the first section. Some of the papers there do not directly refer to stable distributions. Someday I may have the time to edit those out, but for now please ignore those references. This list includes a bibliography file provided by Gena Samorodnitsky from Cornell University. I would like to keep this list correct and up-to-date. If you have corrections or additions, please e-mail them to me at the above address, and suggest where to place your references in one of the sections below. A sentence or two summarizing the content would be useful. Please provide all references in BibTeX form, especially if you have more than one or two additions. (See http://en.wikipedia.org/wiki/BibTeX for basic information on BibTEX.) Please send a copy of your papers along.

59 citations


Additional excerpts

  • ...Doob, J. (1953). Stochastic Processes....

    [...]

Journal ArticleDOI
TL;DR: The global convergence of the algorithm for shape restricted smoothing splines subject to general polyhedral control constraints is shown, using techniques from nonsmooth analysis and polyhedral theory.

18 citations


Cites background from "A Newton Method for Shape-Preservin..."

  • ..., [10, 12, 16, 23, 41], and many critical questions remain open in analysis and computation when general dynamics and control constraints are taken into account....

    [...]

  • ...Hence the computation of the smoothing spline boils down to determining parameters of a polynomial on each interval, which can be further reduced to a quadratic or semidefinite program that attains efficient algorithms [10, 12]....

    [...]

Posted Content
TL;DR: In this paper, the authors developed a method to compute the penalized least squares estimators (PLSEs) of the parametric and nonparametric components given independent and identically distributed (i.i.d.) data.
Abstract: We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernels or regression splines, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.)~data. We prove the consistency and find the rates of convergence of the estimators. We establish asymptotic normality under under mild assumption and prove asymptotic efficiency of the parametric component under homoscedastic errors. A finite sample simulation corroborates our asymptotic theory. We also analyze a car mileage data set and a Ozone concentration data set. The identifiability and existence of the PLSEs are also investigated.

18 citations

Journal ArticleDOI
TL;DR: It is shown that f is a strongly semismooth function if g is continuous and B is affine with respect to t and stronglySemismooth withrespect to x, i.e., B(x, t) = u(x)t + v(x), where u and v are two strongly Semismooth functions in ℝn.
Abstract: As shown by an example, the integral function f : {\bb R}n → {\bb R}, defined by f(x) e ∫ab[B(x, t)]+g(t) dt, may not be a strongly semismooth function, even if g(t) ≡ 1 and B is a quadratic polynomial with respect to t and infinitely many times smooth with respect to x. We show that f is a strongly semismooth function if g is continuous and B is affine with respect to t and strongly semismooth with respect to x, i.e., B(x, t) e u(x)t + v(x), where u and v are two strongly semismooth functions in {\bb R}n. We also show that f is not a piecewise smooth function if u and v are two linearly independent linear functions, g is continuous and g n 0 in [a, b], and n ≥ 2. We apply the first result to the edge convex minimum norm network interpolation problem, which is a two-dimensional interpolation problem.

16 citations

Journal ArticleDOI
TL;DR: The problem of finding an interpolating curve passing through prescribed points in the Euclidean space is studied as an optimal control problem and simple but effective tools of optimal control theory are employed.
Abstract: We study the problem of finding an interpolating curve passing through prescribed points in the Euclidean space. The interpolating curve minimizes the pointwise maximum length, i.e., $L^\infty$-norm, of its acceleration. We reformulate the problem as an optimal control problem and employ simple but effective tools of optimal control theory. We characterize solutions associated with singular and nonsingular controls. Some of the results we obtain are new even for the scalar interpolating function case. We reduce the infinite-dimensional interpolation problem to an ensuing finite-dimensional one and derive closed form expressions for interpolating curves. Consequently we devise efficient numerical techniques and illustrate them with examples.

13 citations


Additional excerpts

  • ...[13] D ow nl oa de d 11 /1 6/ 14 to 1 29 ....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: A general line search scheme is introduced which easily allows to define and analyze known and new semismooth algorithms for the solution of nonlinear complementarity problems and enucleate the basic assumptions that a search direction has to enjoy in order to guarantee global convergence, local superlinear/quadratic convergence or finite convergence.
Abstract: In this paper we introduce a general line search scheme which easily allows us to define and analyze known and new semismooth algorithms for the solution of nonlinear complementarity problems. We enucleate the basic assumptions that a search direction to be used in the general scheme has to enjoy in order to guarantee global convergence, local superlinear/quadratic convergence or finite convergence. We examine in detail several different semismooth algorithms and compare their theoretical features and their practical behavior on a set of large-scale problems.

88 citations


"A Newton Method for Shape-Preservin..." refers methods in this paper

  • ...For more discussion of the inexact Newton method, we refer to [3, 7, 14]....

    [...]

Journal ArticleDOI
TL;DR: This paper solves a class of constrained optimization problems that lead to algorithms for the construction of convex interpolants to convex data.
Abstract: In this paper, we solve a class of constrained optimization problems that lead to algorithms for the construction of convex interpolants to convex data.

83 citations


"A Newton Method for Shape-Preservin..." refers background in this paper

  • ...[15] is as follows: minimize ‖f ′′‖2 (3)...

    [...]

  • ...[15], the attention of a number of researchers has been attracted to spline interpolation problems with constraints....

    [...]

Journal ArticleDOI
TL;DR: In this article, the authors focus on the analysis of numerical techniques for solving the nonlinear system and on the theoretical issues that arise when certain extensions of the convex interpolation problem are considered.
Abstract: Numerical and theoretical questions related to constrained interpolation and smoothing are treated. The prototype problem is that of finding the smoothest convex interpolant to given univariate data. Recent results have shown that this convex programming problem with infinite constraints can be recast as a finite parametric nonlinear system whose solution is closely related to the second derivative of the desired interpolating function. This paper focuses on the analysis of numerical techniques for solving the nonlinear system and on the theoretical issues that arise when certain extensions of the problem are considered. In particular, we show that two standard iteration techniques, the Jacobi and Gauss-Seidel methods, are globally convergent when applied to this problem. In addition we use the problem structure to develop an efficient implementation of Newton's method and observe consistent quadratic convergence. We also develop a theory for the existence, uniqueness, and representation of solutions to the convex interpolation problem with nonzero lower bounds on the second derivative (strict convexity). Finally, a smoothing spline analogue to the convex interpolation problem is studied with reference to the computation of convex approximations to noisy data.

81 citations


"A Newton Method for Shape-Preservin..." refers background or methods or result in this paper

  • ...We report results with the starting point e, the vector of all ones in R , which is commonly selected as a starting point in algorithms for convex best interpolations; see [11, 6]....

    [...]

  • ...We approach the problem of Irvine, Marin, and Smith [11] in a new way, by using recent advances in optimization....

    [...]

  • ...1, we obtain the main result of this paper which settles the question posed in [11]....

    [...]

  • ...As for related results, the conjecture of Irvine, Marin, and Smith [11] was proved in [1] under an additional condition which turned out to be equivalent to smoothness of the function F in (5)....

    [...]

  • ...It was also observed in [11] that the Newton-type iteration (7) reduces to M(λ)λ = d; that is, no evaluations of the function F are needed during iterations....

    [...]

Book ChapterDOI
01 Jan 1996
TL;DR: In this article, the local behavior of inexact Newton methods for the solution of a semismooth system of equations was studied and a complete characterization of the Q-superlinear and Q-quadratic convergence was given.
Abstract: We consider the local behaviour of inexact Newton methods for the solution of a semismooth system of equations. In particular, we give a complete characterization of the Q-superlinear and Q-quadratic convergence of inexact Newton methods. We then apply these results to a particular semismooth system of equations arising from variational inequality problems, and present a globally and locally fast convergent algorithm for its solution.

54 citations


"A Newton Method for Shape-Preservin..." refers methods in this paper

  • ...For more discussion of the inexact Newton method, we refer to [3, 7, 14]....

    [...]

Journal ArticleDOI
TL;DR: This note brings out a property of the functions that enter such equations, for instance through penalty expressions, that is increasingly important in the numerical treatment of complementarity problems and models of equilibrium.
Abstract: Piecewise smooth equations are increasingly important in the numerical treatment of complementarity problems and models of equilibrium. This note brings out a property of the functions that enter such equations, for instance through penalty expressions.

42 citations