# A Class of Singular Control Problems and the Smooth Fit Principle

01 Nov 2008-Siam Journal on Control and Optimization (Society for Industrial and Applied Mathematics)-Vol. 47, Iss: 6, pp 3076-3099

TL;DR: In particular, the authors analyzes a class of singular control problems for which value functions are not necessarily smooth and provides necessary and sufficient conditions for the well-known smooth fit principle, along with the regularity of the value functions.

Abstract: This paper analyzes a class of singular control problems for which value functions are not necessarily smooth. Necessary and sufficient conditions for the well-known smooth fit principle, along with the regularity of the value functions, are given. Explicit solutions for the optimal policy and for the value functions are provided. In particular, when payoff functions satisfy the usual Inada conditions, the boundaries between action and continuation regions are smooth and strictly monotonic, as postulated and exploited in the existing literature (see [A. K. Dixit and R. S. Pindyck, Investment under Uncertainty, Princeton University Press, Princeton, NJ, 1994]; [M. H. A. Davis et al., Adv. in Appl. Probab., 19 (1987), pp. 156-176]; [T. O. Kobila, Stochastics Stochastics Rep., 43 (1993), pp. 29-63]; [A. B. Abel and J. C. Eberly, J. Econom. Dynam. Control, 21 (1997), pp. 831-852]; [A. Oksendal, Finance Stoch., 4 (2000), pp. 223-250]; [J. A. Scheinkman and T. Zariphopoulou, J. Econom. Theory, 96 (2001), pp. 180-207]; [A. Merhi and M. Zervos, SIAM J. Control Optim., 46 (2007), pp. 839-876]; and [L. H. Alvarez, A General Theory of Optimal Capacity Accumulation under Price Uncertainty and Costly Reversibility, Working Paper, Helsinki Center of Economic Research, Helsinski, Finland, 2006]). Illustrative examples for both smooth and nonsmooth cases are discussed to emphasize the pitfall of solving singular control problems with a priori smoothness assumptions.

## Summary (2 min read)

Jump to: [1 Introduction] – [2.1 Problem] – [3 Derivation of the Solution] – [3.1.2 Solution of the Optimal Switching Problems] – [3.2 Derivation of the Optimal Switching Controls] – [3.2.2 Consistency of the Switching Controls] – [3.3 Derivation of the Optimal Singular Control] – [4 Regularity, Smooth Fit and Region Characterization] – [4.1 Regularity and Smooth Fit] – [5 Examples and Discussions] and [5.2 Discussion]

### 1 Introduction

- Consider the following problem in reversible investment/capacity planning that arises naturally in resource extraction and power generation.
- Facing the risk of market uncertainty, companies extract resources (such as oil or gas) and choose the capacity level in response to the random fluctuation of market price for the resources, subject to some capacity constraints, as well as the associated costs for capacity expansion and contraction.
- Both necessary and sufficient conditions on the differentiability of the value function and on the smooth fit principle are established.
- In fact, when the payoff function is not smooth, this paper is the first to rigorously characterize the action and no-action regions, and to explicitly construct both the optimal policy and the value function.

### 2.1 Problem

- This is a continuous time formulation of the aforementioned risk management problem.
- The goal of the company is to maximize its long-term profit with a payoff function that depends on both the resource extraction rate and the market price, with a form of H(Y )Xλ.
- And the control problem can be reduced to an equivalent yet simpler singular control problem .
- In fact, this equivalent control problem is more standard in the existing literature for both singular control and (ir)reversible investment.

### 3 Derivation of the Solution

- The derivation goes as follows: first, a collection of corresponding optimal switching problems is established and solved; then, the consistency of the optimal switching controls is proved; finally, the existence of the corresponding integrable optimal singular control (ξ̂+, ξ̂−) ∈.
- A′y is established and the corresponding value function are derived.
- This method is built on the general correspondence between singular controls and switching controls outlined in Guo and Tomecek (2007).

### 3.1.2 Solution of the Optimal Switching Problems

- The derivation of the value function proceeds analogously to the derivation for the case of h(z) >.
- Moreover, these equations depend on z only through ν(z) and κ(z), implying that there exist unique constants κ, ν such that κ(z) ≡ κ and ν(z) ≡ ν for all z.

### 3.2 Derivation of the Optimal Switching Controls

- The authors then define a collection of these switching controls and prove that this collection satisfies the consistency property from Guo and Tomecek (2007), which implies that it corresponds to an admissible singular control.
- Note that since the regime values κ̂n are alternating, it suffices to define the switching times τ̂n.

### 3.2.2 Consistency of the Switching Controls

- The collection of switching controls (α̂(x, z))z∈(a,b) is consistent.
- Then the consistency follows from the following lemmas.
- For simplicity, the authors omit the dependence on x from the notation.
- Note that after the first switch, each subsequent switch requires that Xx move from (0, F (z)] to [G(z),∞) or vice versa.
- In particular, this quantity is independent of z. Since log(Xx) is a Brownian motion with drift, its sample paths are almost surely uniformly continuous on [0, t].

### 3.3 Derivation of the Optimal Singular Control

- Clearly, the following proposition holds, from Merhi and Zervos (2007).
- Therefore, after verifying the Standing Assumptions A1, A2 and A3, one can invoke Guo and Tomecek (2007, Proposition 2.13, Theorem 3.13, Theorem 3.10) and conclude that there exists a corresponding integrable singular control (ξ̂+, ξ̂−) ∈.
- To prove Theorem 2.5, the authors first establish some Lemmas.
- (The second one follows by a similar argument, and the last one is immediate from the definition of C and the first two.).

### 4 Regularity, Smooth Fit and Region Characterization

- The authors shall establish necessary and sufficient conditions for the smooth fit principle by exploiting both the structure of the payoff function and the explicit solution of the value function.
- This analysis leads to the proof of Theorem 2.6 on region characterization.

### 4.1 Regularity and Smooth Fit

- Therefore, d(x, ·) has only countably many discontinuities.
- This theorem follows naturally from the following Lemma and the Proposition.
- Thus fx must have at least two positive roots.

### 5 Examples and Discussions

- By now, it is clear from their analysis that without sufficient smoothness of the payoff function, the value function may be non-differentiable and the boundaries may be non-smooth or not strictly monotonic.
- Moreover, when the payoff function H is not continuously differentiable, the interior of C may not be simply connected.
- Note, however, the regions S0, S1 and C are mutually disjoint and simply connected by the monotonicity of F and G.
- The authors elaborate on these points with some concrete examples.

### 5.2 Discussion

- The above examples demonstrate how the regularity assumptions typically assumed by the traditional HJB approach may fail.
- Furthermore, Example 5.3 shows that in general, one may not have the smoothness of the boundary, as the boundaries F and G are not necessarily continuous or not even strictly increasing.

Did you find this useful? Give us your feedback

UC Berkeley

Coleman Fung Risk Management Research Center Working

Papers 2006-2013

Title

A Class of Singular Control Problems and the Smooth Fit Principle

Permalink

https://escholarship.org/uc/item/8w46j0td

Authors

Guo, Xin

Tomecek, Pascal

Publication Date

2007-10-07

eScholarship.org Powered by the California Digital Library

University of California

University of California

Berkeley

Working Paper # 2007 -05

A Class of Singular Control Problems

and the Smooth Fit Principle

Xin Guo, UC Berkeley and Cornell University

Pascal Tomecek, Cornell University

October 7,2007

A Class of Singular Control Problems and the Smo oth Fit

Principle

∗

Xin Guo

†

UC Berkeley and Cornell

Pascal Tomecek

‡

Cornell University

October 7, 2007

Abstract

This paper analyzes a class of singular control problems for which value functions

are not necessarily smooth. Necessary and suﬃcient conditions for the well-known

smooth ﬁt principle, along with the regularity of the value functions, are given. Explicit

solutions for the optimal policy and for the value functions are provided. In particular,

when payoﬀ functions satisfy the usual Inada conditions, the boundaries between action

and no-action regions are smooth and strictly monotonic as postulated and exploited in

the existing literature (Dixit and Pindyck (1994); Davis, Dempster, Sethi, and Vermes

(1987); Kobila (1993); Abel and Eberly (1997); Øksendal (2000); Scheinkman and

Zariphopoulou (2001); Merhi and Zervos (2007); Alvarez (2006)). Illustrative examples

for both smooth and non-smooth cases are discussed, to highlight the pitfall of solving

singular control problems with a priori smoothness assumptions.

∗

This research is generously supported by grants from OpenLink Fund at the Coleman Fung Risk Man-

agement Research Center at UC Berkeley

†

Department of Industrial Engineering and Operations Research, UC Berkeley, CA 94720-1777, USA,

Phone (510) 642 3615, email: xinguo@ieor.berkeley.edu.

‡

Scho ol of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY 14853-3801,

USA, Phone (607) 255 1270, Fax (607) 255 9129, email: pascal@orie.cornell.edu.

1

1 Introduction

Consider the following problem in reversible investment/capacity planning that arises nat-

urally in resource extraction and power generation. Facing the risk of market uncertainty,

companies extract resources (such as oil or gas) and choose the capacity level in response

to the random ﬂuctuation of market price for the resources, subject to some capacity con-

straints, as well as the associated costs for capacity expansion and contraction. The goal of

the company is to maximize its long-term proﬁt, subject to these constraints and the rate

of resource extraction.

This kind of capacity planning with price uncertainty and partial (or no) reversibility

originated from the economics literature and has since attracted the interest of the ap-

plied mathematics community. (See Dixit and Pindyck (1994); Brekke and Øksendal (1994);

Davis, Dempster, Sethi, and Vermes (1987); Kobila (1993); Abel and Eberly (1997); Baldurs-

son and Karatzas (1997); Øksendal (2000); Scheinkman and Zariphopoulou (2001); Wang

(2003); Chiarolla and Haussmann (2005); Guo and Pham (2005) and the references therein.)

Mathematical analysis of such control problems has evolved considerably from the initial

heuristics to the more sophisticated and standard stochastic control approach, and from

the very special case study to general payoﬀ functions. (See Harrison and Taksar (1983);

Karatzas (1985); Karatzas and Shreve (1985); El Karoui and Karatzas (1988, 1989); Ma

(1992); Davis and Zervos (1994, 1998); Boetius and Kohlmann (1998); Alvarez (2000, 2001 );

Bank (2005); Boetius (2005)). Most recently, Merhi and Zervos (2007) analyzed this problem

in great generality and provided explicit solutions for the special case where the payoﬀ is of

Cobb-Douglas type. Their method is to directly solve the HJB equations, assuming certain

regularity conditions for both the value function and the boundaries between the action and

no-action regions. Guo and Tomecek (2007) later established suﬃcient conditions for the

smoothness of the value function by connecting the singular control problem with a collection

of optimal switching problems.

However, for many singular control problems in reversible investment and in areas such

as queuing and wireless communications (Martins, Shreve, and Soner (1996); Assaf (1997);

Harrison and Van Mieghem (1997); Ata, Harrison, and Shepp (2005)), there is no regularity

for either the value function or the boundaries. Therefore, two important mathematical issues

remain: 1) necessary conditions for regularity properties; and 2) characterization for the

value function and for the action and no-action regions when these regularity conditions fail.

Understanding these issues is especially important in cases where only numerical solutions

are available, and for which the assumption on the degree of the smoothness is wrong (see

also discussions in Section 5.2).

This paper addresses these two issues via the study of a class of singular control problems.

Both necessary and suﬃcient conditions on the diﬀerentiability of the value function and on

the smooth ﬁt principle are established. Moreover, these conditions lead to a derivative-

based characterization of the investment, disinvestment and continuation regions even for

non-smooth value functions. In fact, when the payoﬀ function is not smooth, this paper is the

2

ﬁrst to rigorously characterize the action and no-action regions, and to explicitly construct

both the optimal policy and the value function. To be consistent with the literature in

(ir)reversible investment, the running payoﬀ function in this paper depends on the resource

extraction rate and the market price in the form of H(Y )X

λ

. It is worth noting that H(·)

is any concave function of the capacity, and may be neither monotonic nor diﬀerentiable.

This includes the special cases investigated by Guo and Pham (2005); Merhi and Zervos

(2007); Guo and Tomecek (2007). In particular, when H satisiﬁes the well-known Inada

conditions (i.e., continuously diﬀerentiable, strictly increasing, strictly concave, with H(0) =

0, H

0

(0

+

) = ∞, H

0

(∞) = 0), our results show that the boundaries between regions are

indeed continuous and strictly increasing as postulated and exploited in previous works:

Dixit and Pindyck (1994); Davis, Dempster, Sethi, and Vermes (1987); Kobila (1993); Abel

and Eberly (1997); Øksendal (2000); Scheinkman and Zariphopoulou (2001); Merhi and

Zervos (2007); Alvarez (2006). Also note that our method can be applied to more general

(diﬀusion) processes for the price dynamics, other than the geometric Brownian motion

assumed for explicitness in this paper. Finally, the construction between the functional form

of the boundaries and the payoﬀ function itself is also novel, as the value function and the

boundaries may be neither smooth nor strictly monotonic as in the existing literature.

The most relevant and recent work to this paper is Alvarez (2006), which provides a

great deal of economic insight into the problem. However, Alvarez (2006) only handles

payoﬀ functions satisfying the Inada conditions. In contrast, our solution is independent of

the regularity of the payoﬀ and value functions.

Outline. The control problem is formally stated, with its value function and optimal p olicy

described in Section 2; details of the derivation are in Section 3. The main result of this

paper regarding the regularity of the value function is in Section 4. Examples are provided

in Section 5, including cases for which the value function is not diﬀerentiable, the optimal

controlled process not continuous, the boundaries of the action regions not smooth, and the

interior of the continuation region not simply connected.

2 Mathematical Problem and Solution

2.1 Problem

Let (Ω, F, F, P) be a ﬁltered probability space and assume a given bounded interval [a, b] ⊂

(−∞, ∞). Consider the following problem:

Problem A.

V

H

(x, y) := sup

(ξ

+

,ξ

−

)∈A

00

y

J

H

(x, y; ξ

+

, ξ

−

),

3

##### Citations

More filters

••

TL;DR: A two-sided singular control problem in a general linear diffusion setting is studied and a set of conditions under which an optimal control exists uniquely and is of singular control type is provided.

Abstract: We study a two-sided singular control problem in a general linear diffusion setting and provide a set of conditions under which an optimal control exists uniquely and is of singular control type. Moreover, under these conditions the associated value function can be written in a quasi-explicit form. Furthermore, we investigate comparative static properties of the solution with respect to the volatility and control parameters. Lastly we illustrate the results with two explicit examples.

34 citations

••

TL;DR: In this paper, a reversible investment problem is studied where a social planner aims to control its capacity production in order to fit optimally the random demand of a good. But the resulting optimization problem leads to a degenerate two-dimensional bounded variation singular stochastic control problem for which explicit solution is not available in general and the standard verification approach cannot be applied a priori.

Abstract: This paper studies a reversible investment problem where a social planner aims to control its capacity production in order to fit optimally the random demand of a good. Our model allows for general diffusion dynamics on the demand as well as general cost functional. The resulting optimization problem leads to a degenerate two-dimensional bounded variation singular stochastic control problem, for which explicit solution is not available in general and the standard verification approach cannot be applied a priori. We use a direct viscosity solutions approach for deriving some features of the optimal free boundary function and for displaying the structure of the solution. In the quadratic cost case, we are able to prove a smooth fit $C^2$ property, which gives rise to a full characterization of the optimal boundaries and value function.

33 citations

••

TL;DR: In this article, the authors consider an irreversible capacity expansion model in which additional investment has a strictly negative effect on the value of an underlying stochastic economic indicator, and the associated optimisation problem takes the form of a singular control problem that admits an explicit solution.

Abstract: We consider an irreversible capacity expansion model in which additional investment has a strictly negative effect on the value of an underlying stochastic economic indicator. The associated optimisation problem takes the form of a singular stochastic control problem that admits an explicit solution. A special characteristic of this stochastic control problem is that changes of the state process due to control action depend on the state process itself in a proportional way.

19 citations

••

TL;DR: This work solves the genuinely two-dimensional stochastic control problem by constructing an explicit solution to an appropriate Hamilton--Jacobi--Bellman equation and by fully characterizing an optimal investment strategy.

Abstract: We consider the problem of determining in a dynamical way the optimal capacity level of an investment project that operates within a random economic environment. In particular, we consider an investment project that yields payoff at a rate that depends on its installed capacity level and on a random economic indicator such as the price of the project's output commodity. We model this economic indicator by means of a general one-dimensional ergodic diffusion. At any time, the project's capacity level can be increased or decreased at given proportional costs. The aim is to maximize an ergodic performance criterion that reflects the long-term average payoff resulting from the project's management. We solve this genuinely two-dimensional stochastic control problem by constructing an explicit solution to an appropriate Hamilton--Jacobi--Bellman equation and by fully characterizing an optimal investment strategy.

17 citations

••

TL;DR: In this article, the authors consider an investment project that produces a single commodity and determine a capacity expansion strategy that maximizes the ergodic or long-term average payoff resulting from the project's management.

Abstract: We consider an investment project that produces a single commodity. The project’s operation yields payoff at a rate that depends on the project’s installed capacity level and on an underlying economic indicator such as the output commodity’s price or demand, which we model by an ergodic, one-dimensional Itˆo diffusion. The project’s capacity level can be increased dynamically over time. The objective is to determine a capacity expansion strategy that maximizes the ergodic or long-term average payoff resulting from the project’s management. We prove that it is optimal to increase the project’s capacity level to a certain value and then take no further actions. The optimal capacity level depends on both the long-term average and the volatility of the underlying diffusion.

17 citations

##### References

More filters

•

01 Jan 1994

TL;DR: In this article, Dixit and Pindyck provide the first detailed exposition of a new theoretical approach to the capital investment decisions of firms, stressing the irreversibility of most investment decisions, and the ongoing uncertainty of the economic environment in which these decisions are made.

Abstract: How should firms decide whether and when to invest in new capital equipment, additions to their workforce, or the development of new products? Why have traditional economic models of investment failed to explain the behavior of investment spending in the United States and other countries? In this book, Avinash Dixit and Robert Pindyck provide the first detailed exposition of a new theoretical approach to the capital investment decisions of firms, stressing the irreversibility of most investment decisions, and the ongoing uncertainty of the economic environment in which these decisions are made. In so doing, they answer important questions about investment decisions and the behavior of investment spending.This new approach to investment recognizes the option value of waiting for better (but never complete) information. It exploits an analogy with the theory of options in financial markets, which permits a much richer dynamic framework than was possible with the traditional theory of investment. The authors present the new theory in a clear and systematic way, and consolidate, synthesize, and extend the various strands of research that have come out of the theory. Their book shows the importance of the theory for understanding investment behavior of firms; develops the implications of this theory for industry dynamics and for government policy concerning investment; and shows how the theory can be applied to specific industries and to a wide variety of business problems.

10,879 citations

•

01 Jan 1990

TL;DR: In this article, the authors propose a method for general stochastic integration and local times, which they call Stochastic Differential Equations (SDEs), and expand the expansion of Filtrations.

Abstract: I Preliminaries.- II Semimartingales and Stochastic Integrals.- III Semimartingales and Decomposable Processes.- IV General Stochastic Integration and Local Times.- V Stochastic Differential Equations.- VI Expansion of Filtrations.- References.

5,254 citations

•

Duke University

^{1}TL;DR: In this paper, a comprehensive introduction to probability theory covering laws of large numbers, central limit theorem, random walks, martingales, Markov chains, ergodic theorems, and Brownian motion is presented.

Abstract: This book is an introduction to probability theory covering laws of large numbers, central limit theorems, random walks, martingales, Markov chains, ergodic theorems, and Brownian motion. It is a comprehensive treatment concentrating on the results that are the most useful for applications. Its philosophy is that the best way to learn probability is to see it in action, so there are 200 examples and 450 problems.

5,168 citations

••

TL;DR: The optimality of a particular control limit policy is proved directly, with heavy reliance on the change of variable formula for semimartingales, to minimize the expected discounted sum of holding costs and control costs over an infinite planning horizon.

Abstract: A controller continuously monitors a storage system, such as an inventory or bank account, whose content Z = {Zt,t≥0} fluctuates as a (μ, σ2) Brownian motion in the absence of control. Holding costs are incurred continuously at rate h(Zt). At any time, the controller may instantaneously increase the content of the system, incurring a proportional cost of r times the size of the increase, or decrease the content at a cost of l times the size of the decrease. We consider the case where h is convex on a finite interval [α, β] and h = ∞ outside this interval. The objective is to minimize the expected discounted sum of holding costs and control costs over an infinite planning horizon. It is shown that there exists an optimal control limit policy, characterized by two parameters a and b (α ≤ a < b ≤ β). Roughly speaking, this policy exerts the minimum amounts of control sufficient to keep Zt ∈ [a, b] for all t ≥ 0. Put another way, the optimal control limit policy imposes on Z a lower reflecting barrier at a an...

273 citations

••

TL;DR: In this article, the problem of finding the optimal sequence of starting and stopping times of a multi-activity production process, given the costs of opening, running, and closing the activities and assuming that the economic system is a stochastic process, is formulated as an extended impulse control problem and solved using stochochastic calculus.

Abstract: This paper considers the problem of finding the optimal sequence of opening (starting) and closing (stopping) times of a multi- activity production process, given the costs of opening, running, and closing the activities and assuming that the state of the economic system is a stochastic process. The problem is formulated as an extended impulse control problem and solved using stochastic calculus. As an application, the optimal starting and stopping strategy are explicitly found for a resource extraction when the price of the resource is following a geometric Brownian motion.

227 citations