Showing papers on "Bounded function published in 2001"

Convex measures of risk and trading constraints

Abstract: We introduce the notion of a convex measure of risk, an extension of the concept of a coherent risk measure defined in Artzner et aL (1999), and we prove a corresponding extension of the representation theorem in terms of probability measures on the underlying space of scenarios. As a case study, we consider convex measures of risk defined in terms of a robust not ion of bounded shortfall risk. In the context of a financial market model, it turns out that the representation theorem is closely related to the superhedging duality under convex constraints.

Bounded Model Checking Using Satisfiability Solving

Abstract: The phrase model checking refers to algorithms for exploring the state space of a transition system to determine if it obeys a specification of its intended behavior. These algorithms can perform exhaustive verification in a highly automatic manner, and, thus, have attracted much interest in industry. Model checking programs are now being commercially marketed. However, model checking has been held back by the state explosion problem, which is the problem that the number of states in a system grows exponentially in the number of system components. Much research has been devoted to ameliorating this problem. In this tutorial, we first give a brief overview of the history of model checking to date, and then focus on recent techniques that combine model checking with satisfiability solving. These techniques, known as bounded model checking, do a very fast exploration of the state space, and for some types of problems seem to offer large performance improvements over previous approaches. We review experiments with bounded model checking on both public domain and industrial designs, and propose a methodology for applying the technique in industry for invariance checking. We then summarize the pros and cons of this new technology and discuss future research efforts to extend its capabilities.

Stabilization of stochastic nonlinear systems driven by noise of unknown covariance

Abstract: This paper poses and solves a new problem of stochastic (nonlinear) disturbance attenuation where the task is to make the system solution bounded by a monotone function of the supremum of the covariance of the noise. This is a natural stochastic counterpart of the problem of input-to-state stabilization in the sense of Sontag (1989). Our development starts with a set of new global stochastic Lyapunov theorems. For an exemplary class of stochastic strict-feedback systems with vanishing nonlinearities, where the equilibrium is preserved in the presence of noise, we develop an adaptive stabilization scheme (based on tuning functions) that requires no a priori knowledge of a bound on the covariance. Next, we introduce a control Lyapunov function formula for stochastic disturbance attenuation. Finally, we address optimality and solve a differential game problem with the control and the noise covariance as opposing players; for strict-feedback systems the resulting Isaacs equation has a closed-form solution.

Regression methods for pricing complex American-style options

Abstract: We introduce and analyze a simulation-based approximate dynamic programming method for pricing complex American-style options, with a possibly high-dimensional underlying state space. We work within a finitely parameterized family of approximate value functions, and introduce a variant of value iteration, adapted to this parametric setting. We also introduce a related method which uses a single (parameterized) value function, which is a function of the time-state pair, as opposed to using a separate (independently parameterized) value function for each time. Our methods involve the evaluation of value functions at a finite set, consisting of "representative" elements of the state space. We show that with an arbitrary choice of this set, the approximation error can grow exponentially with the time horizon (time to expiration). On the other hand, if representative states are chosen by simulating the state process using the underlying risk-neutral probability distribution, then the approximation error remains bounded.

A combinatorial strongly polynomial algorithm for minimizing submodular functions

Abstract: This paper presents a combinatorial polynomial-time algorithm for minimizing submodular functions, answering an open question posed in 1981 by Grotschel, Lovasz, and Schrijver. The algorithm employs a scaling scheme that uses a flow in the complete directed graph on the underlying set with each arc capacity equal to the scaled parameter. The resulting algorithm runs in time bounded by a polynomial in the size of the underlying set and the length of the largest absolute function value. The paper also presents a strongly polynomial version in which the number of steps is bounded by a polynomial in the size of the underlying set, independent of the function values.

Recurrence of Distributional Limits of Finite Planar Graphs

Abstract: Suppose that $G_j$ is a sequence of finite connected planar graphs, and in each $G_j$ a special vertex, called the root, is chosen randomly-uniformly. We introduce the notion of a distributional limit $G$ of such graphs. Assume that the vertex degrees of the vertices in $G_j$ are bounded, and the bound does not depend on $j$. Then after passing to a subsequence, the limit exists, and is a random rooted graph $G$. We prove that with probability one $G$ is recurrent. The proof involves the Circle Packing Theorem. The motivation for this work comes from the theory of random spherical triangulations.

Best approximation in inner product spaces

Abstract: Inner Product Spaces.- Best Approximation.- Existence and Uniqueness of Best Approximations.- Characterization of Best Approximations.- The Metric Projection.- Bounded Linear Functionals and Best Approximation from Hyperplanes and Half-spaces.- Error of Approximation.- Generalized Solutions of Linear Equations.- The Method of Alternating Projections.- Constrained Interpolation from a Convex Set.- Interpolation and Approximation.- Convexity of Chebyshev Sets.

Off-Policy Temporal Difference Learning with Function Approximation

Abstract: We introduce the first algorithm for off-policy temporal-difference learning that is stable with linear function approximation. Off-policy learning is of interest because it forms the basis for popular reinforcement learning methods such as Q-learning, which has been known to diverge with linear function approximation, and because it is critical to the practical utility of multi-scale, multi-goal, learning frameworks such as options, HAMs, and MAXQ. Our new algorithm combines TD(λ) over state–action pairs with importance sampling ideas from our previous work. We prove that, given training under any -soft policy, the algorithm converges w.p.1 to a close approximation (as in Tsitsiklis and Van Roy, 1997; Tadic, 2001) to the action-value function for an arbitrary target policy. Variations of the algorithm designed to reduce variance introduce additional bias but are also guaranteed convergent. We also illustrate our method empirically on a small policy evaluation problem. Our current results are limited to episodic tasks with episodes of bounded length. 1Although Q-learning remains the most popular of all reinforcement learning algorithms, it has been known since about 1996 that it is unsound with linear function approximation (see Gordon, 1995; Bertsekas and Tsitsiklis, 1996). The most telling counterexample, due to Baird (1995) is a seven-state Markov decision process with linearly independent feature vectors, for which an exact solution exists, yet This is a re-typeset version of an article published in the Proceedings of the 18th International Conference on Machine Learning (2001). It differs from the original in line and page breaks, is crisper for electronic viewing, and has this funny footnote, but otherwise it is identical to the published article. for which the approximate values found by Q-learning diverge to infinity. This problem prompted the development of residual gradient methods (Baird, 1995), which are stable but much slower than Q-learning, and fitted value iteration (Gordon, 1995, 1999), which is also stable but limited to restricted, weaker-than-linear function approximators. Of course, Q-learning has been used with linear function approximation since its invention (Watkins, 1989), often with good results, but the soundness of this approach is no longer an open question. There exist non-pathological Markov decision processes for which it diverges; it is absolutely unsound in this sense. A sensible response is to turn to some of the other reinforcement learning methods, such as Sarsa, that are also efficient and for which soundness remains a possibility. An important distinction here is between methods that must follow the policy they are learning about, called on-policy methods, and those that can learn from behavior generated by a different policy, called off-policy methods. Q-learning is an off-policy method in that it learns the optimal policy even when actions are selected according to a more exploratory or even random policy. Q-learning requires only that all actions be tried in all states, whereas on-policy methods like Sarsa require that they be selected with specific probabilities. Although the off-policy capability of Q-learning is appealing, it is also the source of at least part of its instability problems. For example, in one version of Baird’s counterexample, the TD(λ) algorithm, which underlies both Qlearning and Sarsa, is applied with linear function approximation to learn the action-value function Q for a given policy π. Operating in an on-policy mode, updating state– action pairs according to the same distribution they would be experienced under π, this method is stable and convergent near the best possible solution (Tsitsiklis and Van Roy, 1997; Tadic, 2001). However, if state-action pairs are updated according to a different distribution, say that generated by following the greedy policy, then the estimated values again diverge to infinity. This and related counterexamples suggest that at least some of the reason for the instability of Q-learning is that it is an off-policy method; they also make it clear that this part of the problem can be studied in a purely policy-evaluation context. Despite these problems, there remains substantial reason for interest in off-policy learning methods. Several researchers have argued for an ambitious extension of reinforcement learning ideas into modular, multi-scale, and hierarchical architectures (Sutton, Precup & Singh, 1999; Parr, 1998; Parr & Russell, 1998; Dietterich, 2000). These architectures rely on off-policy learning to learn about multiple subgoals and multiple ways of behaving from the singular stream of experience. For these approaches to be feasible, some efficient way of combining off-policy learning and function approximation must be found. Because the problems with current off-policy methods become apparent in a policy evaluation setting, it is there that we focus in this paper. In previous work we considered multi-step off-policy policy evaluation in the tabular case. In this paper we introduce the first off-policy policy evaluation method consistent with linear function approximation. Our mathematical development focuses on the episodic case, and in fact on a single episode. Given a starting state and action, we show that the expected offpolicy update under our algorithm is the same as the expected on-policy update under conventional TD(λ). This, together with some variance conditions, allows us to prove convergence and bounds on the error in the asymptotic approximation identical to those obtained by Tsitsiklis and Van Roy (1997; Bertsekas and Tsitsiklis, 1996). 1. Notation and Main Result We consider the standard episodic reinforcement learning framework (see, e.g., Sutton & Barto, 1998) in which a learning agent interacts with a Markov decision process (MDP). Our notation focuses on a single episode of T time steps, s0, a0, r1, s1, a1, r2, . . . , rT , sT , with states st ∈ S, actions at ∈ A, and rewards rt ∈ <. We take the initial state and action, s0 and a0, to be given arbitrarily. Given a state and action, st and at, the next reward, rt+1, is a random variable with mean rt st and the next state, st+1, is chosen with probabilities pt stst+1 . The final state is a special terminal state that may not occur on any preceding time step. Given a state, st, 0 < t < T , the action at is selected according to probability π(st, at) or b(st, at) depending on whether policy π or policy b is in force. We always use π to denote the target policy, the policy that we are learning about. In the on-policy case, π is also used to generate the actions of the episode. In the off-policy case, the actions are instead generated by b, which we call the behavior policy. In either case, we seek an approximation to the action-value function Q : S ×A 7→ < for the target policy π: Q(s, a) = Eπ { rt+1 + · · ·+ γrT | st = s, at = a } , where 0 ≤ γ ≤ 1 is a discount-rate parameter. We consider approximations that are linear in a set of feature vectors {φsa}, s ∈ S, a ∈ A: Q(s, a) ≈ θφsa = n ∑

Constraint solving for bounded-process cryptographic protocol analysis

Abstract: The reachability problem for cryptographic protocols with non-atomic keys can be solved via a simple constraint satisfaction procedure.

Local limit theorems for partial sums of stationary sequences generated by gibbs–markov maps

Abstract: We introduce Gibbs–Markov maps T as maps with a (possibly countable) Markov partition and a certain type of bounded distortion property, and investigate its Frobenius–Perron operator P acting on (locally) Lipschitz continuous functions ϕ. If such a function ϕ belongs to the domain of attraction of a stable law of order in (0,2), we derive the expansion of the eigenvalue function t↦λ(t) of the characteristic function operators Ptf=Pfexp[i (perturbations of P) around 0. From this representation local and distributional limit theorems for partial sums ϕ+…+ϕ◦ Tn are easily obtained, provided ϕ is aperiodic. Applications to recurrence properties of group extensions are also given.

Robust filtering for discrete-time systems with bounded noise and parametric uncertainty

Abstract: This note presents a new approach to finite-horizon guaranteed state prediction for discrete-time systems affected by bounded noise and unknown-but-bounded parameter uncertainty. Our framework handles possibly nonlinear dependence of the state-space matrices on the uncertain parameters. The main result is that a minimal confidence ellipsoid for the state, consistent with the measured output and the uncertainty description, may be recursively computed in polynomial time, using interior-point methods for convex optimization. With n states, l uncertain parameters appearing linearly in the state-space matrices, with rank-one matrix coefficients, the worst-case complexity grows as O(l(n + l)/sup 3.5/) With unstructured uncertainty in all system matrices, the worst-case complexity reduces to O(n/sup 3.5/).

Uncertainty in mechanics problems-interval-based approach

Abstract: This paper presents a nontraditional uncertainty treatment for mechanics problems. Uncertainties are introduced as bounded possible values (intervals). Interval finite-element methods, developed by the authors, are used in the present formulation. To account for different types of uncertainties in linear static problems an interval linear system of equations is developed. A guaranteed enclosure for the solution of interval linear system of equations is achievable and usually is not sharp and very conservative; however, an exact enclosure is not known to be obtained in the general case of such systems. In this work, a very sharp enclosure for the solution set, due to loading, material and geometric uncertainty in solid mechanics problems, is obtained. The new formulation is based on an element-by-element technique. Element matrices are formulated, based on the physics of materials, and the Lagrange multiplier method is applied to impose the necessary constraints for compatibility and equilibrium. Most sour...

Deciding first-order properties of locally tree-decomposable structures

Abstract: We introduce the concept of a class of graphs, or more generally, relational structures, being locally tree-decomposable. There are numerous examples of locally tree-decomposable classes, among them the class of planar graphs and all classes of bounded valence or of bounded tree-width. We also consider a slightly more general concept of a class of structures having bounded local tree-width.We show that for each property φ of structures that is definable in first-order logic and for each locally tree-decomposable class C of structures, there is a linear time algorithm deciding whether a given structure A ∈ C has property φ. For classes C of bounded local tree-width, we show that for every k ≥ 1 there is an algorithm solving the same problem in time O(n1+(1/k)) (where n is the cardinality of the input structure).

Locality and exponential error reduction in numerical lattice gauge theory

Abstract: In non-abelian gauge theories without matterelds, expectation values oflargeWilsonloopsandloopcorrelationfunctionsaredicult tocomputethrough numerical simulation, because the signal-to-noise ratio is very rapidly decaying for increasing loop sizes. Using a multilevel scheme that exploits the locality of the theory, we show that the statistical errors in such calculations can be exponentially reduced. We explicitly demonstrate this in the SU(3) theory, for the case of the Polyakovloopcorrelationfunction,wheretheeciencyofthesimulationisimproved by many orders of magnitude when the area bounded by the loops exceeds 1fm 2 .

On the fixed parameter complexity of graph enumeration problems definable in monadic second-order logic

Abstract: We discuss the parametrized complexity of counting and evaluation problems on graphs where the range of counting is denable in monadic second-order logic (MSOL). We show that for bounded tree-width these problems are solvable in polynomial time. The same holds for bounded clique width in the cases, where the decomposition, which establishes the bound on the clique-width, can be computed in polynomial time and for problems expressible by monadic second-order formulas without edge set quantication. Such quantications are allowed in the case of graphs with bounded tree-width. As applications we discuss in detail how this aects the parametrized complexity of the permanent and the hamiltonian of a matrix, and more generally, various generating functions of MSOL denable graph properties. Finally, our results are also applicable to SAT and ]SAT. ? 2001 Elsevier Science B.V. All rights reserved.

Weighted Extended B-Spline Approximation of Dirichlet Problems

Abstract: We describe a new finite element method which uses weighted extended B-splines on a regular grid as basis functions for solving Dirichlet problems on bounded domains in arbitrary dimensions. This web-method does not require any grid generation and can be implemented very efficiently. It yields smooth, high order accurate approximations with relatively low dimensional subspaces.

BMO, H^1, and Calderon-Zygmund operators for non doubling measures

Abstract: Given a Radon measure $\mu$ on ${\mathbb R}^d$ , which may be non doubling, we introduce a space of type BMO with respect to this measure. It is shown that many properties which hold for the classical space $BMO(\mu)$ when $\mu$ is a doubling measure remain valid for the space of type BMO introduced in this paper, without assuming $\mu$ doubling. For instance, Calderon-Zygmund operators which are bounded on $L^2(\mu)$ are also bounded from $L^\infty(\mu)$ into the new BMO space. Moreover, this space also satisfies a John-Nirenberg inequality, and its predual is an atomic space $H^1$ . Using a sharp maximal operator it is shown that operators which are bounded from $L^\infty(\mu)$ into the new BMO space and from its predual $H^1$ into $L^1(\mu)$ must be bounded on $L^p(\mu)$ , $1< p< infty$ . From this result one can obtain a new proof of the T(1) theorem for the Cauchy transform for non doubling measures. Finally, it is proved that commutators of Calderon-Zygmund operators bounded on $L^2(\mu)$ with functions of the new BMO are bounded on $L^p(\mu), 1< p < \infty$ .

Robust adaptive observers for nonlinear systems with bounded disturbances

Abstract: Since existing adaptive observers for nonlinear systems may generate unbounded parameter estimates in the presence of bounded disturbances, robust adaptive observers are presented which prevent parameter estimate drift. In addition the input-to-state stability of the error dynamics with respect to disturbances and parameter time-derivatives is guaranteed by generalizing a persistency of excitation result. Asymptotic convergence of state estimation errors is still achieved for systems in adaptive observer form when disturbances are not present, by a suitable extension of Barbalat's lemma.

Hybrid Logics: Characterization, Interpolation and Complexity

Abstract: Hybrid languages are expansions of propositional modal languages which can refer to (or even quantify over) worlds. The use of strong hybrid languages dates back to the work of Arthur Prior in the 1960s, but recent work has focussed on a more constrained system in which variables can only be bound to the current state. We show in detail that the constrained system is modally natural. We begin by studying its expressivity, and provide model theoretic characterizations (via a restricted notion of Ehrenfeucht-Fraisse game, and an enriched notion of bisimulation) and a syntactic characterization (in terms of bounded formulas). The key result to emerge is that the constrained system corresponds to the fragment of first-order logic which is invariant for generated submodels. We then show that the system enjoys (strong) interpolation, provide counterexamples for its finite variable fragments, and show that weak interpolation holds for an important subsystem called H(@). Finally, we provide complexity results for H(@) and other fragments and variants, and sharpen known undecidability results for the full fragment

A generalized entropy criterion for Nevanlinna-Pick interpolation with degree constraint

Abstract: We present a generalized entropy criterion for solving the rational Nevanlinna-Pick problem for n+1 interpolating conditions and the degree of interpolants bounded by n. The primal problem of maximizing this entropy gain has a very well-behaved dual problem. This dual is a convex optimization problem in a finite-dimensional space and gives rise to an algorithm for finding all interpolants which are positive real and rational of degree at most n. The criterion requires a selection of a monic Schur polynomial of degree n. It follows that this class of monic polynomials completely parameterizes all such rational interpolants, and it therefore provides a set of design parameters for specifying such interpolants. The algorithm is implemented in a state-space form and applied to several illustrative problems in systems and control, namely sensitivity minimization, maximal power transfer and spectral estimation.

On a long-standing conjecture of E. De Giorgi: symmetry in 3D for general nonlinearities and a local minimality property

Abstract: This paper studies a conjecture made by De Giorgi in 1978 concerning the one-dimensional character (or symmetry) of bounded, monotone in one direction, solutions of semilinear elliptic equations Δu=F′(u) in all of R n . We extend to all nonlinearities F∈C 2 the symmetry result in dimension n=3 previously established by the second and third authors for a class of nonlinearities F which included the model case F′(u)=u 3−u. The extension of the present paper is based on new energy estimates which follow from a local minimality property of u. In addition, we prove a symmetry result for semilinear equations in the halfspace R + 4. Finally, we establish that an asymptotic version of the conjecture of De Giorgi is true when n≤8, namely that the level sets of u are flat at infinity.

On the Tractable Counting of Theory Models and its Application to Truth Maintenance and Belief Revision

Abstract: We address in this paper the problem of counting the models of a propositional theory under incremental changes to its literals. Specifcally, we show that if a propositional theory Δ is in a special form that we call smooth, deterministic, decomposable negation normal form (sd-DNNF), then for any consistent set of literals S, we can simultaneously count (in time linear in the size of Δ) the models of Δ ∪ S and the models of every theory Δ ∪ T where T results from adding, removing or flipping a literal in S. We present two results relating to the time and space complexity of compiling propositional theories into sd-DNNF. First, we show that if a conjunctive normal form (CNF) has a bounded treewidth, then it can be compiled into an sd-DNNF in time and space which are linear in its size. Second, we show that sd-DNNF is a strictly more space efficient representation than Free Binary Decision Diagrams (FBDDs). Finally, we discuss some applications of the counting results to truth maintenance systems, belief re...

Gagliardo-Nirenberg, composition and products in fractional Sobolev spaces

Abstract: Our main result is that, when $f$ is smooth and has bounded derivatives, and when $u$ belongs to the spaces $W^{s,p}$ and $W^{1,sp}$, the map $f(u)$ is in $W^{s,p}$.

Well Posedness for Pressureless Flow

Abstract: We study the uniqueness problem for pressureless gases. Previous results on this topic are only known for the case when the initial data is assumed to be a bounded measurable function. This assumption is unnatural because the solution is in general a Radon measure. In this paper, the uniqueness of the weak solution is proved for the case when the initial data is a Radon measure. We show that, besides the Oleinik entropy condition, it is also important to require the energy to be weakly continuous initially. Our uniqueness result is obtained in the same functional space as the existence theorem.