scispace - formally typeset
Search or ask a question
Author

L.E. Clarke

Bio: L.E. Clarke is an academic researcher. The author has contributed to research in topics: Measure (physics). The author has an hindex of 1, co-authored 1 publications receiving 134 citations.

Papers
More filters

Cited by
More filters
Proceedings Article
17 Jul 2017
TL;DR: This paper argues for the fundamental importance of the value distribution: the distribution of the random return received by a reinforcement learning agent, and designs a new algorithm which applies Bellman's equation to the learning of approximate value distributions.
Abstract: In this paper we argue for the fundamental importance of the value distribution: the distribution of the random return received by a reinforcement learning agent. This is in contrast to the common approach to reinforcement learning which models the expectation of this return, or value. Although there is an established body of literature studying the value distribution, thus far it has always been used for a specific purpose such as implementing risk-aware behaviour. We begin with theoretical results in both the policy evaluation and control settings, exposing a significant distributional instability in the latter. We then use the distributional perspective to design a new algorithm which applies Bellman's equation to the learning of approximate value distributions. We evaluate our algorithm using the suite of games from the Arcade Learning Environment. We obtain both state-of-the-art results and anecdotal evidence demonstrating the importance of the value distribution in approximate reinforcement learning. Finally, we combine theoretical and empirical evidence to highlight the ways in which the value distribution impacts learning in the approximate setting.

708 citations

Journal ArticleDOI
TL;DR: This paper developed a model with asymmetrically informed agents and costly monitoring of loan contracts, where an equilibrium can exhibit credit rationing, and the aggregate quantity of loans and equilibrium interest rates respond differently depending on whether there is rationing in equilibrium.
Abstract: This paper develops a model with asymmetrically informed agents and costly monitoring of loan contracts, where an equilibrium can exhibit credit rationing. Borrowers are identical ex ante, but some receive loans and others do not. In contrast to existing credit rationing theories, rationing does not occur here due to inflexible prices, adverse selection or moral hazard. Optimizing behaviour produces a standard debt contract in equilibrium. The aggregate quantity of loans and equilibrium interest rates respond differently depending on whether there is rationing in equilibrium.

597 citations

Proceedings Article
01 Jan 2012
TL;DR: In this article, the authors consider differentially private algorithms for convex empirical risk minimization (ERM) problems, in which one aims to find solutions (e.g., regression parameters) with few nonzero coefficients.
Abstract: We consider differentially private algorithms for convex empirical risk minimization (ERM). Differential privacy (Dwork et al., 2006b) is a recently introduced notion of privacy which guarantees that an algorithm’s output does not depend on the data of any individual in the dataset. This is crucial in fields that handle sensitive data, such as genomics, collaborative filtering, and economics. Our motivation is the design of private algorithms for sparse learning problems, in which one aims to find solutions (e.g., regression parameters) with few non-zero coefficients. To this end: (a) We significantly extend the analysis of the “objective perturbation” algorithm of Chaudhuri et al. (2011) for convex ERM problems. We show that their method can be modified to use less noise (be more accurate), and to apply to problems with hard constraints and non-differentiable regularizers. We also give a tighter, data-dependent analysis of the additional error introduced by their method. A key tool in our analysis is a new nontrivial limit theorem for differential privacy which is of independent interest: if a sequence of differentially private algorithms converges, in a weak sense, then the limit algorithm is also differentially private. In particular, our methods give the best known algorithms for differentially private linear regression. These methods work in settings where the number of parametersp is less than the number of samplesn. (b) We give the first two private algorithms for sparse regression problems in high-dimensional settings, where p is much larger than n. We analyze their performance for linear regression: under standard assumptions on the data, our algorithms have vanishing empirical risk for n = poly(s; logp) when there exists a good regression vector with s nonzero coefficients. Our algorithms demonstrate that randomized algorithms for sparse regression problems can be both stable and accurate ‐ a combination which is impossible for deterministic algorithms.

236 citations

Book ChapterDOI
13 Oct 2003
TL;DR: In this article, learning-based anomaly detection systems build models of the expected behavior of applications by analyzing events that are generated during their normal operation, and subsequent events are analyzed to identify deviations, given the assumption that anomalies usually represent evidence of an attack.
Abstract: Learning-based anomaly detection systems build models of the expected behavior of applications by analyzing events that are generated during their normal operation. Once these models have been established, subsequent events are analyzed to identify deviations, given the assumption that anomalies usually represent evidence of an attack.

220 citations

Journal ArticleDOI
TL;DR: The CALM model, designed to deal with uncertainty affecting both assets and liabilities (in the form of scenario dependent payments or borrowing costs) is presented, which is based on the current version of MSLiP.
Abstract: Multistage stochastic programming - in contrast to stochastic control - has found wideapplication in the formulation and solution of financial problems characterized by a largenumber of state variables and a generally low number of possible decision stages. Theliterature on the use of multistage recourse modelling to formalize complex portfolio optimizationproblems dates back to the early seventies, when the technique was first adopted tosolve a fixed income security portfolio problem. We present here the CALM model, whichhas been designed to deal with uncertainty affecting both assets (in either the portfolio orthe market) and liabilities (in the form of scenario dependent payments or borrowing costs).We consider as an instance a pension fund problem in which portfolio rebalancing is allowedover a long-term horizon at discrete time points and where liabilities refer to five differentclasses of pension contracts. The portfolio manager, given an initial wealth, seeks the maximizationof terminal wealth at the horizon, with investment returns modelled as discretestate random vectors. Decision vectors represent possible investments in the market andholding or selling assets in the portfolio, as well as borrowing decisions from a credit lineor deposits with a bank. Computational results are presented for a set of 10-stage portfolioproblems using different solution methods and libraries (OSL, CPLEX, OB1). The portfolioproblem, with an underlying vector data process which allows up to 2688 realizations at the10-year horizon, is solved on an IBM RS6000y590 for a set of twenty-four large-scale testproblems using the simplex and barrier methods provided by CPLEX (the latter for eitherlinear or quadratic objective), the predictorycorrector interior point method provided in OB1,the simplex method of OSL, the MSLiP-OSL code instantiating nested Benders decompositionwith subproblem solution using OSL simplex, and the current version of MSLiP.

177 citations