We describe the system ProVE, an automated prover to verify (innermost) termination of term rewrite systems (TRSs). For this system, we have developed and implemented efficient algorithms based on classical simplification orders, dependency pairs, and the size-change principle. In particular, it contains many new improvements of the dependency pair approach that make automated termination proving more powerful and efficient. In ProVE, termination proofs can be performed with a user-friendly graphical interface and the system is currently among the most powerful termination provers available.

Automated termination proofs with AProVE

Instrumenting programs for performing runtime checking of properties, such as regular shapes, is a common and useful technique that helps programmers detect incorrect program behaviors. This is specially true in dynamic languages such as Prolog. However, such runtime checks inevitably introduce runtime overhead (in execution time, memory, energy, etc.). Several approaches have been proposed for reducing this overhead, such as eliminating the checks that can statically be proved to always succeed, and/or optimizing the way in which the (remaining) checks are performed. However, there are cases in which it is not possible to remove all checks statically (e.g., open libraries which must check their interfaces, complex properties, unknown code, etc.) and in which, even after optimizations, these remaining checks may still introduce an unacceptable level of overhead. It is thus important for programmers to be able to determine the additional cost due to the runtime checks and compare it to some notion of admissible cost. The common practice used for estimating runtime checking overhead is profiling, which is not exhaustive by nature. Instead, we propose a method that uses static analysis to estimate such overhead, with the advantage that the estimations are functions parameterized by input data sizes. Unlike profiling, this approach can provide guarantees for all possible execution traces, and allows assessing how the overhead grows as the size of the input grows. Our method also extends an existing assertion verification framework to express "admissible" overheads, and statically and automatically checks whether the instrumented program conforms with such specifications. Finally, we present an experimental evaluation of our approach that suggests that our method is feasible and promising.

/pdf/static-performance-guarantees-for-programs-with-runtime-guf51bv3si.pdf

Static Performance Guarantees for Programs with Runtime Checks

Instrumenting programs for performing run-time checking of properties, such as regular shapes, is
a common and useful technique that helps programmers detect incorrect program behaviors. This
is specially true in dynamic languages such as Prolog. However, such run-time checks inevitably
introduce run-time overhead (in execution time, memory, energy, etc.). Several approaches have
been proposed for reducing such overhead, such as eliminating the checks that can statically
be proved to always succeed, and/or optimizing the way in which the (remaining) checks are
performed. However, there are cases in which it is not possible to remove all checks statically (e.g.,
open libraries which must check their interfaces, complex properties, unknown code, etc.) and in
which, even after optimizations, these remaining checks still may introduce an unacceptable level of
overhead. It is thus important for programmers to be able to determine the additional cost due to
the run-time checks and compare it to some notion of admissible cost. The common practice used
for estimating run-time checking overhead is profiling, which is not exhaustive by nature. Instead,
we propose a method that uses static analysis to estimate such overhead, with the advantage that
the estimations are functions parameterized by input data sizes. Unlike profiling, this approach
can provide guarantees for all possible execution traces, and allows assessing how the overhead
grows as the size of the input grows. Our method also extends an existing assertion verification
framework to express “admissible” overheads, and statically and automatically checks whether
the instrumented program conforms with such specifications. Finally, we present an experimental
evaluation of our approach that suggests that our method is feasible and promising.

An Approach to Static Performance Guarantees for Programs with Run-time Checks.

Detectar comportamientos incorrectos en los programas es una parte importante en el ciclo de desarrollo de software. Es una tarea compleja y tediosa, especialmente en el contexto de los lenguajes dinamicos. Se han propuesto numerosas tecnicas que ayudan en el proceso, entre las cuales nos hemos centrado en el uso de construcciones a nivel de lenguaje para describir el comportamiento esperado del programa, y en las herramientas necesarias para comparar el comportamiento real del programa en contraposicion con el esperado, como, por ejemplo, analizadores/verificadores estaticos de codigo y entornos de verificacion en tiempo de ejecucion. En la practica, sin embargo, el alto coste durante la ejecucion hace que el uso de estas herramientas sea poco viable, especialmente para propiedades complejas. Esto reduce el interes en hacer comprobaciones en tiempo de ejecucion desde el punto de vista de los programadores y programadoras, quienes esporadicamente permitiran comprobaciones de condiciones muy sencillas pero tenderan a desactivarlas para propiedades complejas. Algunos trabajos optan por limitar la expresividad del lenguaje de aserciones para reducir este coste. Con esta motivacion, el objetivo de esta tesis es doble: - primero, pretendemos mejorar la expresividad del lenguaje de aserciones para reflejar todas las caracteristicas relacionadas con el lenguaje de programacion, incluyendo, por ejemplo, construcciones de orden superior, haciendolo de forma que el/la programador/ a pueda escribir especificaciones sin necesidad de aprender o programar para ello; - al mismo tiempo, nuestra meta es comprobar de forma eficiente dichas especificaciones, reduciendo el coste asociado en tiempo de ejecucion en la medida de lo posible y sin comprometer las garantias de seguridad que proporcionan dichas comprobaciones. Esta tesis presenta varias mejoras para la comprobacion de especificaciones en tiempo de ejecucion entre las que se encuentran: - un mecanismo discreto de memorizacion de resultados intermedios de comprobacion, de forma que pueden ser reutilizados en el proceso de comprobacion en lugar de recalcularlos; - un tecnica que combina comprobacion en tiempo de compilacion y en tiempo de ejecucion, que usa las propiedades de esta ultima como informacion adicional en tiempo de compilacion, lo que implica que mas propiedades se puedan comprobar estaticamente, aligerando el trabajo en tiempo de ejecucion; - y otra tecnica para mejorar la inferencia de estructuras durante el analisis estatico de programas, que aprovecha las reglas de visibilidad de terminos del entorno modular subyacente, lo que permite simplificar las comprobaciones de propiedades del programa consiguiendo un sobrecoste constante en casos relevantes. Finalmente, para atacar el problema de la expresividad limitada de los lenguajes de especificaciones, esta tesis se enfoca en el caso concreto de aportar especificaciones detalladas para rutinas de orden superior. Las tecnicas y herramientas estudiadas en esta tesis se presentan, por concrecion, en el entorno de comprobacion en tiempo de ejecucion Ciao. No obstante, los resultados son generales e independientes del sistema, y creemos que pueden trasladarse de forma sencilla a otros lenguajes de programacion declarativos. Ademas, dados los avances en verificacion en gran parte de los lenguajes de programacion, incluyendo los imperativos, mediante la traduccion a clausulas de Horn y probando propiedades a este nivel, y el hecho de que este enfoque esta totalmente soportado en el sistema Ciao, argumentamos que nuestros resultados se pueden adaptar facilmente a un espectro mucho mas amplio de lenguajes. ----------ABSTRACT---------- Detecting incorrect program behaviors is an important part of the software development life cycle. It is also a complex and tedious one, in which dynamic languages bring special challenges. A number of techniques have been proposed to aid in the process, among which we center our attention on the use of language-level constructs to describe expected program behavior, and of associated tools to compare actual program behavior against expectations, such as static code analyzers/ verifiers and run-time verification frameworks. In practice, however, the run-time overhead associated with these tools often remains impractically high, specially for non-trivial properties, or complex data structure tests. This reduces the attractiveness of run-time checking to programmers, who may allow sporadic checking of very simple conditions, but will tend to turn off run-time checking for more complex properties in favor of faster execution. Some approaches even opt for limiting the expressiveness of the assertion language in order to reduce the overhead. Our research objective in this thesis is twofold: - first, we aim to enhance the expressiveness of the assertion language to reflect all the features of the related programming language, including, e.g., higher-order constructs, and to do so in a way that allows the programmer to write precise program specifications while not imposing a learning or programming burden on them; - at the same time, our goal is to efficiently check specifications, mitigating the associated run-time overhead as much as possible without compromising the safety guarantees that the checks provide. With respect to checking specifications efficiently this dissertation presents several improvements for run-time specification checking, including: - a mechanism for unobtrusive caching of intermediate run-time checking results so that they can be re-used in the checking process instead of being re-evaluated, contributing to undesirable (and unnecessary) run-time overhead; - a technique of combining compile- and run-time checking in a way that uses the properties from the program specification as an additional information source during static specification checking, which results in more properties checked statically and fewer of them turned into run-time checks; - and another technique for improving term shape inference during static program analysis, exploiting term visibility rules of the underlying module system, which allows to simplify property checks in a program in a way that constant run-time overhead is achievable in relevant cases. Finally, to address the limited expressiveness of the specification languages, this dissertation targets the concrete case of providing detailed specifications for higher-order program routines. The techniques and tools discussed in this thesis are presented for concreteness in the context of the Ciao run-time checking framework. Nevertheless, these results are general and system-independent, and we believe they can be straightforwardly transferred to the contexts of other declarative languages. In addition, given the recent advances in verification of a wide class of programming languages, including imperative ones, by translation into Horn clauses and proving properties at this level, and the fact that this approach is fully supported in the Ciao system, we argue that our results can easily be adapted to a much broader spectrum of languages.

Improving Run-time Checking in Dynamic Programming Languages

Approximations during program analysis are a necessary evil, as they ensure essential properties, such as soundness and termination of the analysis, but they also imply not always producing useful results. Automatic techniques have been studied to prevent precision loss, typically at the expense of larger resource consumption. In both cases (i.e., when analysis produces inaccurate results and when resource consumption is too high), it is necessary to have some means for users to provide information to guide analysis and thus improve precision and/or performance. We present techniques for supporting within an abstract interpretation framework a rich set of assertions that can deal with multivariance/context-sensitivity, and can handle different run-time semantics for those assertions that cannot be discharged at compile time. We show how the proposed approach can be applied to both improving precision and accelerating analysis. We also provide some formal results on the effects of such assertions on the analysis results.

Nataliia Stulova

Papers

Some trade-offs in reducing the overhead of assertion run-time checks via static analysis