Under-constrained execution: making automatic code destruction easy and scalable

TLDR: Software testing is well-recognized as a crucial part of the modern software development process, however, manual testing is labor intensive and often fails to produce impressive coverage results.

Abstract: Software testing is well-recognized as a crucial part of the modern software development process. However, manual testing is labor intensive and often fails to produce impressive coverage results. Random testing is easily applied but gets poor coverage on complex code. Recent work has attacked these problems using symbolic execution to automatically generate high-coverage test inputs [3, 6, 4, 8, 5, 2]. At a high-level these tools use variations on the following idea. Instead of running code on manually or randomly constructed input, they run it on symbolic input initially allowed to be “anything.” They substitute program variables with symbolic values and replaces concrete program operations with ones that manipulate symbolic values. When program execution branches based on a symbolic value the system (conceptually) follows both branches at once, maintaining a set of constraints called the path condition which must hold on execution of that path. When a path terminates or hits a bug, a test case can be generated by solving the current path condition to find concrete values. Assuming deterministic code, feeding this concrete input to an uninstrumented version of the checked code will cause it to follow the same path and hit the same bug. However, these tools (and all dynamic tools) assume you can run the code you want to check in the first place. In the easiest case, testing just runs an entire application. This requires no special work: just compile the program and execute it. However, the exponential number of code paths in a