Showing papers by "James A. Jones published in 2012"

History slicing: assisting code-evolution tasks

Abstract: Many software-engineering tasks require developers to understand the history and evolution of source code. However, today's software-development techniques and tools are not well suited for the easy and efficient procurement of such information. In this paper, we present an approach called history slicing that can automatically identify a minimal number of code modifications, across any number of revisions, for any arbitrary segment of source code at fine granularity. We also present our implementation of history slicing, Chronos, that includes a novel visualization of the entire evolution for the code of interest. We provide two experiments: one experiment automatically computes 16,000 history slices to determine the benefit brought by various levels of automation, and another experiment that assesses the practical implications of history slicing for actual developers using the technique for actual software-maintenance tasks that involve code evolution. The experiments show that history slicing offered drastic improvements over the conventional techniques in three ways: (1) the amount of information needed to be examined and traced by developers was reduced by up to three orders of magnitude; (2) the correctness of developers attempting to solve software-maintenance tasks was more than doubled; and (3) the time to completion of these software-maintenance tasks was almost halved.

WhoseFault: automatic developer-to-fault assignment through fault localization

Abstract: This paper describes a new technique, which automatically selects the most appropriate developers for fixing the fault represented by a failing test case, and provides a diagnosis of where to look for the fault. This technique works by incorporating three key components: (1) fault localization to inform locations whose execution correlate with failure, (2) history mining to inform which developers edited each line of code and when, and (3) expertise assignment to map locations to developers. To our knowledge, the technique is the first to assign developers to execution failures, without the need for textual bug reports. We implement this technique in our tool, WhoseFault, and describe an experiment where we utilize a large, open-source project to determine the frequency in which our tool suggests an assignment to the actual developer who fixed the fault. Our results show that 81% of the time, WhoseFault produced the same developer that actually fixed the fault within the top three suggestions. We also show that our technique improved by a difference between 4% and 40% the results of a baseline technique. Finally, we explore the influence of each of the three components of our technique over its results, and compare our expertise algorithm against an existing expertise assessment technique and find that our algorithm provides greater accuracy, by up to 37%.

Concept-based failure clustering

Abstract: When attempting to determine the number and set of execution failures that are caused by particular faults, developers must perform an arduous task of investigating and diagnosing each individual failure. Researchers proposed failure-clustering techniques to automatically categorize failures, with the intention of isolating each culpable fault. The current techniques utilize dynamic control flow to characterize each failure to then cluster them. These existing techniques, however, are blind to the intent or purpose of each execution, other than what can be inferred by the control-flow profile. We hypothesize that semantically rich execution information can aid clustering effectiveness by categorizing failures according to which functionality they exhibit in the software. This paper presents a novel clustering method that utilizes latent-semantic-analysis techniques to categorize each failure by the semantic concepts that are expressed in the executed source code. We present an experiment comparing this new technique to traditional control-flow-based clustering. The results of the experiment showed that the semantic-concept clustering was more precise in the number of clusters produced than the traditional approach, without sacrificing cluster accuracy.

Software Behavior and Failure Clustering: An Empirical Study of Fault Causality

Abstract: To cluster executions that exhibit faulty behavior by the faults that cause them, researchers have proposed using internal execution events, such as statement profiles, to (1) measure execution similarities, (2) categorize executions based on those similarity results, and (3) suggest the resulting categories as sets of executions exhibiting uniform fault behavior. However, due to a paucity of evidence correlating profiles and output behavior, researchers employ multiple simplifying assumptions in order to justify such approaches. In this paper we present an empirical study of profile correlation with output behavior, and we reexamine the suitability of such simplifying assumptions. We examine over 4 billion test-case outputs and execution profiles from multiple programs with over 9000 versions. Our data provides evidence that with current techniques many executions should be omitted from the clustering analysis to provide clusters that each represent a single fault. In addition, our data reveals the previously undocumented effects of multiple faults on failures, which has implications for techniques' ability (and inability) to properly cluster. Our results suggest directions for the improvement of future failure-clustering techniques that better account for software-fault behavior.

Weighted System Dependence Graph

Abstract: In this paper, we present a weighted, hybrid program-dependence model that represents the relevance of highly related, dependent code to assist developer comprehension of the program for multiple software-engineering tasks. Programmers often need to understand the dependencies among program elements, which may exist across multiple modules. Although such dependencies can be gathered from traditional models, such as slices, the scalability of these approaches is often prohibitive for direct, practical use. To address this scalability issue, as well as to assist developer comprehension, we introduce a program model that includes static dependencies as well as information about any number of executions, which inform the weight and relevance of the dependencies. Additionally, classes of executions can be differentiated in such a way as to support multiple software-engineering tasks. We evaluate this weighted, hybrid model for a task that involves exploring the structural context while debugging. The results demonstrate that the new model more effectively reveals relevant failure-correlated code than the static-only model, thus enabling a more scalable exploration or post hoc analysis.

Semantic fault diagnosis: automatic natural-language fault descriptions

Abstract: Before a fault can be fixed, it first must be understood. However, understanding why a system fails is often a difficult and time consuming process. While current automated-debugging techniques provide assistance in knowing where a fault is, developers are left unaided in understanding what a fault is, and why the system is failing. We present Semantic Fault Diagnosis (SFD), a technique that leverages lexicographic and dynamic information to automatically capture natural-language fault descriptors. SFD utilizes class names, method names, variable expressions, developer comments, and keywords from the source code to describe a fault. SFD can be used immediately after observing a failing execution and requires no input from developers or bug reports. In addition we present motivating examples and results from a SFD prototype to serve as a proof of concept.