Domain-specific languages (DSLs) are languages tailored to a specific application domain. They offer substantial gains in expressiveness and ease of use compared with general-purpose programming languages in their domain of application. DSL development is hard, requiring both domain knowledge and language development expertise. Few people have both. Not surprisingly, the decision to develop a DSL is often postponed indefinitely, if considered at all, and most DSLs never get beyond the application library stage.Although many articles have been written on the development of particular DSLs, there is very limited literature on DSL development methodologies and many questions remain regarding when and how to develop a DSL. To aid the DSL developer, we identify patterns in the decision, analysis, design, and implementation phases of DSL development. Our patterns improve and extend earlier work on DSL design patterns. We also discuss domain analysis tools and language development systems that may help to speed up DSL development. Finally, we present a number of open problems.

/pdf/when-and-how-to-develop-domain-specific-languages-29nvxv3ngp.pdf

When and how to develop domain-specific languages

A code clone is a code portion in source files that is identical or similar to another. Since code clones are believed to reduce the maintainability of software, several code clone detection techniques and tools have been proposed. This paper proposes a new clone detection technique, which consists of the transformation of input source text and a token-by-token comparison. For its implementation with several useful optimization techniques, we have developed a tool, named CCFinder (Code Clone Finder), which extracts code clones in C, C++, Java, COBOL and other source files. In addition, metrics for the code clones have been developed. In order to evaluate the usefulness of CCFinder and metrics, we conducted several case studies where we applied the new tool to the source code of JDK, FreeBSD, NetBSD, Linux, and many other systems. As a result, CCFinder has effectively found clones and the metrics have been able to effectively identify the characteristics of the systems. In addition, we have compared the proposed technique with other clone detection techniques.

/pdf/ccfinder-a-multilinguistic-token-based-code-clone-detection-3uimzrhga0.pdf

CCFinder: a multilinguistic token-based code clone detection system for large scale source code

Existing research suggests that a considerable fraction (5-10%) of the source code of large scale computer programs is duplicate code ("clones"). Detection and removal of such clones promises decreased software maintenance costs of possibly the same magnitude. Previous work was limited to detection of either near misses differing only in single lexems, or near misses only between complete functions. The paper presents simple and practical methods for detecting exact and near miss clones over arbitrary program fragments in program source code by using abstract syntax trees. Previous work also did not suggest practical means for removing detected clones. Since our methods operate in terms of the program structure, clones could be removed by mechanical methods producing in-lined procedures or standard preprocessor macros. A tool using these techniques is applied to a C production software system of some 400 K source lines, and the results confirm detected levels of duplication found by previous work. The tool produces macro bodies needed for clone removal, and macro invocations to replace the clones. The tool uses a variation of the well known compiler method for detecting common sub expressions. This method determines exact tree matches; a number of adjustments are needed to detect equivalent statement sequences, commutative operands, and nearly exact matches. We additionally suggest that clone detection could also be useful in producing more structured code, and in reverse engineering to discover domain concepts and their implementations.

/pdf/clone-detection-using-abstract-syntax-trees-4rz51b6h24.pdf

Clone detection using abstract syntax trees

Detecting code clones has many software engineering applications. Existing approaches either do not scale to large code bases or are not robust against minor code modifications. In this paper, we present an efficient algorithm for identifying similar subtrees and apply it to tree representations of source code. Our algorithm is based on a novel characterization of subtrees with numerical vectors in the Euclidean space \mathbb{R}^n and an efficient algorithm to cluster these vectors w.r.t. the Euclidean distance metric. Subtrees with vectors in one cluster are considered similar. We have implemented our tree similarity algorithm as a clone detection tool called DECKARD and evaluated it on large code bases written in C and Java including the Linux kernel and JDK. Our experiments show that DECKARD is both scalable and accurate. It is also language independent, applicable to any language with a formally specified grammar.

/pdf/deckard-scalable-and-accurate-tree-based-detection-of-code-37e0p0rfkn.pdf

DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones

https://www.cs.usask.ca/~croy/papers/2009/RCK_SCP_Clones.pdf

Comparison and evaluation of code clone detection techniques and tools: A qualitative approach

This paper describes the scaling issues and progress towards constructing a practical program transformation system to support software evolution.

/pdf/dms-program-transformations-for-practical-scalable-software-2y6z4axu9s.pdf

DMS: program transformations for practical scalable software evolution

While a number of research systems have demonstratedthe potential value of program transformations, very few ofthese systems have made it into practice. The core technologyfor such systems is well understood; what remains isintegration and more importantly, the problem of handlingthe scale of the applications to be processed.This paper describes DMS, a practical, commercialprogram analysis and transformation system, and sketchesa variety of tasks to which it has been applied, from redocumentingto large-scale system migration. Its successderives partly from a vision of design maintenance and theconstruction of infrastructure that appears necessary tosupport that vision. DMS handles program scale by carefulspace management, computational scale via parallelismand knowledge acquisition scale via domains.

DMS®: Program Transformations for Practical Scalable Software Evolution

We are interested in lowering the cost of software maintenance. We believe this can be best aided by tools that have some idea of what the maintained code does and how it works. Conventional software designs lose too much information to allow effective tools for this purpose, so we turn to designs captured during transformational synthesis. Maintenance in this context requires revising the transformational design to obtain revised code, and so we call our scheme a Design Maintenance System (DMS).

Design maintenance systems

While a number of research systems have demonstrated the potential value of program transformations, very few of these systems have made it into practice The core technology for such systems is well understood; what remains is integration and more importantly, the problem of handling the scale of the applications to be processed This paper describes DMS, a practical, commercial program analysis and transformation system, and sketches a variety of tasks to which it has been applied, from redocumenting to large-scale system migration Its success derives partly from a vision of design maintenance and the construction of infrastructure that appears necessary to support that vision DMS handles program scale by careful space management, computational scale via parallelism and knowledge acquisition scale via domains

Ira D. Baxter

Papers

Clone detection using abstract syntax trees

DMS: program transformations for practical scalable software evolution

DMS®: Program Transformations for Practical Scalable Software Evolution

Design maintenance systems

DMS/spl reg/: program transformations for practical scalable software evolution