The recognition of standard computational structures (cliches) in a program can help an experienced programmer understand the program. Based on the known relationships between the cliches, a hierarchical description of the program's design can be recovered. We develop and study a graph parsing approach to automating program recognition in which programs are represented as attributed dataflow graphs and a library of cliches is encoded as an attributed graph grammar. Graph parsing is used to recognize cliches in the code.
We demonstrate that this graph parsing approach is a feasible and useful way to automate program recognition. In studying this approach, we have experimented with two medium-sized, real-world simulator programs. There are three aspects of our study.
First, we evaluate our representation's ability to suppress many common forms of program variation which hinder recognition. We find that our recognition approach is robust under syntactic, organizational, and implementational variation, as well as variation due to delocalization, unfamiliar code, and some function-sharing optimizations.
Second, we investigate the expressiveness of our graph formalism for capturing programming cliches. The formalism is able to concisely encode algorithmic and data aggregation cliches.
Third, we empirically and analytically study the computational cost of our recognition approach with respect to the real-world simulator programs. We find that although our graph parsing algorithm is exponential in the worst case, its complexity is reduced in its practical application to program recognition. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

Automated program recognition by graph parsing

Syntax Graph (ASG) Derivation Spatial Relations Graph (SRG) Derivation translation syntax-directed editor interpreter/compiler layout algorithm graphics editor scanning rendering parsing Figure 3.1: Three levels of visual sentence representation. Please note that the derivation histories of SRGs and ASGs have to be modeled as separate data structures. A usual syntax-directed editor for ASGs or SRGs manipulates thereby the explicitly represented derivation history of a graph together with the graph itself. We will see later on in Section 3.4 that it is often possible to replace the graph grammar de nition of a visual language by a graph transformation system, which de nes all legal syntax-directed editor operations for a language of graphs without maintaining any derivation histories. It is then no longer possible to manipulate SRGs using a low-level graphics editor and to use a parser for regenerating the corresponding derivation history as well as the related ASG as shown in Figure 3.2. Abstract Syntax Graph (ASG) Spatial Relations Graph (SRG) translation structure-oriented editor interpreter layout algorithm layout editor rendering compiler Figure 3.2: A simpli ed visual sentence representation approach. Until now, we did not explain how ASGs and SRGs are related to each other, i.e., how an update of one graph is translated into an update of the related 120 CHAPTER 3. APPLICATION TO VISUAL LANGUAGES graph. In the most general case the relationships between ASGs and SRGs are modeled as coupled graph grammars as introduced in [80,93]. Assuming that any ASG graph grammar production is related to a single SRG graph grammar production and vice versa, it is a straightforward task to keep the ASG and SRG derivation histories and thereby the underlying graphs in a consistent state. In the absence of explicitly represented derivation histories a more primitive procedure has to be used to translate directly any ASG or SRG update into an update of the corresponding graph (as sketched in Figure 3.2.) The following subsections explain in more detail how graph grammars may be used to de ne the syntax of visual sentences in order to distinguish correct visual sentences from incorrect ones and to de ne their structure. Graph grammars are brie y introduced in the following Subsection 3.3.2. Subsections 3.3.3 and 3.3.4 then present two di erent approaches how to represent visual sentences by graphs. Subsection 3.3.3 uses hypergraphs, whereas Subsection 3.3.4 uses graph structures. For each of these approaches, di erent types of graph grammars used for describing VL syntax are discussed, too. These subsections as well as the following Section 3.5 about graph grammar parsers disregard the distinction between ASGs and SRGs to a certain extent. The discussion related to Figure 3.1 and Figure 3.2 will be resumed later on in Section 3.4. There we will see that all presented graph-based VL environment (editor) generators realize only subsets of the conceptual VL representation model of Figure 3.1. 3.3.2 Graph Grammars A visual language with a set of valid visual sentences that are represented by some kind of graphs (hypergraphs in Subsection 3.3.3 and graph structures in Subsection 3.3.4) corresponds with the language of the visual sentences' representations. Now we will brie y introduce general graph and hypergraph grammars. In the following subsections, we will then de ne several classes of grammars|context-free (CF) hypergraph grammars, context-free hypergraph grammars with embeddings (CFe), and unrestricted graph grammars|which are apt to de ne such languages of graph representations. For these grammars, we will present parsing algorithms in Section 3.5. Such a parser is necessary for distinguishing correct visual sentences from incorrect ones and for constructing the logical structure of correct diagrams. A graph grammar is similar to a string grammar in the sense that the grammar consists of nite sets of labels for nodes and edges, an axiom, i.e., an initial graph, and a nite set of productions. Each node, respectively edge, can be equipped with an element of the label set. These elements can be used as types or as terminal or nonterminal symbols. Each production consists of a 3.3. DEFINING THE SYNTAX OF VISUAL LANGUAGES 121 left-hand-side graph (LHS) and a right-hand-side graph (RHS). It describes how an occurrence of the LHS in a graph can be replaced by the RHS; the resulting graph is said to be derived from the previous one by a rewrite step. The language of a graph grammar is the set of all graphs which can be derived in a nite sequence of rewrite steps from the initial graph. A hypergraph grammar is a graph grammar with hypergraphs instead of graphs and hyperedges instead of edges. The axiom of a hypergraph grammar is a graph consisting of a single hyperedge that visits each of the axioms nodes and that has a nonterminal label (\axiom label"). Similar to string grammars certain types of graph grammars with di erent generative power are distinguished. Each type is de ned by imposing certain restrictions on their productions and permitted derivation sequences. In Subsections 3.3.3 and 3.3.4 we will discuss three grammar types. The reason for restricting grammars is that the existence of parsers for a grammar and their e ciency highly depend on these restrictions. To motivate and illustrate the need for di erent graph grammar types we will use Nassi-Shneiderman diagrams (NSDs), Message Sequence Charts (MSCs), and a speci c kind of UML class diagrams as running examples. NSDs are a VL for representing structured programs [76] by attaching boxes to obtain rectangular diagrams. MSC is a language for the description of interaction between entities [57]. A diagram in MSC describes which messages are interchanged between process instances, and what internal actions they perform. UML [82] o ers a comprehensive VL for system modelling. It contains, e.g., class diagrams for describing object-oriented classes and their relationships. In our example we restrict class diagrams to classes represented by boxes and associations represented by arrows between classes. Associations may be augmented by association classes; this is visualized by a box representing the association class connected to the association with a dashed line. Figure 3.3 shows sample diagrams for NSD, MSC, and UML class diagrams. Also similar to string grammars, graph grammars can make use of attributes. As described in the following, we extend nodes and edges by attributes which may be used to represent actual positions and sizes of pictorial objects. E.g., for the NSD example, we will use two node attributes x and y for the coordinates of the box vertex represented by this node. Of course, other information like color and texture can also be represented by attributes. Among other things, the next subsection and Section 3.4 brie y discuss how attributed graph grammars may be used to de ne the syntax of VL sentences. Attributed graph grammars may process attributes quite similar as attributed string grammars [62]: each node and each (hyper)edge with the same label has to carry attributes of the 122 CHAPTER 3. APPLICATION TO VISUAL LANGUAGES input x while x>1 x := 3x-1 x := x/2 y n x even ? i1 i2 m0 m1 msc simple a works_for Job Tasks Employee Company (b) (c) (a) Figure 3.3: Instance of a Nassi-Shneiderman diagram (a), of a Message Sequence Chart (b), and of a UML class diagram (c). same types. Each production describes relationships between attributes which are used by the production. Whenever a production is going to be applied, i.e., when the LHS is matched with a host subgraph, matching attributes are identi ed. The relationships between the production's attributes either restrict application of the production (if the attributes of the matched subgraph do not satisfy required relations) or they contain information how to compute attribute values which were yet unknown. 3.3.3 Hypergraph Representation of Visual Sentences This subsection explains how hypergraphs and hypergraph grammars can be used to model visual sentences. Primarily, we will discuss graphs which are closely related to the visual appearance of a VL sentence, i.e., the sentence's SRG. Of course, the same graph concepts can be used for the ASG. However, the structure of ASGs is less determined by the sentence's visual appearance, but more by its logical structure and how the sentence is further processed within the VL environment. Since we focus on VLs as such but less on their use within VL environments, we emphasize SRGs. We will rst describe how hypergraphs may be used as a representation of visual sentences. We will then discuss how the syntax of certain VLs can be expressed by context-free hypergraph grammars. A more expressive grammar approach that allows to describe a less restricted syntax is discussed afterwards. Even more general kinds of hypergraph grammars may be used in this context, but we restrict our discussion to these kinds of syntax since they allow for more or less e cient parsing which is described in Section 3.5. Subsection 3.3.4 will continue with a di erent approach for representing visual sentences. Since parsing is not an issue of this approach, a more general kind of syntax is described there. 3.3. DEFINING THE SYNTAX OF VISUAL LANGUAGES 123 Hypergraph Model There are two ways to use graphs for modeling visual sentences: Pictorial objects are either represented by nodes and relationships between them by edges, or pictorial objects are represented by edges and nodes are used for connections between related objects. A variant of this rst approach is discussed in detail in Subsection 3.3.4. This subsection explains the second approach in detail. It is an appropriate means for representing visual sentences in a uniform way. In particular, free-hand editing as used by DiaGen (cf. Subsection 3.4.3) bene ts from this approach. However, it requires hypergraphs instead of graphs as in 

Application of graph transformation to visual languages

One of the most fundamental questions in visual language research is how to specify a visual language. By a visual language we mean a set of diagrams which are valid “sentences” in that language where a diagram is a collection of “symbols” in a two or three dimensional space. Which sentences are valid and what their meaning is depends on spatial relationships between the symbols. Thus, for example, mathematical expressions, plans, and musical notation are commonly used visual languages. In this chapter we survey the many formalisms which have been suggested over the last three decades for visual language specification, and also indicate their possible applications.

A survey of visual language specification and recognition

Graph grammars may be used as natural and powerful syntax-definition formalisms for visual programming languages. Yet most graph-grammar parsing algorithms presented so far are either unable to recognize interesting visual languages or tend to be inefficient (with exponential time complexity) when applied to graphs with a large number of nodes and edges. This paper presents a context-sensitive graph grammar called reserved graph grammar, which can explicitly and completely describe the syntax of a wide range of diagrams using labeled graphs. The parsing algorithm of a reserved graph grammar uses a marking mechanism to avoid ambiguity in parsing and has polynomial time complexity in most cases. The paper defines a constraint condition under which a graph defined in a reserved graph grammar can be parsed in polynomial time. An algorithm for checking the condition is also provided.

/pdf/a-context-sensitive-graph-grammar-formalism-for-the-2cjot6ndfd.pdf

A context-sensitive graph grammar formalism for the specification of visual languages

We present a new graph grammar based approach for defining the syntax of visual languages and for generating visual language parsers. Its main advantage-in comparison to other visual language parsing approaches-is its ability to handle context-sensitive productions which may replace more than one non-terminal at the same time and which may contain very complex context requirements. Its implementation will be part of a forthcoming parsing toolkit for visual languages and the already existing graph grammar programming environment PROGRES.

/pdf/a-graph-grammar-approach-to-graphical-parsing-28vsk7ftbz.pdf

A graph grammar approach to graphical parsing

Plex grammars according to [13], generating two-dimensional plex structures, are a generalization of string grammars. In this paper we describe a parser for context free plex grammars. The parser is an extension of Earley's algorithm, which was originally developed for context free string grammars. Our parser is able to recognize not only complete structures generated by a plex grammar but also partial ones. The algorithm has been implemented and tested on a number of examples. The time complexity of the parser is exponential in general, but there exist subclasses of plex languages for which the parser has a polynomial time complexity.

B. Haller

Papers

A Parser for Context Free Plex Grammars