A user-friendly editor for structurally represented computer programs is disclosed. The present editor combines advantages of text editors and structure editors by transforming, as the user types, the event stream stream entered by the user into a token stream, where the tokens of the token stream constitute the words of the program being entered. Each of the tokens is classified by the editor as one of group of extended lexemes defined by the language in which the program being edited is written. These extended lexemes are defined similar to lexemes that might be used in a batch lexer, but are more numerous as the present editor must account for the incomplete and ill-formed lexemes that arise as the user types the program. Based on information in the token stream, the editor prettyprints the program as the user types. The editor also allows the user to edit the program from the prettyprinted display as if the program were internally represented as text. The present editor also allows the user to periodically request a syntactical analysis of the program, wherein the structure of the program is represented in a syntax tree that references the token stream.

System and method for textual editing of structurally-represented computer programs with on-the-fly typographical display

Powerful editing systems for developing complex software documents are difficult to engineer. Besides requiring efficient incremental algorithms and complex data structures, such editors must accommodate flexible editing styles, provide a consistent, coherent, and powerful user interface, support individual variations and projectwide configurations, maintain a sharable database of information concerning the documents being edited, and integrate smoothly with the other tools in the environment. Pan is a language-based editing and browsing system that exhibits these characteristics. This paper surveys the design and engineering of Pan, paying particular attention to a number of issues that pervade the system: incremental checking and analysis, information retention in the presence of change, tolerance for errors and anomalies, and extension facilities.

/pdf/the-pan-language-based-editing-system-6d6w3ige7p.pdf

The Pan language-based editing system

An editor for structurally represented computer programs transforms user-entered text on-the-fly into a stream of tokens that constitute words of the program under edit. Each token is classified as one of group of extended lexemes, and based upon token stream information the editor prettyprint displays the program as the user types. Prettyprinting involves typesetting each token in a visually distinct manner and displaying a varying amount of visual inter-token whitespace between the tokens, based upon token lexical type. The program may be user-edited from the prettyprinted display as though the program were internally represented as text. Cursor position and display appearance depend on the lexical types of tokens adjacent the cursor. To improve aesthetics of the prettyprinted display, a user may insert one or more alignment markers into lines of associated text. The presence of such marker(s) forces horizontal alignment between associated text lines containing such markers. The presence, number, and occurrence of such markers in associated lines of text is noted, and the pixel distance from a boundary edge to the first occurring marker in each line is calculated. The maximum such distance determines relative position of the first alignment marker. Pixel units are added to the whitespace gap preceding the first marker in the other associated lines to force such markers into alignment with the marker whose position represented the maximum distance. This process is then repeated for second alignment markers in each line, third alignment markers, and so on.

System and method for horizontal alignment of tokens in a structural representation program editor

technology to benefit software developers. The Pan sysPowerful editing systems for developing complex software documents are difficult to engineer. Besides requiring efficient incremental algorithms and complex data structures, such editors must integrate smoothly with the other tools in the environment, maintain a sharable database of information concerning the documents being edited, accommodate flexible editing styles, provide a consistent, coherent, and empowering user interface, and support individual variations and project-wide configurations. Pun is a languagebased editing and browsing system that exhibits these characteristics. This paper surveys the design and engineering of Pan, paying particular attention to a number of issues that pervade the system: incremental checking and analysis, information retention in the presence of change, tolerance for errors and anomalies, and extension facilities. tem 1. rests on three fundamental assumptions. Software is composed of textual documents having many structures that can be derived using language-based analysis. “Syntax” is just one such derived structure.

/pdf/the-pan-language-based-editing-system-for-integrated-3ft5rsxck5.pdf

The Pan Language-Based Editing System For Integrated Development Environments*

Previously published algorithms for LR (k) incremental parsing are inefficient, unnecessarily restrictive, and in some cases incorrect. We present a simple algorithm based on parsing LR(k) sentential forms that can incrementally parse an arbitrary number of textual and/or structural modifications in optimal time and with no storage overhead. The central role of balanced sequences in achieving truly incremental behavior from analysis algorithms is described, along with automated methods to support balancing during parse table generation and parsing. Our approach extends the theory of sentential-form parsing to allow for ambiguity in the grammar, exploiting it for notational convenience, to denote sequences, and to construct compact (“abstract”) syntax trees directly. Combined, these techniques make the use of automatically generated incremental parsers in interactive software development environments both practical and effective. In addition, we address information preservation in these environments: Optimal node reuse is defined; previous definitions are shown to be insufficient; and a method for detecting node reuse is provided that is both simpler and faster than existing techniques. A program representation based on self-versioning documents is used to detect changes in the program, generate efficient change reports for subsequent analyses, and allow the parsing transformation itself to be treated as a reversible modification in the edit log.

Efficient and flexible incremental parsing

Processors for programming languages and other formal languages typically use a concrete syntax to describe the user's view of a program and an abstract syntax to represent language structures internally. Grammatical abstraction is defined as a relationship between two context-free grammars. It formalizes the notion of one syntax being “more abstract” than another. Two variants of abstraction are presented. Weak grammatical abstraction supports (i) the construction during LR parsing of an internal representation that is closely related to the abstract syntax and (ii) incremental LR parsing using that internal representation as its base. Strong grammatical abstraction tightens the correspondence so that top-down construction of incrementally-parsable internal representations is possible. These results arise from an investigation into language-based editing systems, but apply to any program that transforms a linguistic object from a representation in its concrete syntax to a representation in its abstract syntax or vice versa.

J. Butcher

Papers

Grammatical abstraction and incremental syntax analysis in a language-based editor