Showing papers by "Björn Hartmann published in 2017"

Learning syntactic program transformations from examples

Abstract: Automatic program transformation tools can be valuable for programmers to help them with refactoring tasks, and for Computer Science students in the form of tutoring systems that suggest repairs to programming assignments. However, manually creating catalogs of transformations is complex and time-consuming. In this paper, we present Refazer, a technique for automatically learning program transformations. Refazer builds on the observation that code edits performed by developers can be used as input-output examples for learning program transformations. Example edits may share the same structure but involve different variables and subexpressions, which must be generalized in a transformation at the right level of abstraction. To learn transformations, Refazer leverages state-of-the-art programming-by-example methodology using the following key components: (a) a novel domain-specific language (DSL) for describing program transformations, (b) domain-specific deductive algorithms for efficiently synthesizing transformations in the DSL, and (c) functions for ranking the synthesized transformations. We instantiate and evaluate Refazer in two domains. First, given examples of code edits used by students to fix incorrect programming assignment submissions, we learn program transformations that can fix other students' submissions with similar faults. In our evaluation conducted on 4 programming tasks performed by 720 students, our technique helped to fix incorrect submissions for 87% of the students. In the second domain, we use repetitive code edits applied by developers to the same project to synthesize a program transformation that applies these edits to other locations in the code. In our evaluation conducted on 56 scenarios of repetitive edits taken from three large C# open-source projects, Refazer learns the intended program transformation in 84% of the cases using only 2.9 examples on average.

Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis

Abstract: In large introductory programming classes, teacher feedback on individual incorrect student submissions is often infeasible. Program synthesis techniques are capable of fixing student bugs and generating hints automatically, but they lack the deep domain knowledge of a teacher and can generate functionally correct but stylistically poor fixes. We introduce a mixed-initiative approach which combines teacher expertise with data-driven program synthesis techniques. We demonstrate our novel approach in two systems that use different interaction mechanisms. Our systems use program synthesis to learn bug-fixing code transformations and then cluster incorrect submissions by the transformations that correct them. The MistakeBrowser system learns transformations from examples of students fixing bugs in their own submissions. The FixPropagator system learns transformations from teachers fixing bugs in incorrect student submissions. Teachers can write feedback about a single submission or a cluster of submissions and propagate the feedback to all other submissions that can be fixed by the same transformation. Two studies suggest this approach helps teachers better understand student bugs and write reusable feedback that scales to a massive introductory programming classroom.

Shot Orientation Controls for Interactive Cinematography with 360 Video

Abstract: Virtual reality filmmakers creating 360-degree video currently rely on cinematography techniques that were developed for traditional narrow field of view film. They typically edit together a sequence of shots so that they appear at a fixed-orientation irrespective of the viewer's field of view. But because viewers set their own camera orientation they may miss important story content while looking in the wrong direction. We present new interactive shot orientation techniques that are designed to help viewers see all of the important content in 360-degree video stories. Our viewpoint-oriented technique reorients the shot at each cut so that the most important content lies in the the viewer's current field of view. Our active reorientation technique, lets the viewer press a button to immediately reorient the shot so that important content lies in their field of view. We present a 360-degree video player which implements these techniques and conduct a user study which finds that users spend 5.2-9.5% more time viewing the important points (manually labelled) of the scene with our techniques compared to the traditional fixed-orientation cuts. In practice, 360-degree video creators may label important content, but we also provide an automatic method for determining important content in existing 360-degree videos.

Bifröst: Visualizing and Checking Behavior of Embedded Systems across Hardware and Software

Abstract: The Maker movement has encouraged more people to start working with electronics and embedded processors. A key challenge in developing and debugging custom embedded systems is understanding their behavior, particularly at the boundary between hardware and software. Existing tools such as step debuggers and logic analyzers only focus on software or hardware, respectively. This paper presents a new development environment designed to illuminate the boundary between embedded code and circuits. Bifrost automatically instruments and captures the progress of the user's code, variable values, and the electrical and bus activity occurring at the interface between the processor and the circuit it operates in. This data is displayed in a linked visualization that allows navigation through time and program execution, enabling comparisons between variables in code and signals in circuits. Automatic checks can detect low-level hardware configuration and protocol issues, while user-authored checks can test particular application semantics. In an exploratory study with ten participants, we investigated how Bifrost influences debugging workflows.

Turning coders into makers: the promise of embedded design generation

Abstract: As personal fabrication becomes increasingly accessible and popular, a larger number of makers, many without formal training, are dabbling in embedded and electronics design. However, existing general-purpose, board-level circuit design techniques do not share desirable properties of modern software development, like rich abstraction layers and automated compiler checks, which facilitate powerful tools that ultimately lower the barrier to entry for programming, by allowing a higher level of design-separating specification from implementation-and providing automated guidance and feedback. In this paper, we present a novel methodology for embedded design generation that allows the generation of complete designs from high-level specifications. We present an implementation capable of synthesizing a variety of examples to show that our approach is viable. Starting from user-specified requirements and a library of available components, our tool encodes the design space as a system of constraints. Off-the-shelf solvers then reason over these constraints to create a block diagram with sufficient information to generate the device firmware and circuit netlist.

Exploring the Design Space of Automatically Synthesized Hints for Introductory Programming Assignments

Abstract: For massive programming classrooms, recent advances in program synthesis offer means to automatically grade and debug student submissions, and generate feedback at scale. A key challenge for synthesis-based autograders is how to design personalized feedback for students that is as effective as manual feedback given by teachers today. To understand the state of hint-giving practice, we analyzed 132 online Q&A posts and conducted a semi-structured interview with a teacher from a local massive programming class. We identified five types of teacher hints that can also be generated by program synthesis. These hints describe transformations, locations, data, behavior, and examples. We describe our implementation of three of these hint types. This work paves the way for future deployments of automatic, pedagogically-useful programming hints driven by program synthesis.

Direct and immediate drawing with CNC machines

Abstract: We investigate how the immediate control of computer numerical control (CNC) machines can narrow the design-fabrication gap and combine manual art practice with digital fabrication. LINC (Live Interactive Numeric Control) is a sketch-based digital design tool for authoring 2D or 3D artworks in near-real time. To use LINC, users draw strokes which are then executed in one of three modes---static, manual, or as soon as possible---by either a large-scale or desktop-sized modified CNC router. We evaluate LINC through a study with eight artists.

TraceDiff: Debugging unexpected code behavior using trace divergences

Abstract: Recent advances in program synthesis offer means to automatically debug student submissions and generate personalized feedback in massive programming classrooms. When automatically generating feedback for programming assignments, a key challenge is designing pedagogically useful hints that are as effective as the manual feedback given by teachers. Through an analysis of teachers' hint-giving practices in 132 online Q&A posts, we establish three design guidelines that an effective feedback design should follow. Based on these guidelines, we develop a feedback system that leverages both program synthesis and visualization techniques. Our system compares the dynamic code execution of both incorrect and fixed code and highlights how the error leads to a difference in behavior and where the incorrect code trace diverges from the expected solution. Results from our study suggest that our system enables students to detect and fix bugs that are not caught by students using another existing visual debugging tool.

Machine Learning for Makers: Interactive Sensor Data Classification Based on Augmented Code Examples

Abstract: Although many software libraries and hardware modules support reading data from sensors, makers of interactive systems often struggle to extract higher-level information from raw sensor data. Available general-purpose machine learning (ML) libraries remain difficult to use for non-experts. Prior research has sought to bridge this gap through domain-specific user interfaces for particular types of sensors or algorithms. Our ESP (Example-based Sensor Prediction) system introduces a more general approach in which interactive visualizations and control interfaces are dynamically generated from augmented code examples written by experts. ESP's augmented examples allow experts to write logic that guides makers through important steps such as sensor calibration, parameter tuning, and assessing signal quality and classification performance. Writing augmented examples requires additional effort. ESP leverages a fundamental dynamic of online communities: experts are often willing to invest such effort to teach and train novices. Thus support for particular sensing domains does not have to be hard-wired a priori by system authors, but can be provided later by its community of users. We illustrate ESP's flexibility by detailing pipelines for four distinct sensors and classification algorithms. We validated the usability and flexibility of our example-based approach through a one-day workshop with 11 participants.

CHI 2017 Stories Overview

Abstract: CHI Stories is a new venue introduced at CHI 2017. A diverse set of storytellers describe personal experiences that shaped who they are and how they came to the field of Human-Computer Interaction.

TraceDiff: Debugging Unexpected Code Behavior Using Trace Divergences

Abstract: Recent advances in program synthesis offer means to automatically debug student submissions and generate personalized feedback in massive programming classrooms. When automatically generating feedback for programming assignments, a key challenge is designing pedagogically useful hints that are as effective as the manual feedback given by teachers. Through an analysis of teachers' hint-giving practices in 132 online Q&A posts, we establish three design guidelines that an effective feedback design should follow. Based on these guidelines, we develop a feedback system that leverages both program synthesis and visualization techniques. Our system compares the dynamic code execution of both incorrect and fixed code and highlights how the error leads to a difference in behavior and where the incorrect code trace diverges from the expected solution. Results from our study suggest that our system enables students to detect and fix bugs that are not caught by students using another existing visual debugging tool.