Showing papers by "Luke Zettlemoyer published in 2001"

Visual generalization in programming by example

Abstract: Publisher Summary In programming-by-example (PBE) systems, the system records the actions performed by a user in the interface and produces a generalized program that can be used later in analogous examples. A key issue is how to describe the actions and objects selected by the user, which determines what kind of generalizations will be possible. This chapter explores an approach using visual properties of the interaction elements themselves, such as size, shape, color, and appearance of graphical objects to describe user intentions. Visual information can supplement information available from other sources and opens up the possibility of new kinds of generalizations not possible from the application data alone. In addition, these generalizations can map more closely to the intentions of users, especially beginning users, who rely on the same visual information when making selections. Finally, visual generalization can sometimes remove one of the worst obstacles preventing the use of PBE with commercial applications—that is—reliance on application program interfaces (APIs). When necessary, PBE systems can work exclusively from the visual appearance of applications and do not need explicit cooperation from the API.