Traditional approaches to human information processing tend to deal with perception and action planning in isolation, so that an adequate account of the perception-action interface is still missing On the perceptual side, the dominant cognitive view largely underestimates, and thus fails to account for, the impact of action-related processes on both the processing of perceptual information and on perceptual learning On the action side, most approaches conceive of action planning as a mere continuation of stimulus processing, thus failing to account for the goal-directedness of even the simplest reaction in an experimental task We propose a new framework for a more adequate theoretical treatment of perception and action planning, in which perceptual contents and action plans are coded in a common representational medium by feature codes with distal reference Perceived events (perceptions) and to-be-produced events (actions) are equally represented by integrated, task-tuned networks of feature codes – cognitive structures we call event codes We give an overview of evidence from a wide variety of empirical domains, such as spatial stimulus-response compatibility, sensorimotor synchronization, and ideomotor action, showing that our main assumptions are well supported by the data

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X01000103

The Theory of Event Coding (TEC): a framework for perception and action planning.

Toolglass™ widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor. They can be positioned with one hand while the other positions the cursor. The widgets provide a rich and concise vocabulary for operating on application objects. These widgets may incorporate visual filters, called Magic Lens™ filters, that modify the presentation of application objects to reveal hidden information, to enhance data of interest, or to suppress distracting information. Together, these tools form a see-through interface that offers many advantages over traditional controls. They provide a new style of interaction that better exploits the user’ s everyday skills. They can reduce steps, cursor motion, and errors. Many widgets can be provided in a user inter face, by designers and by users, without requiring dedicated screen space. In addition, lenses provide rich context -dependent feedback and the ability to view details and context simultaneous ly. Our widgets and lenses can be combined to form operation and viewing macros, and can be used over multiple applications. CR Categories and Subject Descriptors: I.3.6 [Computer Graphics]: Methodology and Techniques- interaction techniques; H.5.2 [Information Interfaces and Presentation]: User Interfaces- interaction styles; I.3.3 [ Computer Graphics]: Picture/Image Generation- viewing algorithms; I.3.4 [Computer Graphics]: Graphics Utilities - graphics editors

http://pages.cpsc.ucalgary.ca/~saul/581/exer.eps/1toolglassp445-bier.pdf

Toolglass and magic lenses: the see-through interface

We present steps toward a conceptual framework for tangible user interfaces. We introduce the MCRpd interaction model for tangible interfaces, which relates the role of physical and digital representations, physical control, and underlying digital models. This model serves as a foundation for identifying and discussing several key characteristics of tangible user interfaces. We identify a number of systems exhibiting these characteristics, and situate these within 12 application domains. Finally, we discuss tangible interfaces in the context of related research themes, both within and outside of the human-computer interaction domain.

/pdf/emerging-frameworks-for-tangible-user-interfaces-254dznxose.pdf

Emerging frameworks for tangible user interfaces

We introduce the concept of Graspable User Interfaces that allow direct control of electronic or virtual objects through physical handles for control. These physical artifacts, which we call "bricks," are essentially new input devices that can be tightly coupled or “attached” to virtual objects for manipulation or for expressing action (e.g., to set parameters or for initiating processes). Our bricks operate on top of a large horizontal display surface known as the "ActiveDesk." We present four stages in the development of Graspable UIs: (1) a series of exploratory studies on hand gestures and grasping; (2) interaction simulations using mock-ups and rapid prototyping tools; (3) a working prototype and sample application called GraspDraw; and (4) the initial integrating of the Graspable UI concepts into a commercial application. Finally, we conclude by presenting a design space for Bricks which lay the foundation for further exploring and developing Graspable User Interfaces.

/pdf/bricks-laying-the-foundations-for-graspable-user-interfaces-u3epik5896.pdf

Bricks: laying the foundations for graspable user interfaces

Accounts of visually directed actions usually assume that their planning begins with an intention to act. This article describes three experiments that challenged this view through the use of a stimulus-response compatibility paradigm with photographs of common graspable objects as stimuli. Participants had to decide as fast as possible whether each object was upright or inverted. Experiments 1 and 2 examined the effect of the irrelevant dimension of left-fight object orientation on bimanual and tmimanual keypress responses. Experiment 3 examined wrist rotation responses to objects requiring either clockwise or anticlockwise wrist rotations when grasped. The results (a) are consistent with the view that seen objects automatically potentiate components of the actions they afford, (b) show that compatibility effects of an irrelevant stimulus dimension can be obtained across a wide variety of naturally occurring stimuli, and (c) support the view that intentions to act operate on already existing motor representations of the possible actions in a visual scene. The use of vision to control actions has typically been framed as a problem that begins with the intention to act. How we use visual information depends, after all, on the goal of the action. Grasping a ball and kicking it require both representing different visual information and transforming that information into very different muscle commands. In this article we explore the possibility that visual objects potentiate actions even in the absence of explicit intentions to act. There are many reasons for supposing that a representation of the visual world includes information about possible actions. Perception and action are intimately linked. Our decisions to act are not made in a vacuum but are informed by the possibilities inherent in any visual scene. In this sense, vision is important for providing information about what actions are possible, as well as for the on-line control of their execution. Furthermore, knowledge of the possibilities for action depends critically on the relation between the visual world and the physical apparatus of the perceiver--a point long emphasized in the ecological approach to perception and action. How might such action possibilities be represented? A plausible proposal is that the perception of an object (or scene) results in the potentiation of the actions that

On the relations between seen objects and components of potential actions.

This article presents a tentative theoretical framework for the study of asymmetry in the context of human bimanual action. It is emphasized that in man most skilled manual activities involve two hands playing different roles, a fact that has been often overlooked in the experimental study of human manual lateralization. As an alternative to the current concepts of manual preference and manual superiority-whose relevance is limited to the particular case of unimanual actions-the more general concept of lateral preference is proposed to denote preference for one of the two possible ways of assigning two roles to two hands. A simple model describing man's favored intermanual division of labor in the model are the following. 1) The two hands represent two motors, that is, decomplexity is ignored in the suggested approach. 2) In man, the two manual motors cooperate with one another as if they were assembled in series, thereby forming a kinematic chain: In a right-hander allowed to follow his or her lateral preferences, motion produced by the right hand tends to articulate with motion produced by the left. It is suggested that the kinematic chain model may help in understanding the adaptive advantage of human manual specialization.

Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model.

We introduce semantic pointing, a novel interaction technique that improves target acquisition in graphical user interfaces (GUIs). Semantic pointing uses two independent sizes for each potential target presented to the user: one size in motor space adapted to its importance for the manipulation, and one size in visual space adapted to the amount of information it conveys. This decoupling between visual and motor size is achieved by changing the control-to-display ratio according to cursor distance to nearby targets. We present a controlled experiment supporting our hypothesis that the performance of semantic pointing is given by Fitts' index of difficulty in motor rather than visual space. We apply semantic pointing to the redesign of traditional GUI widgets by taking advantage of the independent manipulation of motor and visual widget sizes.

/pdf/semantic-pointing-improving-target-acquisition-with-control-37fzd0ozxj.pdf

Semantic pointing: improving target acquisition with control-display ratio adaptation

On Fitts's and Hooke's laws: simple harmonic movement in upper-limb cyclical aiming.

Pointing has been conceptualized and implemented so far as the act of selecting pixels in bitmap displays. We show that the current technique, which we call bitmap pointing (BMP), is often sub-optimal as it requires continuous information from the mouse while the system often just needs the discrete specification of objects. The paper introduces object pointing (OP), a novel interaction technique based on a special screen cursor that skips empty spaces, thus drastically reducing the waste of input information. We report data from 1D and 2D Fitts' law experiments showing that OP outperforms BMP and that the performance facilitation increases with the task's index of difficulty. We discuss the implementation of OP in current interfaces.

Object pointing: a complement to bitmap pointing in GUIs

The current view that the Simon effect (Simon & Small, 1969) reflects stimulus-response compatibility (SRC) is questioned Previous accounts of the Simon effect have overlooked stimulus congruity (SC), the correspondence relation borne by the two simultaneous aspects of the stimulus, a factor inevitably confounded in the Simon paradigm with irrelevant spatial S-R correspondence The Hedge and Marsh (1975) reversal effect, replicated in Experiment 1, is reinterpreted as decisive evidence that the Simon effect is entirely accounted for by SC Furthermore, in Experiment 2 irrelevant S-R correspondence was manipulated in the absence of any variation of SC, and the Simon effect vanished It is concluded that the Simon effect, contrary to current opinion, represents a spatial variant of the Stroop effect and is irrelevant to the SRC issue The view that mental operations proceed automatically at the stage of response determination loses one of its strongest empirical arguments

Yves Guiard

Papers

Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model.

Semantic pointing: improving target acquisition with control-display ratio adaptation

On Fitts's and Hooke's laws: simple harmonic movement in upper-limb cyclical aiming.

Object pointing: a complement to bitmap pointing in GUIs

Stimulus-response compatibility and the Simon effect: toward a conceptual clarification