https://www.ee.columbia.edu/%7Eajaimes/Pubs/ajaimes_iccv05.pdf

Multimodal human-computer interaction: A survey

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Mobile systems and methods of supporting natural language human-machine interactions

Our physical bodies play a central role in shaping human experience in the world, understandingof the world, and interactions in the world. This paper draws on theories of embodiment - from psychology, sociology, and philosophy - synthesizing five themes we believe are particularly salient for interaction design: thinking through doing, performance, visibility, risk, and thick practice. We intro-duce aspects of human embodied engagement in the world with the goal of inspiring new interaction design ap-proaches and evaluations that better integrate the physical and computational worlds.

/pdf/how-bodies-matter-five-themes-for-interaction-design-44b44pp70s.pdf

How bodies matter: five themes for interaction design

An enhanced text entry system using word-level analysis to automatically correct inaccuracies in user keystroke entries on reduced keyboards. The keyboard (105) may be a part of a touch-sensitive panel or display screen (100) or on a mechanical keyboard system. A method and system are defined which determine one or more alternative textual interpretations of each sequence of inputs detected within a designated auto-correcting keyboard region (106). The actual contact locations for the keystrokes may occur outside the boundaries of the specific keyboard key regions, where the distance from each contact location to each corresponding intended character may in general increase with the expected frequency of the intended word in the language or in a particular context. The user can easily select the intended word from among the generated interpretations.

Keyboard system with automatic correction

This paper provides overview and instruction regarding the evaluation of interactive information retrieval systems with users The primary goal of this article is to catalog and compile material related to this topic into a single source This article (1) provides historical background on the development of user-centered approaches to the evaluation of interactive information retrieval systems; (2) describes the major components of interactive information retrieval system evaluation; (3) describes different experimental designs and sampling strategies; (4) presents core instruments and data collection techniques and measures; (5) explains basic data analysis techniques; and (4) reviews and discusses previous studies This article also discusses validity and reliability issues with respect to both measures and methods, presents background information on research ethics and discusses some ethical issues which are specific to studies of interactive information retrieval (IIR) Finally, this article concludes with a discussion of outstanding challenges and future research directions

https://www.nowpublishers.com/article/DownloadSummary/INR-012

Methods for Evaluating Interactive Information Retrieval Systems with Users

JMUI (Journal on Multimodal User Interfaces), Special issue “Best of affective computing and intelligent Guidelines for multimodal user interface design. support, human multi-modal information processing. characteristics to the design of a user-oriented and guidelines of multimodal interface design. Artifact lifecycle management, Consumer and user, Interfaces in Automated.Aug 2 Aug 7Los Angeles, CA, USAThursday, 6 August 2015 / HCI International 20152015.hci.international/thursday​CachedDefining and Optimizing User Interfaces Information Complexity for AI Design and Development of Multimodal Applications: A Vision on Key Issues and Traditional Heuristics and Industry Guidelines to Evaluate Multimodal Digital Artifacts

Guidelines for multimodal user interface design

Given time and location information about digital photographs we can automatically generate an abundance of related contextual metadata, using off-the-shelf and Web-based data sources. Among these are the local daylight status and weather conditions at the time and place a photo was taken. This metadata has the potential of serving as memory cues and filters when browsing photo collections, especially as these collections grow into the tens of thousands and span dozens of years.We describe the contextual metadata that we automatically assemble for a photograph, given time and location, as well as a browser interface that utilizes that metadata. We then present the results of a user study and a survey that together expose which categories of contextual metadata are most useful for recalling and finding photographs. We identify among still unavailable metadata categories those that are most promising to develop next.

http://ilpubs.stanford.edu:8090/660/1/2004-43.pdf

Context data in geo-referenced digital photo collections

Text recognition has attracted considerable research interests because of its various applications. The cutting-edge text recognition methods are based on attention mechanisms. However, most of attention methods usually suffer from serious alignment problem due to its recurrency alignment operation, where the alignment relies on historical decoding results. To remedy this issue, we propose a decoupled attention network (DAN), which decouples the alignment operation from using historical decoding results. DAN is an effective, flexible and robust end-to-end text recognizer, which consists of three components: 1) a feature encoder that extracts visual features from the input image; 2) a convolutional alignment module that performs the alignment operation based on visual features from the encoder; and 3) a decoupled text decoder that makes final prediction by jointly using the feature map and attention maps. Experimental results show that DAN achieves state-of-the-art performance on multiple text recognition tasks, including offline handwritten text recognition and regular/irregular scene text recognition. Codes will be released.1

/pdf/decoupled-attention-network-for-text-recognition-27eyi2h304.pdf

Decoupled Attention Network for Text Recognition

Text recognition has attracted considerable research interests because of its various applications. The cutting-edge text recognition methods are based on attention mechanisms. However, most of attention methods usually suffer from serious alignment problem due to its recurrency alignment operation, where the alignment relies on historical decoding results. To remedy this issue, we propose a decoupled attention network (DAN), which decouples the alignment operation from using historical decoding results. DAN is an effective, flexible and robust end-to-end text recognizer, which consists of three components: 1) a feature encoder that extracts visual features from the input image; 2) a convolutional alignment module that performs the alignment operation based on visual features from the encoder; and 3) a decoupled text decoder that makes final prediction by jointly using the feature map and attention maps. Experimental results show that DAN achieves state-of-the-art performance on multiple text recognition tasks, including offline handwritten text recognition and regular/irregular scene text recognition.

An approach is provided that uses a foldable transformable display case that includes two panels. The two panels are attached using a hinge. Two components are attachable to at least one of the panels. One of the two components is an integrated display screen. Another component is a keyboard unit. At least one of the components can be removed from the panels.

Qianying Wang

Papers

Guidelines for multimodal user interface design

Context data in geo-referenced digital photo collections

Decoupled Attention Network for Text Recognition

Decoupled Attention Network for Text Recognition

Transformer Case for Notebook Slate Computer with Wireless Keyboard