Future computing environments will free the user from the constraints of the desktop. Applications for a mobile environment should take advantage of contextual information, such as position, to offer greater services to the user. In this paper, we present the Cyberguide project, in which we are building prototypes of a mobile context-aware tour guide. Knowledge of the user's current location, as well as a history of past locations, are used to provide more of the kind of services that we come to expect from a real tour guide. We describe the architecture and features of a variety of Cyberguide prototypes developed for indoor and outdoor use on a number of different hand-held platforms. We also discuss the general research issues that have emerged in our context-aware applications development in a mobile environment.

https://condor.depaul.edu/dmumaugh/readings/handouts/SE430/p421-abowd-97.pdf

Cyberguide: a mobile context-aware tour guide

The proliferation of computing into the physical world promises more than the ubiquitous availability of computing infrastructure; it suggest new paradigms of interaction inspired by constant access to information and computational capabilities. For the past decade, application-driven research on abiquitous computing (ubicomp) has pushed three interaction themes:natural interfaces, context-aware applications,andautomated capture and access. To chart a course for future research in ubiquitous computing, we review the accomplishments of these efforts and point to remaining research challenges. Research in ubiquitious computing implicitly requires addressing some notion of scale, whether in the number and type of devices, the physical space of distributed computing, or the number of people using a system. We posit a new area of applications research, everyday computing, focussed on scaling interaction with respect to time. Just as pushing the availiability of computing away from the traditional desktop fundamentally changes the relationship between humans and computers, providing continuous interaction moves computing from a localized tool to a constant companion. Designing for continous interaction requires addressing interruption and reumption of intreaction, representing passages of time and providing associative storage models. Inherent in all of these interaction themes are difficult issues in the social implications of ubiquitous computing and the challenges of evaluating> ubiquitious computing research. Although cumulative experience points to lessons in privacy, security, visibility, and control, there are no simple guidelines for steering research efforts. Akin to any efforts involving new technologies, evaluation strategies form a spectrum from technology feasibility efforts to long-term use studies—but a user-centric perspective is always possible and necessary

/pdf/charting-past-present-and-future-research-in-ubiquitous-2aoexf7fgc.pdf

Charting past, present, and future research in ubiquitous computing

There is more to context than location

An adaptive interface for a programmable system, for predicting a desired user function, based on user history, as well as machine internal status and context. The apparatus receives an input from the user and other data. A predicted input is presented for confirmation by the user, and the predictive mechanism is updated based on this feedback. Also provided is a pattern recognition system for a multimedia device, wherein a user input is matched to a video stream on a conceptual basis, allowing inexact programming of a multimedia device. The system analyzes a data stream for correspondence with a data pattern for processing and storage. The data stream is subjected to adaptive pattern recognition to extract features of interest to provide a highly compressed representation that may be efficiently processed to determine correspondence. Applications of the interface and system include a video cassette recorder (VCR), medical device, vehicle control system, audio device, environmental control system, securities trading terminal, and smart house. The system optionally includes an actuator for effecting the environment of operation, allowing closed-loop feedback operation and automated learning.

Adaptive pattern recognition based control system and method

A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Intuitive computing methods and systems

Mobile workers need seamless access to communication and information services while on the move. However, current solutions overwhelm users with intrusive interfaces and ambiguous notifications. This article discusses the interaction techniques developed for Nomadic Radio, a wearable computing platform for managing voice and text-based messages in a nomadic environment. Nomadic Radio employs an auditory user interface, which synchronizes speech recognition, speech synthesis, nonspeech audio, and spatial presentation of digital audio, for navigating among messages as well as asynchonous notific ation of nely arrived messages. Emphasis is placed on an auditory modality as Nomadic Radio is designed to be used while performing other tasks in a user's everyday environment; a range of auditory cues provides peripheral awareness of incoming messages. Notification is adaptive and cntext sensitive; messages are presented as more or less obtrsive based on importance inferred from content filtering, whether the user is engaged in conversation and his or her own recent responses to prior messages. Auditory notifications are dynamically scaled from ambient sound through recorded voice cues up to message summaries. Iterative design and a preliminary user evaluation suggest that audio is an appropriate medium for mobile messaging, but that care must be taken to minimally intrude on the wearer's social and physical environment.

https://interruptions.net/literature/Sawhney-TOCHI00-p353-sawhney.pdf

Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments

We view college classroom teaching and learning as a multimedia authoring activity. The classroom provides a rich setting in which a number of different forms of communication co-exist, such as speech, writing and projected images. Much of the information in a lecture is poorly recorded or lost currently. Our hypothesis is that tools to aid in the capture and subsequent access of classroom information will enhance both the learning and teaching experience. To test that hypothesis, we initiated the Classroom 2000 project at Georgia Tech. The purpose of the project is to apply ubiquitous computing technology to facilitate automatic capture, integration and access of multimedia information in the educational setting of the university classroom. In this paper, we discuss various prototype tools we have created and used in a variety of courses and provide an initial evaluation of the acceptance and effectiveness of the technology. We also share some lessons learned in applying ubiquitous computing technology in a real setting.

http://www.aquaphoenix.com/presentation/candidacy/abowd96.pdf

Teaching and learning as multimedia authoring: the classroom 2000 project

Dynomite is a portable electronic notebook for the capture and retrieval of handwritten and audio notes. Dynomite provides a casual, paper like interface, allowing the user to write or draw on a screen resembling an ordinary paper notebook. Indexing is provided by allowing users to attach properties, or data types, to specific ink, and by allowing the user to associate text keywords with pages of notes. Dynomite notes can be organized into sub notebooks based on these indexes. In addition, Dynomite records audio synchronously with notes and allows users to mark or highlight significant sections, so that they may be easily located and replayed. In order to understand how well the features of Dynomite support natural note taking practices, we conducted usage studies, which showed that Dynomite's indexing capabilities clearly improve the ability to find information in notes. (4 pages)

Dynomite: a dynamically organized ink and audio notebook

HyperCafe is an experimental hypermedia prototype, developed as an illustration of a general hypervideo system. This program places the user in a virtual cafe, composed primarily of digital video clips of actors involved in fictional conversations in the cafe; HyperCafe allows the user to follow different conversations, and offers dynamic opportunities of interaction via temporal, spatio-temporal and textual links to present alternative narratives. Textual elements are also present in the form of explanatory text, contradictory subtitles, and intruding narratives. Based on our work with HyperCafe, we discuss the components and a framework for hypervideo structures, along with the underlying aesthetic considerations.

HyperCafe: narrative and aesthetic properties of hypervideo

An electronic notebook assigns properties to digital ink of note marks and keywords to digital pages. The properties indicate the type of note mark and the keywords indicate general content of the note marks on a digital page. The notebook also synchronizes audio data with the note marks and permits audio highlighting. The electronic notebook provides an updatable database of note marks and audio that is capable of being searched based on the properties, keywords, and audio highlights.

Nitin Sawhney

Papers

Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments

Teaching and learning as multimedia authoring: the classroom 2000 project

Dynomite: a dynamically organized ink and audio notebook

HyperCafe: narrative and aesthetic properties of hypervideo

System for capturing and retrieving audio data and corresponding hand-written notes