Book ChapterDOI
A Multimodal framework for Interacting with Virtual Environments
Rajeev Sharma,Thomas S. Huang,Vladimir Pavlovic +2 more
- pp 53-71
TLDR
A conceptual framework for multimodal human-computer interaction for manipulating a virtual object and specific techniques are presented for using a combination of speech and gesture for manipulating virtual objects.Abstract:
Although there has been a tremendous progress in recent years in 3-D, immersive display and virtual reality (VR) technologies, the corresponding interface technologies have lagged behind. To fully exploit the potential that VR offers as a means of visualizing and interacting with complex information, it is important to develop “natural” means for interacting with the virtual display. Such natural interaction can be achieved by using an integrated approach where multiple, possibly redundant modes of input such as speech, hand gesture, gaze, and graphical feedback are used simultaneously. This paper presents a conceptual framework for multimodal human-computer interaction for manipulating a virtual object. Specific techniques are presented for using a combination of speech and gesture for manipulating virtual objects. Free hand gestures are analyzed and recognized using computer vision. The gesture analysis is done cooperatively with the speech recognition system and the graphic system. This is demonstrated with the help of an experimental VR setup used by molecular biologists for simulating and visualizing complex molecular structures.read more
Citations
More filters
Journal ArticleDOI
Visual interpretation of hand gestures for human-computer interaction: a review
TL;DR: A fraction of the recycle slurry is treated with sulphuric acid to convert at least some of the gypsum to calcium sulphate hemihydrate and the slurry comprising hemihYDrate is returned to contact the mixture of phosphate rock, phosphoric acid and recycle Gypsum slurry.
Journal ArticleDOI
Toward multimodal human-computer interface
TL;DR: It is clear that further research is needed for interpreting and fitting multiple sensing modalities in the context of HCI and the fundamental issues in integrating them at various levels, from early signal level to intermediate feature level to late decision level.
Journal ArticleDOI
Model-based estimation of 3D human motion
TL;DR: This paper presents the formulations and techniques that are developed for the 3D model-based, motion estimation of human movement from multiple cameras based on the spatio-temporal analysis of the subject's silhouette, which has the advantage that the subject does not have to wear markers or other devices.
Book ChapterDOI
Emotion analysis in man-machine interaction systems
Themis Balomenos,Amaryllis Raouzaiou,Spiros Ioannou,Athanasios Drosopoulos,Kostas Karpouzis,Stefanos Kollias +5 more
TL;DR: This paper presents a systematic approach to extracting expression related features from image sequences and inferring an emotional state via an intelligent rule-based system.
Gestural Interface to a Visual Computing Environment for Molecular
TL;DR: In this paper, the authors describe the use of visual hand gesture analysis enhanced with speech recognition for developing a bimodal gesture/speech inter-ace for controlling a 3D display.
References
More filters
Journal ArticleDOI
So you think gestures are nonverbal
TL;DR: This paper argued that gestures and speech are parts of the same psychological structure and share a computational stage, based on the very close temporal, semantic, pragmatic, pathological, and developmental parallels between speech and referential and discourse-oriented gestures.
Dissertation
Visual Recognition of American Sign Language Using Hidden Markov Models.
TL;DR: Using hidden Markov models (HMM's), an unobstrusive single view camera system is developed that can recognize hand gestures, namely, a subset of American Sign Language (ASL), achieving high recognition rates for full sentence ASL using only visual cues.
Journal ArticleDOI
A survey of glove-based input
D.J. Sturman,D. Zeltzer +1 more
TL;DR: A detailed overview of the field of glove-based input devices can be found in this paper, where the authors provide a basis for understanding the field by describing key hand-tracking technologies and applications using gloves.
Proceedings ArticleDOI
Indexing via color histograms
Michael J. Swain,Dana H. Ballard +1 more
TL;DR: In this article, color histograms are used to represent stable object representations over change in view, and demonstrates that they can differentiate among a large number of objects and can be used to search for the location of an object.
Proceedings ArticleDOI
Space-time gestures
Trevor Darrell,Alex Pentland +1 more
TL;DR: A method for learning, tracking, and recognizing human gestures using a view-based approach to model articulated objects is presented and results showing tracking and recognition of human hand gestures at over 10 Hz are presented.