Rajeev Sharma

Bio: Rajeev Sharma is an academic researcher from Pennsylvania State University. The author has contributed to research in topics: Gesture & Gesture recognition. The author has an hindex of 34, co-authored 107 publications receiving 5446 citations. Previous affiliations of Rajeev Sharma include University of Illinois at Urbana–Champaign.

Tracking body parts of multiple people: a new approach

Abstract: Tracking body parts of multiple people in a video sequence is very useful for face/gesture recognition systems as well as human computer interaction (HCI) interfaces. This paper describes a framework for tracking multiple objects (e.g., hands and faces of multiple people) in a video stream. We use a probabilistic model to fuse the color and motion information to localize the body parts and employ a multiple hypothesis tracking (MHT) algorithm to track these features simultaneously. The MHT algorithm is capable of tracking multiple objects with limited occlusions and is suitable for resolving any data association uncertainty. We incorporated a path coherence function along with MHT to reduce the negative effects of spurious measurements that produce unconvincing tracks and needless computations. The performance of the framework has been validated using experiments on synthetic and real sequence of images.

GeoCollaborative crisis management: using maps to mediate EOC--mobile team collaboration

Abstract: Managing large scale and distributed crisis events is a national priority; and it is a priority that presents information technology challenges to the responsible government agencies. Geographical information systems (with their ability to map out evolving crisis events, affected human and infrastructure assets, as well as actions taken and resources applied) have been indispensable in all stages of crisis management. Their use, however, has been mostly confined to single users within single agencies. The potential for maps and related geospatial technologies to be the media for collaborative activities among distributed agencies and teams have been discussed [1-4], but feasible technological infrastructure and tools are not yet available. An interdisciplinary team from Penn State University (comprised of GIScientists, information Scientists and computer scientists), currently funded by the NSF/DG program, have joined efforts with collaborators from federal, state, and local agencies to develop an approach to and technology to support "GeoCollaborative Crisis Management" (NSF-EIA-0306845). The dual goals of this project are: (1) to understand the roles of geographical information distributed crisis management activities; and (2) to develop enabling geospatial information technologies and human-computer systems to facilitate geocollaborative crisis management. This demonstration presents initial progress towards supporting geocollaborative activities, focusing on one type of collaboration involving crisis managers in the field coordinating with those in an emergency operation center (EOC).

Improving continuous gesture recognition with spoken prosody

Abstract: Despite recent advances in gesture recognition, reliance on the visual signal alone to classify unrestricted continuous gesticulation is inherently error-prone. Since spontaneous gesticulation is mostly coverbal in nature, there have been some attempts of using speech cues to improve gesture recognition. Some attempts have been made in using speech cues to improve gesture recognition, e.g., keyword-gesture co-analysis. Use of such scheme is burdened by the complexity of natural language understanding. This paper offers a "signal-level" perspective by exploring prosodic phenomena of spontaneous gesture and speech co-production. We present a computational framework for improving continuous gesture recognition based on two phenomena that capture voluntary (co-articulation) and involuntary (physiological) contributions of prosodic synchronization. Physiological constraints, manifested as signal interruptions in multimodal production, are exploited in an audio-visual feature integration framework using hidden Markov models (HMMs). Co-articulation is analyzed using a Bayesian network of naive classifiers to explore alignment of intonationally prominent speech segments and hand kinematics. The efficacy of the proposed approach was demonstrated on a multimodal corpus created from the Weather Channel broadcast. Both schemas were found to contribute uniquely by reducing different error types, which subsequently improves the performance of continuous gesture recognition.

Enabling GeoCollaborative crisis management through advanced geoinformation technologies

Abstract: Crisis events have dramatic impact on human society, economy and our environment. Geographical information and intelligence play a key role in crisis management activities. However, the use of geographical information technologies in responsible government agencies has been mostly confined to single users, and within single agency. An interdisciplinary team from Penn State University (comprised of GIScientists, information scientists, and computer scientists) has joined efforts with collaborators from federal, state, and local agencies to develop advanced geospatial information technologies that support GeoCollaborative Crisis Management (GCCM). In this demonstration, we present our progress in the design and implementation of a GIS-mediated collaborative environment that enables crisis managers and collaborating agencies to work together with geographical information. The system features multimodal interactions, mixed-initiative conversational dialogues, and map-mediated communication. It can be used by managers in emergency operation centers (EOC) as well as first responders in the field.

Occlusion handling in augmented reality using background-foreground segmentation and projective geometry

Abstract: Developing a seamless merging of real and virtual image streams and 3D models is an active research topic in augmented reality (AR). We propose a method for real-time augmentation of real videos with 2D and 3D objects by addressing the occlusion issue in an unique fashion. For virtual planar objects (such as images), the 2D overlay is automatically overlaid in a planar region selected by the user in the video. The overlay is robust to arbitrary camera motion. Furthermore, a unique background-foreground segmentation algorithm renders this augmented overlay as part of the background if it coincides with foreground objects in the video stream, giving the impression that it is occluded by foreground objects. The proposed technique does not require multiple cameras, camera calibration, use of fiducials, or a structural model of the scene to work. Extending the work further, we propose a novel method of augmentation by using trifocal tensors to augment 3D objects in 3D scenes to similar effect and implement it in real time as a proof of concept. We show several results of the successful working of our algorithm in real-life situations. The technique works on a real-time video from a USB camera, Creative Webcam III, on a P IV 1.6 GHz system without any special hardware support.

A survey of augmented reality

Abstract: This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.

Planning Algorithms: Introductory Material

Abstract: Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

A tutorial on visual servo control

Abstract: This article provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed in detail. Since any visual servo system must be capable of tracking image features in a sequence of images, we also include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control.

Robot Modeling and Control

Abstract: Preface. 1. Introduction. 2. Rigid Motions and Homogeneous Transformations. 3. Forward and Inverse Kinematics. 4. Velocity Kinematics-The Jacobian. 5. Path and Trajectory Planning. 6. Independent Joint Control. 7. Dynamics. 8. Multivariable Control. 9. Force Control. 10. Geometric Nonlinear Control. 11. Computer Vision. 12. Vision-Based Control. Appendix A: Trigonometry. Appendix B: Linear Algebra. Appendix C: Dynamical Systems. Appendix D: Lyapunov Stability. Index.