In an effort to make object recognition efficient and accurate enough for real applications; we have developed three probabilistic techniques-sensor modeling, probabilistic hypothesis generation, and robust localization-which form the basis of a promising paradigm for object recognition. Our techniques effectively exploit prior knowledge to reduce the number of hypotheses that must be tested during recognition. Our recognition approach utilizes statistical constraints on the matches between image and model features. These statistical constraints are computed using a model of the entire sensing process-resulting in more realistic and tighter constraints on matches. The candidate hypotheses are pruned by probabilistic constraint satisfaction to select likely matches based on the image evidence and prior statistical constraints. The resulting hypotheses are ordered most-likely first for verification. Thus minimizing unnecessary verifications. The reliability of the verification decision is significantly increased by the use of a robust localization algorithm. >

Sensor modeling, probabilistic hypothesis generation, and robust localization for object recognition

: This report describes work on the temporal segmentation of grasping task sequences based on human hand motion. The segmentation process results in the identification of motion breakpoints separating the different constituent phases of the grasping task. A grasping task is composed of three basic phases: pregrasp phase, static grasp phase, and manipulation phase. The pregrasp phase is the initial stage of the grasping task prior to the establishment of a stable grasp (static grasp phase) involved in the task. The manipulation phase refers to the purposeful hand-object interaction performed to achieve a goal in the task. In the pregrasp phase, the trajectory of the hand and the movement of the fingers follow an established pattern. Specifically, it comprises two parallel and simultaneous components: the hand transportation, and the hand preshape. We show that by analyzing the fingertip polygon area (which is an indication of the hand preshape) and the speed of hand movement (which is an indication of the hand transportation), we can divide a task into meaningful action segments such as approach object (which corresponds to the pregrasp phase), grasp object, manipulate object, place object, and depart (a special case of the pregrasp phase which signals the termination of the task). We introduce a measure called the volume sweep rate, which is the product of the fingertip polygon area and the hand speed. The profile of this measure is also used in the determination of the task breakpoints.

Temporal Segmentation of Tasks from Human Hand Motion

This thesis describes the approach of programming a robot by human demonstration. Our system observes a human performing a task, recognizes the human grasp, and maps it onto the manipulator. Using human actions to guide robot execution greatly reduces the planning complexity.
In analyzing the task sequence, the system first divides the observed sensory data into meaningful temporal segments, namely the pregrasp, grasp, and manipulation phases. This is done by analyzing the human hand motion profiles. The features used are the fingertip polygon area (the fingertip polygon being the polygon whose vertices are the fingertips), hand speed, and the volume sweep rate, which is the product of the first two. Segmentation is achieved by using the fact that the pregrasp phase motion profiles have characteristic bell curves.
Subsequent to task segmentation, a grasp taxonomy is used to recognize the human hand grasp. The grasp taxonomy is based on the contact web, which is a 3-D graphical representation of contact points between the hand and the grasped object. By considering the higher level concept of the virtual finger, which is the collection of physical fingers acting in a similar manner, we can recognize the type of human grasp used in the task.
The recognized grasp is used to guide the grasp planning of the manipulator. Grasp planning is performed at two levels: the functional and physical levels. At the functional level, the grasp is planned using virtual finger information. Subsequently, at the physical level, the geometric properties of the object and manipulator are considered in fine-tuning the manipulator grasp. The trajectory of the manipulator follows that of the human hand during the execution of the task. The human grasp and trajectory information enables the robot system to replicate the task.
In summary, this thesis describes a novel way of programming robots--by direct human demonstration of the task. This thesis shows that by appropriately segmenting the stream of observed data and producing abstract representations of the task, we can enable the robot to replicate human grasping tasks.

/pdf/robot-instruction-by-human-demonstration-359z8sybqy.pdf

Robot instruction by human demonstration

To alleviate the problem of overwhelming complexity in grasp synthesis and path planning associated with robot task planning, we adopt the approach of teaching the robot by demonstrating in front of it. The system has four components: the observation system, the grasping task recognition module, the task translator and the robot system. The observation system comprises an active multibaseline stereo system and a dataglove. The data stream recorded is then used to track object motion; this paper illustrates how complimentary sensory data can be used for this purpose. The data stream is also interpreted by the grasping task recognition module, which produces higher levels of abstraction to describe both the motion and actions taken in the task. The resulting information are provided to the task translator which creates commands for the robot system to replicate the observed task. In this paper we describe how these components work with special emphasis on the observation system. The robot system that we use to perform the grasping tasks comprises the PUMA 560 arm and the Utah/MIT hand. >

A robot system that observes and replicates grasping tasks

We are developing a system capable of observing a human performing a task and understanding the task well enough to replicate it. This approach is called Assembly Plan from Observation. In order to replicate the observed task, we have to analyze the entire sequence. This can be done by first segmenting the task sequence into its constituent pre-grasp, grasp, and manipulation phases. This paper describes the different analyses that can be done subsequent to the temporal segmentation. These include human grasp recognition, extraction of object motion, and the spatiofrequency (spectrogram) analysis of the manipulation phase. >

Grasp recognition and manipulative motion characterization from human hand motion sequences

: Geometric modeling systems allow users to create, store, and manipulate models of three-dimensional (3-D) solid objects. these geometric modeling systems have found many applications in CAD/CAM and robotics areas. Graphic display capability which rivals photographic techniques allows realistic visualization of design and simulation. Capabilities to compute spatial and physical properties of objects, such as mass property calculation and static interference check, are used in the design and analysis of mechanical parts and assembly. Output from the geometric modelers can be used for automatic programming of NC machines and robots. These geometric modeling systems are powerful in many application domains, but have severe limitations to be used for tasks such as model-based computer vision. Among others, (1) there is no explicit symbolic representation of the two-dimensional information obtained by the projection of the 3-D model. The output image displayed on the screen is a set of pixel intensity values, with no knowledge of the logical grouping of points, lines and polygons. Also, the relationship between 3-D and 2-D information is not maintained properly. (2) Most of the current 3-D geometric modeling systems are designed with a closed architecture, with a minimum of documentation describing the internal data structures. Moreover, some of the data structures are packed into bit-fields, making understanding and modification difficult. (3) They run as stand-alone interactive systems and cannot easily be interfaced to other programs.

B. Kumar

Papers

VANTAGE: A Frame-Based Geometric Modeling System Programmer/User's Manual, V2.0