http://cvssp-data.eps.surrey.ac.uk/Personal/AdrianHilton/Publications_files/survey_techrep.pdf

A survey of advances in vision-based human motion capture and analysis

We introduce the SCAPE method (Shape Completion and Animation for PEople)---a data-driven method for building a human shape model that spans variation in both subject shape and pose. The method is based on a representation that incorporates both articulated and non-rigid deformations. We learn a pose deformation model that derives the non-rigid surface deformation as a function of the pose of the articulated skeleton. We also learn a separate model of variation based on body shape. Our two models can be combined to produce 3D surface models with realistic muscle deformation for different people in different poses, when neither appear in the training set. We show how the model can be used for shape completion --- generating a complete surface mesh given a limited set of markers specifying the target shape. We present applications of shape completion to partial view completion and motion capture animation. In particular, our method is capable of constructing a high-quality animated surface model of a moving person, with realistic muscle deformation, using just a single static scan and a marker motion capture sequence of the person.

/pdf/scape-shape-completion-and-animation-of-people-2tnl8cqmkm.pdf

SCAPE: shape completion and animation of people

Find the secret to improve the quality of life by reading this adaptive blind signal and image processing. This is a kind of book that you need now. Besides, it can be your favorite book to read after having this book. Do you ask why? Well, this is a book that has different characteristic with others. You may not need to know who the author is, how well-known the work is. As wise word, never judge the words from who speaks, but make the words as your good value to your life.

Adaptive blind signal and image processing

http://www2.imm.dtu.dk/projects/manifold/Papers/Vision-based%20human%20motion.pdf

Vision-based human motion analysis: An overview

Details in mesh animations are difficult to generate but they have great impact on visual quality. In this work, we demonstrate a practical software system for capturing such details from multi-view video recordings. Given a stream of synchronized video images that record a human performance from multiple viewpoints and an articulated template of the performer, our system captures the motion of both the skeleton and the shape. The output mesh animation is enhanced with the details observed in the image silhouettes. For example, a performance in casual loose-fitting clothes will generate mesh animations with flowing garment motions. We accomplish this with a fast pose tracking method followed by nonrigid deformation of the template to fit the silhouettes. The entire process takes less than sixteen seconds per frame and requires no markers or texture cues. Captured meshes are in full correspondence making them readily usable for editing operations including texturing, deformation transfer, and deformation model learning.

/pdf/articulated-mesh-animation-from-multi-view-silhouettes-2hk2pvjy6f.pdf

Articulated mesh animation from multi-view silhouettes

Shape-from-silhouette (SFS), also known as visual hull (VH) construction, is a popular 3D reconstruction method, which estimates the shape of an object from multiple silhouette images. The original SFS formulation assumes that the entire silhouette images are captured either at the same time or while the object is static. This assumption is violated when the object moves or changes shape. Hence the use of SFS with moving objects has been restricted to treating each time instant sequentially and independently. Recently we have successfully extended the traditional SFS formulation to refine the shape of a rigidly moving object over time. We further extend SFS to apply to dynamic articulated objects. Given silhouettes of a moving articulated object, the process of recovering the shape and motion requires two steps: (1) correctly segmenting (points on the boundary of) the silhouettes to each articulated part of the object, (2) estimating the motion of each individual part using the segmented silhouette. In this paper, we propose an iterative algorithm to solve this simultaneous assignment and alignment problem. Once we have estimated the shape and motion of each part of the object, the articulation points between each pair of rigid parts are obtained by solving a simple motion constraint between the connected parts. To validate our algorithm, we first apply it to segment the different body parts and estimate the joint positions of a person. The acquired kinematic (shape and joint) information is then used to track the motion of the person in new video sequences.

/pdf/shape-from-silhouette-of-articulated-objects-and-its-use-for-4uir7hapz9.pdf

Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture

In Part I of this paper we developed the theory and algorithms for performing Shape-From-Silhouette (SFS) across time. In this second part, we show how our temporal SFS algorithms can be used in the applications of human modeling and markerless motion tracking. First we build a system to acquire human kinematic models consisting of precise shape (constructed using the temporal SFS algorithm for rigid objects), joint locations, and body part segmentation (estimated using the temporal SFS algorithm for articulated objects). Once the kinematic models have been built, we show how they can be used to track the motion of the person in new video sequences. This marker-less tracking algorithm is based on the Visual Hull alignment algorithm used in both temporal SFS algorithms and utilizes both geometric (silhouette) and photometric (color) information.

Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

Shape-From-Silhouette (SFS) is a shape reconstruction method which constructs a 3D shape estimate of an object using silhouette images of the object. The output of a SFS algorithm is known as the Visual Hull (VH). Traditionally SFS is either performed on static objects, or separately at each time instant in the case of videos of moving objects. In this paper we develop a theory of performing SFS across time: estimating the shape of a dynamic object (with unknown motion) by combining all of the silhouette images of the object over time. We first introduce a one dimensional element called a Bounding Edge to represent the Visual Hull. We then show that aligning two Visual Hulls using just their silhouettes is in general ambiguous and derive the geometric constraints (in terms of Bounding Edges) that govern the alignment. To break the alignment ambiguity, we combine stereo information with silhouette information and derive a Temporal SFS algorithm which consists of two steps: (1) estimate the motion of the objects over time (Visual Hull Alignment) and (2) combine the silhouette information using the estimated motion (Visual Hull Refinement). The algorithm is first developed for rigid objects and then extended to articulated objects. In the Part II of this paper we apply our temporal SFS algorithm to two human-related applications: (1) the acquisition of detailed human kinematic models and (2) marker-less motion tracking.

/pdf/shape-from-silhouette-across-time-part-i-theory-and-4lxcz6e1yf.pdf

Shape-From-Silhouette Across Time Part I: Theory and Algorithms

The abilities to build precise human kinematic models and to perform accurate human motion tracking are essential in a wide variety of applications. Due to the complexity of the human bodies and the problem of self-occlusion, modeling and tracking humans using cameras are challenging tasks. In this thesis, we develop algorithms to perform these two tasks based on the shape estimation method Shape-From-Silhouette (SFS) which constructs a shape estimate (known as Visual Hull) of an object using its silhouettes images. 
In the first half of this thesis we extend the traditional SFS algorithm so that it can be used effectively for the human related applications. To perform SFS in real-time, we propose a fast testing/projection algorithm for voxel-based SFS algorithms. Moreover, we combine silhouette information over time to effectively increase the number of cameras (and hence reconstruction details) for SFS without physically adding new cameras. We first propose a new Visual Hull representation called Bounding Edges. We then analyze the ambiguity problem of aligning two Visual Hulls. Based on the analysis, we develop an algorithm to align Visual Hulls over time using stereo and an important property of the Shape-From-Silhouette principle. This temporal SFS algorithm combines both geometric constraints and photometric consistency to align Colored Surface Points of the object extracted from the silhouette and color images. Once the Visual Hulls are aligned, they are refined by compensating for the motion of the object. The algorithm is developed for both rigid and articulated objects. 
In the second half of this thesis we show how the improved SFS algorithms are used to perform the tasks of human modeling and motion tracking. First we build a system to acquire human kinematic models consisting of precise shape and joint locations. Once the kinematic models are built, they are used to track the motion of the person in new video sequences. The tracking algorithm is based on the Visual Hull alignment idea used in the temporal SFS algorithms. Finally we demonstrate how the kinematic model and the tracked motion data can be used for image-based rendering and motion transfer between two people.

Visual hull construction, alignment and refinement for human kinematic modeling, motion tracking and rendering

The problem of determining the unknown responses of a system which is continuously excited by cyclostationary signals is considered. By exploiting the periodicity of the input impulses, an approach based on array signal processing techniques is proposed to estimate the system responses. This method generalizes Bresler's (1989) idea of resolving overlapping echoes by partitioning the signals into portions which fit the mathematical formulation used by Bresler. A compensating algorithm is also devised to compensate the random uncertainties and perturbations of the system and the signals. Once the system responses are determined, standard methods can be used to find the positions and amplitudes of the impulses. Prospective and promising simulation results are obtained.

Kong-Man (German) Cheung

Papers

Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture

Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

Shape-From-Silhouette Across Time Part I: Theory and Algorithms

Visual hull construction, alignment and refinement for human kinematic modeling, motion tracking and rendering

Blind deconvolution of system with unknown response excited by cyclostationary impulses