We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. We quantitatively evaluate variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend-SCAPE model trained on the same data. We also extend SMPL to realistically model dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

SMPL: a skinned multi-person linear model

Diffraction characteristics of general dielectric planar (slab) gratings and surface-relief (corrugated) gratings are reviewed. Applications to laser-beam deflection, guidance, modulation, coupling, filtering, wavefront reconstruction, and distributed feedback in the fields of acoustooptics, integrated optics, holography, and spectral analysis are discussed. An exact formulation of the grating diffraction problem without approximations (rigorous coupled-wave theory developed by the authors) is presented. The method of solution is in terms of state variables and this is presented in detail. Then, using a series of fundamental assumptions, this rigorous theory is shown to reduce to the various existing approximate theories in the appropriate limits. The effects of these fundamental assumptions in the approximate theories are quantified and discussed.

Analysis and applications of optical diffraction by gratings

We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors. Our DynamicFusion approach reconstructs scene geometry whilst simultaneously estimating a dense volumetric 6D motion field that warps the estimated geometry into a live frame. Like KinectFusion, our system produces increasingly denoised, detailed, and complete reconstructions as more measurements are fused, and displays the updated model in real time. Because we do not require a template or other prior scene model, the approach is applicable to a wide range of moving objects and scenes.

/pdf/dynamicfusion-reconstruction-and-tracking-of-non-rigid-2c3vjxgb6q.pdf

DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time

In order to enable optical systems to operate with a high degree of compactness and reliability it is necessary to combine large number of optical functions in small monolithic structures. A development, somewhat reminiscent of that that took place in Integrated Electronics, is now beginning to take place in optics. The initial challenge in this emerging field, known appropriately as "Integrated Optics", is to demonstrate the possibility of performing basic optical functions such as light generation, coupling, modulation, and guiding in Integrated Optical configurations. The talk will review the main theoretical and experimental developments to date in Integrated Optics. Specific topics to be discussed include: Material considerations, guiding mechanisms, modulation, coupling and mode losses. The fabrication and applications of periodic thin film structures will be discussed.

Integrated optics

We describe the fabrication and characterization of optical waveguides formed in LiNbO3 by proton exchange in benzoic acid melts at 200–250 °C. Proton exchange, in LiNbO3 the replacement of lithium ions with protons, takes place when the substrate is immersed in the molten acid. We observe a surface increase in the refractive index of 0.12, for the extraordinary polarization only, with a step function index profile. This is the highest index increase obtainable to date for LiNbO3. Measured diffusion rates for x‐cut crystals are 1.37 μm2/h at T = 249 °C and 0.37 μm2/h at T = 217 °C, so that very deep guides can be formed in short times. Diffusion is somewhat slower in the z direction. The process as described is not useful for y‐cut crystals, since it damgaes this surface. Losses, measured on x‐cut samples, were <0.5 dB/cm. All measurements were made at 0.633 μm.

Proton exchange for high‐index waveguides in LiNbO3

True2Form is a sketch-based modeling system that reconstructs 3D curves from typical design sketches. Our approach to infer 3D form from 2D drawings is a novel mathematical framework of insights derived from perception and design literature. We note that designers favor viewpoints that maximally reveal 3D shape information, and strategically sketch descriptive curves that convey intrinsic shape properties, such as curvature, symmetry, or parallelism. Studies indicate that viewers apply these properties selectively to envision a globally consistent 3D shape. We mimic this selective regularization algorithmically, by progressively detecting and enforcing applicable properties, accounting for their global impact on an evolving 3D curve network. Balancing regularity enforcement against sketch fidelity at each step allows us to correct for inaccuracy inherent in free-hand sketching. We perceptually validate our approach by showing agreement between our algorithm and viewers in selecting applicable regularities. We further evaluate our solution by: reconstructing a range of 3D models from diversely sourced sketches; comparisons to prior art; and visual comparison to both ground-truth and 3D reconstructions by designers.

https://hal.inria.fr/hal-01060850/document

True2Form: 3D curve networks from 2D sketches via selective regularization

We present a novel scanning system for capturing a full 3D human body model using just a single depth camera and no auxiliary equipment. We claim that data captured from a single Kinect is sufficient to produce a good quality full 3D human model. In this setting, the challenges we face are the sensor's low resolution with random noise and the subject's non-rigid movement when capturing the data. To overcome these challenges, we develop an improved super-resolution algorithm that takes color constraints into account. We then align the super-resolved scans using a combination of automatic rigid and non-rigid registration. As the system is of low price and obtains impressive results in several minutes, full 3D human body scanning technology can now become more accessible to everyday users at home.

/pdf/kinectavatar-fully-automatic-body-capture-using-a-single-z0pjejdlgg.pdf

KinectAvatar: fully automatic body capture using a single kinect

We present an unsupervised algorithm for aligning a pair of shapes in the presence of significant articulated motion and missing data, while assuming no knowledge of a template, user-placed markers, segmentation, or the skeletal structure of the shape. We explicitly sample the motion, which gives a priori the set of possible rigid transformations between parts of the shapes. This transforms the problem into a discrete labeling problem, where the goal is to find an optimal assignment of transformations for aligning the shapes. We then apply graph cuts to optimize a novel cost function, which encodes a preference for a consistent motion assignment from both source to target and target to source. We demonstrate the robustness of our method by aligning several synthetic and real-world datasets.

/pdf/automatic-registration-for-articulated-shapes-5aoichhdc7.pdf

Automatic registration for articulated shapes

We accurately capture the shape and appearance of a person's hairstyle. We use triangulation and a sweep with planes of light for the geometry. Multiple projectors and cameras address the challenges raised by the reflectance and intricate geometry of hair. We introduce the use of structure tensors to infer the hidden geometry between the hair surface and the scalp. Our triangulation approach affords substantial accuracy improvement and we are able to measure elaborate hair geometry including complex curls and concavities. To reproduce the hair appearance, we capture a six-dimensional reflectance field. We introduce a new reflectance interpolation technique that leverages an analytical reflectance model to alleviate cross-fading artifacts caused by linear methods. Our results closely match the real hairstyles and can be used for animation.

Hair photobooth: geometric and photometric acquisition of real hairstyles

We present the articulated global registration algorithm to reconstruct articulated 3D models from dynamic range scan sequences. This new algorithm aligns multiple range scans simultaneously to reconstruct a full 3D model from the geometry of these scans. Unlike other methods, we express the surface motion in terms of a reduced deformable model and solve for joints and skinning weights. This allows a user to interactively manipulate the reconstructed 3D model to create new animations.We express the global registration as an optimization of both the alignment of the range scans and the articulated structure of the model. We employ a graph-based representation for the skinning weights that successfully handles difficult topological cases well. Joints between parts are estimated automatically and are used in the optimization to preserve the connectivity between parts. The algorithm also robustly handles difficult cases where parts suddenly disappear or reappear in the range scans. The global registration produces a more accurate registration compared to a sequential registration approach, because it estimates the articulated structure based on the motion observed in all input frames. We show that we can automatically reconstruct a variety of articulated models without the use of markers, user-placed correspondences, segmentation, or template model.

/pdf/global-registration-of-dynamic-range-scans-for-articulated-4j5b3yjl8b.pdf

William S. C. Chang

Papers

True2Form: 3D curve networks from 2D sketches via selective regularization

KinectAvatar: fully automatic body capture using a single kinect

Automatic registration for articulated shapes

Hair photobooth: geometric and photometric acquisition of real hairstyles

Global registration of dynamic range scans for articulated model reconstruction