Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

A morphable model for the synthesis of 3D faces

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

Generative 3D face models are a powerful tool in computer vision. They provide pose and illumination invariance by modeling the space of 3D faces and the imaging process. The power of these models comes at the cost of an expensive and tedious construction process, which has led the community to focus on more easily constructed but less powerful models. With this paper we publish a generative 3D shape and texture model, the Basel Face Model (BFM), and demonstrate its application to several face recognition task. We improve on previous models by offering higher shape and texture accuracy due to a better scanning device and less correspondence artifacts due to an improved registration algorithm. The same 3D face model can be fit to 2D or 3D images acquired under different situations and with different sensors using an analysis by synthesis method. The resulting model parameters separate pose, lighting, imaging and identity parameters, which facilitates invariant face recognition across sensors and data sets by comparing only the identity parameters. We hope that the availability of this registered face model will spur research in generative models. Together with the model we publish a set of detailed recognition and reconstruction results on standard databases to allow complete algorithm comparisons.

A 3D Face Model for Pose and Illumination Invariant Face Recognition

We present FaceWarehouse, a database of 3D facial expressions for visual computing applications. We use Kinect, an off-the-shelf RGBD camera, to capture 150 individuals aged 7-80 from various ethnic backgrounds. For each person, we captured the RGBD data of her different expressions, including the neutral expression and 19 other expressions such as mouth-opening, smile, kiss, etc. For every RGBD raw data record, a set of facial feature points on the color image such as eye corners, mouth contour, and the nose tip are automatically localized, and manually adjusted if better accuracy is required. We then deform a template facial mesh to fit the depth data as closely as possible while matching the feature points on the color image to their corresponding points on the mesh. Starting from these fitted face meshes, we construct a set of individual-specific expression blendshapes for each person. These meshes with consistent topology are assembled as a rank-3 tensor to build a bilinear face model with two attributes: identity and expression. Compared with previous 3D facial databases, for every person in our database, there is a much richer matching collection of expressions, enabling depiction of most human facial actions. We demonstrate the potential of FaceWarehouse for visual computing with four applications: facial image manipulation, face component transfer, real-time performance-based facial image animation, and facial animation retargeting from video to image.

/pdf/facewarehouse-a-3d-facial-expression-database-for-visual-1cff6jod66.pdf

FaceWarehouse: A 3D Facial Expression Database for Visual Computing

This paper presents a method for photo-realistic animation of any face shown in a single image or a video. The technique does not require example data of the person’s mouth movements, and the image to be animated is not restricted in pose and illumination. Video reanimation allows for head rotations and speech in the original sequence, yet neither of these motions is required. In order to animate novel faces, the system transfers mouth movements and expressions across individuals, based a common representation of different identities and facial expressions in a vector space of 3D shapes and textures. This space is computed from 3D scans of different neutral faces, and scans of facial expressions. The 3D model’s versatility with respect to pose and illumination is conveyed to photo-realistic image and video processing by a framework of analysis and synthesis algorithms: The system automatically estimates 3D shape, pose and other rendering parameters from single images, and tracks head pose and mouth movements in video. Reanimated with new mouth movements, the 3D face is rendered into the original images.

/pdf/reanimating-faces-in-images-and-video-4ote6t806d.pdf

Reanimating Faces in Images and Video

Objective. To determine the capability and reliability of dynamic contrast-enhanced MRI (DCE-MRI) in the assessment of disease activity in juvenile idiopathic arthritis (JIA). Methods. DCE-MRI of the clinically more affected wrist or hip joints was undertaken in 21 patients, coupled with standard clinical assessment and biochemical analysis. Synovial inflammation was assessed by computing the maximum level of synovial enhancement (ME), the maximum rate of enhancement (MV) and the rate of early enhancement (REE) from the enhancement curves generated from region of interest independently delineated by two readers in the area of the ME. Correlations between dynamic parameters and clinical measures of disease activity, and static MRI synovitis score were investigated. Results. In patients with wrist arthritis, REE correlated with the wrist swelling score (r s = 0.72), ESR (r s = 0.69), pain assessment scale (r s = 0.63) and childhood HAQ (r s = 0.60). In patients with hip arthritis, ME correlated with the hip limitation of motion (r s = 0.69). Static MRI synovitis score based on post-gadolinium enhancement correlated with MV (r s = 0.63) in patients with wrist arthritis and with ME (r=0.68) in those with hip arthritis. The inter-reader agreement assessed by intra-class correlation coefficient (ICC) for ME, MV and REE (ICC = 0.98, 0.97 and 0.84, respectively) was excellent. Conclusions. DCE-MRI represents a promising method for the assessment of disease activity in JIA, especially in patients with wrist arthritis. As far as we know, this study is the first to demonstrate the feasibility, reliability and construct validity of DCE-MRI in JIA. These results should be confirmed in large-scale longitudinal studies in view of its further application in therapeutic decision making and in clinical trials.

/pdf/dynamic-contrast-enhanced-magnetic-resonance-imaging-in-the-546qi2ybqd.pdf

Dynamic contrast-enhanced magnetic resonance imaging in the assessment of disease activity in patients with juvenile idiopathic arthritis

The quantitative analysis of contrast-enhanced Computed Tomography Angiography (CTA) is essential to assess aortic anatomy, identify pathologies, and perform preoperative planning in vascular surgery. To overcome the limitations given by manual and semi-automatic segmentation tools, we apply a deep learning-based pipeline to automatically segment the CTA scans of the aortic lumen, from the ascending aorta to the iliac arteries, accounting for 3D spatial coherence. A first convolutional neural network (CNN) is used to coarsely segment and locate the aorta in the whole sub-sampled CTA volume, then three single-view CNNs are used to effectively segment the aortic lumen from axial, sagittal, and coronal planes under higher resolution. Finally, the predictions of the three orthogonal networks are integrated to obtain a segmentation with spatial coherence. The coarse segmentation performed to identify the aortic lumen achieved a Dice coefficient (DSC) of 0.92 ± 0.01. Single-view axial, sagittal, and coronal CNNs provided a DSC of 0.92 ± 0.02, 0.92 ± 0.04, and 0.91 ± 0.02, respectively. Multi-view integration provided a DSC of 0.93 ± 0.02 and an average surface distance of 0.80 ± 0.26 mm on a test set of 10 CTA scans. The generation of the ground truth dataset took about 150 h and the overall training process took 18 h. In prediction phase, the adopted pipeline takes around 25 ± 1 s to get the final segmentation. The achieved results show that the proposed pipeline can effectively localize and segment the aortic lumen in subjects with aneurysm.

/pdf/3d-automatic-segmentation-of-aortic-computed-tomography-58h4wfex97.pdf

3D Automatic Segmentation of Aortic Computed Tomography Angiography Combining Multi-View 2D Convolutional Neural Networks.

Three-dimensional morphable models of objects classes are a powerful tool in modeling, animation and recognition. We introduce the new concept of regularized 3D morphable models, along with an iterative learning algorithm, by adding in the statistical model a noise/regularization term which is estimated from the examples set. With regularized 3D morphable models we are able to handle missing information, as it often occurs with data obtained by 3D acquisition systems; additionally, the new models are less complex than, but as powerful as the non-regularized ones. We present the results obtained for a set of 3D face models and a comparison with the new ones obtained by a traditional morphable model on the same data set.

Regularized 3D morphable models

The registration of 3D scans of faces is a key step for many applications, in particular for building 3D morphable models. Although a number of algorithms are already available for registering data with neutral expression, the registration of scans with arbitrary expressions is typically performed under the assumption of a known, fixed identity. We present a novel algorithm which breaks this restriction, allowing to register 3D scans of faces with arbitrary identity and expression. Furthermore, our algorithm can process incomplete data, yielding results which are both continuous and with low reconstruction error. Even in the case of complete, expression-less data, our method can yield better results than previous algorithms, due to an adaptive smoothing, which regularizes the results surface only where the estimated correspondence is unreliable.

/pdf/registration-of-expressions-data-using-a-3d-morphable-model-59dquooja3.pdf

Curzio Basso

Papers

Reanimating Faces in Images and Video

Dynamic contrast-enhanced magnetic resonance imaging in the assessment of disease activity in patients with juvenile idiopathic arthritis

3D Automatic Segmentation of Aortic Computed Tomography Angiography Combining Multi-View 2D Convolutional Neural Networks.

Regularized 3D morphable models

Registration of expressions data using a 3D morphable model