Showing papers on "Active appearance model published in 2002"

Extraction of visual features for lipreading

Abstract: The multimodal nature of speech is often ignored in human-computer interaction, but lip deformations and other body motion, such as those of the head, convey additional information. We integrate speech cues from many sources and this improves intelligibility, especially when the acoustic signal is degraded. The paper shows how this additional, often complementary, visual speech information can be used for speech recognition. Three methods for parameterizing lip image sequences for recognition using hidden Markov models are compared. Two of these are top-down approaches that fit a model of the inner and outer lip contours and derive lipreading features from a principal component analysis of shape or shape and appearance, respectively. The third, bottom-up, method uses a nonlinear scale-space analysis to form features directly from the pixel intensity. All methods are compared on a multitalker visual speech recognition task of isolated letters.

3-D active appearance models: segmentation of cardiac MR and ultrasound images

Abstract: A model-based method for three-dimensional image segmentation was developed and its performance assessed in segmentation of volumetric cardiac magnetic resonance (MR) images and echocardiographic temporal image sequences. Comprehensive design of a three-dimensional (3-D) active appearance model (AAM) is reported for the first time as an involved extension of the AAM framework introduced by Cootes et al. The model's behavior is learned from manually traced segmentation examples during an automated training stage. Information about shape and image appearance of the cardiac structures is contained in a single model. This ensures a spatially and/or temporally consistent segmentation of three-dimensional cardiac images. The clinical potential of the 3-D AAM is demonstrated in short-axis cardiac MR images and four-chamber echocardiographic sequences. The method's performance was assessed by comparison with manually identified independent standards in 56 clinical MR and 64 clinical echo image sequences. The AAM method showed good agreement with the independent standard using quantitative indexes of border positioning errors, endo- and epicardial volumes, and left ventricular mass. In MR, the endocardial volumes, epicardial volumes, and left ventricular wall mass correlation coefficients between manual and AAM were R/sup 2/=0.94,0.97,0.82, respectively. For echocardiographic analysis, the area correlation was R/sup 2/=0.79. The AAM method shows high promise for successful application to MR and echocardiographic image analysis in a clinical setting.

Automatic segmentation of echocardiographic sequences by active appearance motion models

Abstract: A novel extension of active appearance models (AAMs) for automated border detection in echocardiographic image sequences is reported. The active appearance motion model (AAMM) technique allows fully automated robust and time-continuous delineation of left ventricular (LV) endocardial contours over the full heart cycle with good results. Nonlinear intensity normalization was developed and employed to accommodate ultrasound-specific intensity distributions. The method was trained and tested on 16-frame phase-normalized transthoracic four-chamber sequences of 129 unselected infarct patients, split randomly into a training set (n=65) and a test set (n=64). Borders were compared to expert drawn endocardial contours. On the test set, fully automated AAMM performed well in 97% of the cases (average distance between manual and automatic landmark points was 3.3 mm, comparable to human interobserver variabilities). The ultrasound-specific intensity normalization proved to be of great value for good results in echocardiograms. The AAMM was significantly more accurate than an equivalent set of two-dimensional AAMs.

Face Recognition from Long-Term Observations

Abstract: We address the problem of face recognition from a large set of images obtained over time - a task arising in many surveillance and authentication applications. A set or a sequence of images provides information about the variability in the appearance of the face which can be used for more robust recognition. We discuss different approaches to the use of this information, and show that when cast as a statistical hypothesis testing problem, the classification task leads naturally to an information-theoretic algorithm that classifies sets of images using the relative entropy (Kullback-Leibler divergence) between the estimated density of the input set and that of stored collections of images for each class. We demonstrate the performance of the proposed algorithm on two medium-sized data sets of approximately frontal face images, and describe an application of the method as part of a view-independent recognition system.

Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions

Abstract: This paper presents a novel algorithm aiming at analysis and identification of faces viewed from different poses and illumination conditions. Face analysis from a single image is performed by recovering the shape and textures parameters of a 3D Morphable Model in an analysis-by-synthesis fashion. The shape parameters are computed from a shape error estimated by optical flow and the texture parameters are obtained from a texture error. The algorithm uses linear equations to recover the shape and texture parameters irrespective of pose and lighting conditions of the face image. Identification experiments are reported on more than 5000 images from the publicly available CMU-PIE database which includes faces viewed from 13 different poses and under 22 different illuminations. Extensive identification results are available on our web page for future comparison with novel algorithms.

Real-time eye detection and tracking under various light conditions

Abstract: Non-intrusive methods based on active remote IR illumination for eye tracking are important for many applications of vision-based man-machine interaction. One problem that has plagued those methods is their sensitivity to lighting condition change. This tends to significantly limit their scope of application. In this paper, we present a new real-time eye detection and tracking methodology that works under variable and realistic lighting conditions. Based on combining the bright-pupil effect resulted from IR light and the conventional appearance-based object recognition technique, our method can robustly track eyes when the pupils are not very bright due to significant external illumination interferences. The appearance model is incorporated in both eyes detection and tracking via the use of support vector machine and the mean shift tracking. Additional improvement is achieved from modifying the image acquisition apparatus including the illuminator and the camera.

Comparing Variations on the Active Appearance Model Algorithm

Abstract: The Active Appearance Model (AAM) algorithm has proved to be a successful method for matching statistical models of appearance to new images. Since the original algorithm was described there have been a variety of suggested modifications to the basic algorithm, each typically claiming to be in some way superior. We review these algorithms and report the results of experiments comparing their performance. We also investigate the effects of different methods of estimating the update matrix used in the algorithm. We find that careful choice of the latter has at least as much effect as the choice of updating technique.

Eye typing using Markov and active appearance models

Abstract: We propose a non-intrusive eye tracking system intended for the use of everyday gaze typing using web cameras. We argue that high precision in gaze tracking is not needed for on-screen typing due to natural language redundancy. This facilitates the use of low-cost video components for advanced multi-modal interactions based on video tracking systems. Robust methods are needed to track the eyes using web cameras due to the poor image quality. A realtime tracking scheme using a mean-shift color tracker and an Active Appearance Model of the eye is proposed. It is possible from this model to infer the state of the eye such as eye corners and the pupil location under scale and rotational changes.

An active model for facial feature tracking

Abstract: We present a system for finding and tracking a face and extract global and local animation parameters from a video sequence. The system uses an initial colour processing step for finding a rough estimate of the position, size, and inplane rotation of the face, followed by a refinement step drived by an active model. The latter step refines the previous estimate, and also extracts local animation parameters. The system is able to track the face and some facial features in near real-time, and can compress the result to a bitstream compliant to MPEG-4 face and body animation.

Visual motion analysis method for detecting arbitrary numbers of moving objects in image sequences

Abstract: A visual motion analysis method that uses multiple layered global motion models to both detect and reliably track an arbitrary number of moving objects appearing in image sequences Each global model includes a background layer and one or more foreground “polybones”, each foreground polybone including a parametric shape model, an appearance model, and a motion model describing an associated moving object Each polybone includes an exclusive spatial support region and a probabilistic boundary region, and is assigned an explicit depth ordering Multiple global models having different numbers of layers, depth orderings, motions, etc, corresponding to detected objects are generated, refined using, for example, an EM algorithm, and then ranked/compared Initial guesses for the model parameters are drawn from a proposal distribution over the set of potential (likely) models Bayesian model selection is used to compare/rank the different models, and models having relatively high posterior probability are retained for subsequent analysis

CMC 2000 Chromatic Adaptation Transform: CMCCAT2000

Abstract: CMCCAT97 is a chromatic adaptation transform included in CIECAM97s, the CIE 1997 colour appearance model, for describing colour appearance under different viewing conditions and is recommended by the Colour Measurement Committee of the Society of Dyers and Colourists for predicting the degree of colour inconstancy of surface colours. Among the many transforms tested, this transform gave the most accurate predictions to a number of experimental data sets. However, the structure of CMCCAT97 is considered complicated and causes problems when applications require the use of its reverse mode. This article describes a simplified version of CMCCAT97— CMCCAT2000—which not only is significantly simpler and eliminates the problems of reversibility, but also gives a more accurate prediction to almost all experimental data sets than does the original transform. © 2002 John Wiley & Sons, Inc. Col Res Appl, 27, 49–58, 2002

Meet iCAM: A Next-Generation Color Appearance Model

Abstract: For over 20 years, color appearance models have evolved to the point of international standardization These models are capable of predicting the appearance of spatially-simple color stimuli under a wide variety viewing conditions and have been applied to images by treating each pixel as an independent stimulus It has been more recently recognized that revolutionary advances in color appearance modeling would require more rigorous treatment of spatial (and perhaps temporal) appearance phenomena In addition, color appearance models are often more complex than warranted by the available visual data and limitations in the accuracy and precision of practical viewing conditions Lastly, issues of color difference measurement are typically treated separate from color appearance Thus, the stage has been set for a new generation of color appearance models This paper presents one such model called iCAM, for image color appearance model The objectives in formulating iCAM were to simultaneously provide traditional color appearance capabilities, spatial vision attributes, and color difference metrics, in a model simple enough for practical applications The framework and initial implementation of the model are presented along with examples that illustrate its performance for chromatic adaptation, appearance scales, color difference, crispening, spreading, high-dynamic-range tone mapping, and image quality measurement It is expected that the implementation of this model framework will be refined in the coming years as new data become available

A Statistical Shape Model for the Liver

Abstract: The use of statistical shape models is a promising approach for robust segmentation of medical images. One of the major challenges in building a 3D shape model from a training set of segmented instances of an object is the determination of the correspondence between them. We propose a novel geometric approach that is based on minimizing the distortion of the mapping between two surfaces. In this work we investigate the accuracy and completeness of a 3D statistical shape model for the liver built from 20 manually segmented individual CT data sets. The quality of the shape model is crucial for its application as a segmentation tool.

Generic model abstraction from examples

Abstract: The recognition community has long avoided bridging the representational gap between traditional, low-level image features and generic models. Instead, the gap has been eliminated by either bringing the image closer to the models, using simple scenes containing idealized, textureless objects, or by bringing the models closer to the images, using 3-D CAD model templates or 2-D appearance model templates. In this paper, we attempt to bridge the representational gap for the domain of model acquisition. Specifically, we address the problem of automatically acquiring a generic 2-D view-based class model from a set of images, each containing an exemplar object belonging to that class. We introduce a novel graph-theoretical formulation of the problem, and demonstrate the approach on real imagery.

Multi-view face alignment using direct appearance models

Abstract: Alignment makes face distribution statistically more compact than un-aligned faces and provides a good basis for face modeling, recognition and synthesis. In this paper we present a method for multi-view face alignment using a new model called direct appearance model (DAM). Like active appearance model (AAM), DAM also makes use of both shape and texture constraints; however it does this without combining shape and texture as in AAM. The way that DAM models shapes and textures has the following advantages as compared to AAM: (1) DAM subspaces include admissible appearances previously unseen in AAM, (2) the convergence and accuracy are improved, and (3) the memory requirement is cut down to a large extent. Extensive experiments are presented to evaluate the DAM alignment in comparison with AAM.

Analysis and Segmentation of Face Images using Point Annotations and Linear Subspace Techniques

Abstract: Abstract This report provides an analysis of 37 annotated frontal face images. All results presented have been obtained using our freely available Active Appearance Model (AAM) implementation. To ensure the reproducibility of the presented experiments, the data set has also been made available. As such, the data and this report may serve as a point of reference to compare other AAM implementations against. In addition, we address the problem of AAM model truncation using parallel analysis along with a comparable study of the two prevalent AAM learning methods; principal component regression and estimation of fixed Jacobian matrices. To assess applicability and efficiency, timings for model building, warping and optimisation are given together with a description of how to exploit the warping capabilities of contemporary consumer-level graphics hardware.

Learning surveillance tracking models for the self-calibrated ground plane

Abstract: Tracking strategies usually employ motion and appearance models to locate observations of the tracked object in successive frames. The subsequent model update procedure renders the approach highly sensitive to the inevitable observation and occlusion noise processes. In this work, two robust mechanisms are proposed which rely on knowledge about the ground plane. First a highly constrained bounding box appearance model is proposed which is determined solely from predicted image location and visual motion. Second, tracking is performed on the ground plane enabling global real-world observation and dynamic noise models to be defined. Finally, a novelauto-calibrationprocedureis developedto recoverthe imageto ground plane homographyby simply accumulating event observations.

Integrated Approach for Matching Statistical Shape Models with Intra-operative 2D and 3D Data

Abstract: This paper presents an approach to the problem of intraoperative reconstruction of 3D anatomical surfaces. The method is based on the integration of intra-operatively available shape and image data of different dimensionality such as 3D scattered point data, 2.5D ultra sound data, X-ray images etc. by matching them to a statistical shape model, thus providing the surgeon with a complete surface representation of the object of interest. Previous papers of the authors describe the matching of either 3D or 2D data to a statistical model and clinical applications. The here presented work combines former published ideas with a new approach for the complex task of shape analysis required for the computation of the statistical model, thus providing a generic approach for intra-operative surface reconstruction based on statistical models. The method for shape extraction/analysis is based on a generic model of the object and is used to segment training shapes and to establish point to point correspondence simultaneously in a set of CT images. Reconstruction experiments are performed on a statistical model of lumbar vertebrae. Results are provided for 3D/3D, 2D/3D and hybrid matching with simulated data and for 3D/2D matching for a cadaveric spine.

Diaphragm dome surface segmentation in CT data sets: a 3D active appearance model approach

Abstract: Knowledge about the location of the diaphragm dome surface, which separates the lungs and the heart from the abdominal cavity, is of vital importance for applications like automated segmentation of adjacent organs (e.g., liver) or functional analysis of the respiratory cycle. We present a new 3D Active Appearance Model (AAM) approach to segmentation of the top layer of the diaphragm dome. The 3D AAM consists of three parts: a 2D closed curve (reference curve), an elevation image and texture layers. The first two parts combined represent 3D shape information and the third part image intensity of the diaphragm dome and the surrounding layers. Differences in height between dome voxels and a reference plane are stored in the elevation image. The reference curve is generated by a parallel projection of the diaphragm dome outline in the axial direction. Landmark point placement is only done on the (2D) reference curve, which can be seen as the bounding curve of the elevation image. Matching is based on a gradient-descent optimization process and uses image intensity appearance around the actual dome shape. Results achieved in 60 computer generated phantom data sets show a high degree of accuracy (positioning error -0.07+/-1.29 mm). Validation using real CT data sets yielded a positioning error of -0.16+/-2.95 mm. Additional training and testing on in-vivo CT image data is ongoing.

Robust Parameterized Component Analysis

Abstract: Principal Component Analysis (PCA) has been successfully applied to construct linear models of shape, graylevel, and motion. In particular, PCA has been widely used to model the variation in the appearance of people's faces. We extend previous work on facial modeling for tracking faces in video sequences as they undergo significant changes due to facial expressions. Here we develop person-specific facial appearance models (PSFAM), which use modular PCA to model complex intra-person appearance changes. Such models require aligned visual training data; in previous work, this has involved a time consuming and errorprone hand alignment and cropping process. Instead, we introduce parameterized component analysis to learn a subspace that is invariant to affine (or higher order) geometric transformations. The automatic learning of a PSFAM given a training image sequence is posed as a continuous optimization problem and is solved with a mixture of stochastic and deterministic techniques achieving sub-pixel accuracy. We illustrate the use of the 2D PSFAM model with several applications including video-conferencing, realistic avatar animation and eye tracking.

Vision models-based identification of traffic signs.

Abstract: During the last 10 years,computer hardware technology has been improved rapidly.Large memory,storage is no longer a problem.Therefore some trade-off (dirty and quick algorithms)for traffic sign recognition between accuracy and speed should be improved.In this study,a new approach has been developed for accurate and fast recognition of traffic signs based on human vision models.It applies colour appearance model CIECAM97s to segment traffic signs from the rest of scenes.A Behavioural Model of Vision (BMV)is then utilised to identify the signs after segmented images are converted into grey-level representation.Two standard traffic sign databases are established.One is British traffic signs and the other is Russian traffic signs.Preliminary results show that around 90%signs taken from the British road with various viewing conditions have been correctly identified.

Automatic landmarking for building biological shape models

Abstract: We present a new method for automatic landmark extraction from the contours of biological specimens. Our ultimate goal is to enable automatic identification of biological specimens in photographs and drawings held in a database. We propose to use active appearance models for visual indexing of both photographs and drawings. Automatic landmark extraction will assist us in building the models. We describe the results of using our method on drawings and photographs of examples of diatoms, and present an active shape model built using automatically extracted data.

Hands-on experience with active appearance models

Abstract: The aim of this work is to explore the performance of Active Appearance Models (AAMs) in reconstruction and interpretation of bones in hand radiographs. AAM is a generative approach that unifies image segmentation and image understanding. Initial locations for the AAM search are generated by an exhaustive filtering method. A series of AAMs for smaller groups of bones are used. It is found that AAM successful reconstructs 99% of metacarpals, proximal and medial phalanges and the distal 3 cm of radius and ulna. The rms accuracy is better than 240 microns (point-to-curve). The generative property is used (1) to define a measure of fit that allows the models to self-evaluate and chose between the multiple found solutions, (2) to overcome obstacles in the image in the form of rings by predicting the missing part, and (3) to detect anomalies, e.g. rheumatoid arthritis. The shape scores are used as a biometrics to check the identity of patients in a longitudinal study. The conclusion is that AAM provides a highly efficient and unified framework for various tasks in diagnosis and assessment of bone related disorders.

Maximum a posteriori estimation of image boundaries by dynamic programming

Abstract: Summary. We seek a computationally fast method for solving a difficult image segmentation problem: the positioning of boundaries on medical scanner images to delineate tissues of interest. We formulate a Bayesian model for image boundaries such that the maximum a posteriori estimator is obtainable very efficiently by dynamic programming. The prior model for the boundary is a biased random walk and the likelihood is based on a border appearance model, with parameter values obtained from training images. The method is applied successfully to the segmentation of ultrasound images and X-ray computed tomographs of sheep, for application in sheep breeding programmes.

Video Object Tracking Based On Extended Active Shape Models With Color Information

Abstract: Tracking and recognizing non-rigid objects in video image sequences are complex tasks of increa sing importance to many applications. For the tracking of such objects in a video sequence e.g. "active shape models" can be applied. The existing active shape models are usually based on intensity information and they do not consider color information. However, active shape models are sensitive to outliers, especially in the case of partial object occlusions. In this paper, we present an extension of the active shape model for color images and we examine to what extent the use of color information can contribute to the solution of the outlier problem.

Segmentation of cardiac MR volume data using 3D active appearance models

Abstract: Active Appearance Models (AAMs) are useful for the segmentation of cardiac MR images since they exploit prior knowledge about the cardiac shape and image appearance. However, traditional AAMs only process 2D images, not taking into account the 3D data inherent to MR. This paper presents a novel, true 3D Active Appearance Model that models the intrinsic 3D shape and image appearance of the left ventricle in cardiac MR data. In 3D-AAM, shape and appearance of the Left Ventricle (LV) is modeled from a set of expert drawn contours. The contours are then resampled to a manually defined set of landmark points, and subsequently aligned. Appearance variations in both shape and texture are captured using Principal Component Analysis (PCA) on the training set. Segmentation is achieved by minimizing the model appearance-to-target differences by adjusting the model eigen-coefficients using a gradient descent approach. The clinical potential of the 3D-AAM is demonstrated in short-axis cardiac magnetic resonance (MR) images. The method's performance was assessed by comparison with manually-identified independent standards in 56 clinical MR sequences. The method showed good agreement with the independent standards using quantitative indices such as border positioning errors, endo- and epicardial volumes, and left ventricular mass. The 3D AAM method shows high promise for successful segmentation of three-dimensional images in MR.