Showing papers by "Stan Z. Li published in 2007"

Learning multi-scale block local binary patterns for face recognition

Abstract: In this paper, we propose a novel representation, calledMultiscale Block Local Binary Pattern (MB-LBP), and apply it to face recognition. The Local Binary Pattern (LBP) has been proved to be effective for image representation, but it is too local to be robust. In MB-LBP, the computation is done based on average values of block subregions, instead of individual pixels. In this way, MB-LBP code presents several advantages: (1) It ismore robust than LBP; (2) it encodes not only microstructures but also macrostructures of image patterns, and hence provides a more complete image representation than the basic LBP operator; and (3) MB-LBP can be computed very efficiently using integral images. Furthermore, in order to reflect the uniform appearance of MB-LBP, we redefine the uniform patterns via statistical analysis. Finally, AdaBoost learning is applied to select most effective uniform MB-LBP features and construct face classifiers. Experiments on Face Recognition Grand Challenge (FRGC) ver2.0 database show that the proposed MB-LBP method significantly outperforms other LBP based face recognition algorithms.

Illumination Invariant Face Recognition Using Near-Infrared Images

Abstract: Most current face recognition systems are designed for indoor, cooperative-user applications. However, even in thus-constrained applications, most existing systems, academic and commercial, are compromised in accuracy by changes in environmental illumination. In this paper, we present a novel solution for illumination invariant face recognition for indoor, cooperative-user applications. First, we present an active near infrared (NIR) imaging system that is able to produce face images of good condition regardless of visible lights in the environment. Second, we show that the resulting face images encode intrinsic information of the face, subject only to a monotonic transform in the gray tone; based on this, we use local binary pattern (LBP) features to compensate for the monotonic transform, thus deriving an illumination invariant face representation. Then, we present methods for face recognition using NIR images; statistical learning algorithms are used to extract most discriminative features from a large pool of invariant LBP features and construct a highly accurate face matching engine. Finally, we present a system that is able to achieve accurate and fast face recognition in practice, in which a method is provided to deal with specular reflections of active NIR lights on eyeglasses, a critical issue in active NIR image-based face recognition. Extensive, comparative results are provided to evaluate the imaging hardware, the face and eye detection algorithms, and the face recognition algorithms and systems, with respect to various factors, including illumination, eyeglasses, time lapse, and ethnic groups

Face detection based on multi-block LBP representation

Abstract: Effective and real-time face detection has been made possible by using the method of rectangle Haar-like features with AdaBoost learning since Viola and Jones' work [12]. In this paper, we present the use of a new set of distinctive rectangle features, called Multi-block Local Binary Patterns (MB-LBP), for face detection. The MB-LBP encodes rectangular regions' intensities by local binary pattern operator, and the resulting binary patterns can describe diverse local structures of images. Based on the MB-LBP features, a boosting-based learning method is developed to achieve the goal of face detection. To deal with the non-metric feature value of MB-LBP features, the boosting algorithm uses multibranch regression tree as its weak classifiers. The experiments show the weak classifiers based on MB-LBP are more discriminative than Haar-like features and original LBP features. Given the same number of features, the proposed face detector illustrates 15% higher correct rate at a given false alarm rate of 0.001 than haar-like feature and 8% higher than original LBP feature. This indicates that MB-LBP features can capture more information about the image structure and show more distinctive performance than traditional haar-like features, which simply measure the differences between rectangles. Another advantage of MB-LBP feature is its smaller feature set, this makes much less training time.

Face matching between near infrared and visible light images

Abstract: In many applications, such as E-Passport and driver's license, the enrollment of face templates is done using visible light (VIS) face images. Such images are normally acquired in controlled environment where the lighting is approximately frontal. However, Authentication is done in variable lighting conditions. Matching of faces in VIS images taken in different lighting conditions is still a big challenge. A recent development in near infrared (NIR) image based face recognition [1] has well overcome the difficulty arising from lighting changes. However, it requires that enrollment face images be acquired using NIR as well. In this paper, we present a new problem, that of matching a face in an NIR image against one in a VIS images, and propose a solution to it. The work is aimed to develop a new solution for meeting the accuracy requirement of face-based biometric recognition, by taking advantages of the recent NIR face technology while allowing the use of existing VIS face photos as gallery templates. Face recognition is done by matching an NIR probe face against a VIS gallery face. Based on an analysis of properties of NIR and VIS face images, we propose a learning-based approach for the different modality matching. A mechanism of correlation between NIR and VIS faces is learned from NIR → VIS face pairs, and the learned correlation is used to evaluate similarity between an NIR face and a VIS face. We provide preliminary results of NIR → VIS face matching for recognition under different illumination conditions. The results demonstrate advantages of NIR → VIS matching over VIS → VIS matching.

Standardization of face image sample quality

Abstract: Performance of biometric systems is dependent on quality of acquired biometric samples. Poor sample quality is a main reason for matching errors in biometric systems and may be the main weakness of some implementations. In this paper, we present an approach for standardization of facial image quality, and develop facial symmetry based methods for its assessment by which facial asymmetries caused by non-frontal lighting and improper facial pose can be measured. Experimental results are provided to illustrate the concepts, definitions and effectiveness.

Real-time Object Classification in Video Surveillance Based on Appearance Learning

Abstract: Classifying moving objects to semantically meaningful categories is important for automatic visual surveillance. However, this is a challenging problem due to the factors related to the limited object size, large intra-class variations of objects in a same class owing to different viewing angles and lighting, and real-time performance requirement in real-world applications. This paper describes an appearance-based method to achieve real-time and robust objects classification in diverse camera viewing angles. A new descriptor, i.e., the multi-block local binary pattern (MB-LBP), is proposed to capture the large-scale structures in object appearances. Based on MB-LBP features, an adaBoost algorithm is introduced to select a subset of discriminative features as well as construct the strong two-class classifier. To deal with the non-metric feature value of MB-LBP features, a multi-branch regression tree is developed as the weak classifiers of the boosting. Finally, the error correcting output code (ECOC) is introduced to achieve robust multi-class classification performance. Experimental results show that our approach can achieve real-time and robust object classification in diverse scenes.

Face recognition with local gabor textons

Abstract: This paper proposes a novel face representation and recognition method based on local Gabor textons. Textons, defined as a vocabulary of local characteristic features, are a good description of the perceptually distinguishable micro-structures on objects. In this paper, we incorporate the advantages of Gabor feature and textons strategy together to form Gabor textons. And for the specificity of face images, we propose local Gabor textons (LGT) to portray faces more precisely and eficiently. The local Gabor textons histogram sequence is then utilized for face representation and a weighted histogram sequence matching mechanism is introduced for face recognition. Preliminary experiments on FERET database show promising results of the proposed method.

Fusion of near infrared face and Iris biometrics

Abstract: In this paper, we present a method for fusing face and iris biometrics using single near infrared (NIR) image. Fusion of NIR face and iris modalities is a natural way of doing multi-model biometrics because they can be acquired in a single image. An NIR face image is taken using a high resolution NIR camera. Face and iris are segmented from the same NIR image. Face and iris features are then extracted from the segmented parts. Matching of face and iris is done using the respective features. The matching scores are fused using various rules. Experiments give promising results.

Learning personal specific facial dynamics for face recognition from videos

Abstract: In this paper, we present an effective approach for spatiotemporal face recognition from videos using an Extended set of Volume LBP (Local Binary Pattern features) and a boosting scheme. Among the key properties of our approach are: (1) the use of local Extended Volume LBP based spatiotemporal description instead of the holistic representations commonly used in previous works; (2) the selection of only personal specific facial dynamics while discarding the intrapersonal temporal information; and (3) the incorporation of the contribution of each local spatiotemporal information. To the best of our knowledge, this is the first work addressing the issue of learning the personal specific facial dynamics for face recognition. We experimented with three different publicly available video face databases (MoBo, CRIM and Honda/UCSD) and considered five benchmark methods (PCA, LDA, LBP, HMMs and ARMA) for comparison. Our extensive experimental analysis clearly assessed the excellent performance of the proposed approach, significantly outperforming the comparative methods and thus advancing the state-of-the-art.

Face recognition by discriminant analysis with gabor tensor representation

Abstract: This paper proposes a novel face recognition method based on discriminant analysis with Gabor tensor representation. Although the Gabor face representation has achieved great success in face recognition, its huge number of features often brings about the problem of curse of dimensionality. In this paper, we propose a 3rd-order Gabor tensor representation derived from a complete response set of Gabor filters across pixel locations and filter types. 2D discriminant analysis is then applied to unfolded tensors to extract three discriminative subspaces. The dimension reduction is done in such a way that most useful information is retained. The subspaces are finally integrated for classification. Experimental results on FERET database show promising results of the proposed method.

Learning Gabor magnitude features for palmprint recognition

Abstract: Palmprint recognition, as a new branch of biometric technology, has attracted much attention in recent years. Various palmprint representations have been proposed for recognition. Gabor feature has been recognized as one of the most effective representations for palmprint recognition, where Gabor phase and orientation feature representations are extensively studied. In this paper, we explore a novel Gabor magnitude feature-based method for palmprint recognition. The novelties are as follows: First, we propose an illumination normalization method for palmprint images to decrease the influence of illumination variations caused by different sensors and lighting conditions. Second, we propose to use Gabor magnitude features for palmprint representation. Third, we utilize AdaBoost learning to extract most effective features and apply Local Discriminant Analysis (LDA) to reduce the dimension further for palmprint recognition. Experimental results on three large palmprint databases demonstrate the effectiveness of proposed method. Compared with state-of-the-art Gabor-based methods, our method achieves higher accuracy.

Part-based Face Recognition Using Near Infrared Images

Abstract: Recently, the authors developed NIR based face recognition for highly accurate face recognition under illumination variations. In this paper, we present a part-based method for improving its robustness with respect to pose variations. An NIR face is decomposed into parts. A part classifier is built for each part, using the most discriminative LBP histogram features selected by AdaBoost learning. The outputs of part classifiers are fused to give the final score. Experiments show that the present method outperforms the whole face-based method by 4.53%.

Fusion of Face and Palmprint for Personal Identification Based on Ordinal Features

Abstract: In this paper, we present a face and palmprint multimodal biometric identification method and system to improve the identification performance. Effective classifiers based on ordinal features are constructed for faces and palmprints, respectively. Then, the matching scores from the two classifiers are combined using several fusion strategies. Experimental results on a middle-scale data set have demonstrated the effectiveness of the proposed system.

Outdoor face recognition using enhanced near infrared imaging

Abstract: In this paper, we present a robust and accurate system for outdoor (as well as indoor) face recognition, based on a recently developed enhanced near-infrared (ENIR) imaging device. Using a narrow band NIR laser generator instead of LED lights for active frontal illumination, the ENIR device can provide face images of good quality even under sunlight. Experiments show that the ENIR system performs similarly to the existing NIR system when used indoors, but outperforms it significantly outdoors especially under sunlight.

On Constrained Sparse Matrix Factorization

Abstract: Various linear subspace methods can be formulated in the notion of matrix factorization in which a cost function is minimized subject to some constraints. Among them, constraints on sparseness have received much attention recently. Some popular constraints such as non-negativity, lasso penalty, and (plain) orthogonality etc have been so far applied to extract sparse features. However, little work has been done to give theoretical and experimental analyses on the differences of the impacts of different constraints within a framework. In this paper, we analyze the problem in a more general framework called Constrained Sparse Matrix Factorization (CSMF). In CSMF, a particular case called CSMF with non-negative components (CSMFnc) is further discussed. Unlike NMF, CSMFnc allows not only additive but also subtractive combinations of non-negative sparse components. It is useful to produce much sparser features than those produced by NMF and meanwhile have better reconstruction ability, achieving a trade-off between sparseness and low MSE value. Moreover, for optimization, an alternating algorithm is developed and a gentle update strategy is further proposed for handling the alternating process. Experimental analyses are performed on the Swimmer data set and CBCLface database. In particular, CSMF can successfully extract all the proper components without any ghost on Swimmer, gaining a significant improvement over the compared well-known algorithms.

Using Near-Infrared Images

Abstract: Most current face recognition systems are designed for indoor, cooperative-user applications. However, even in thus- constrained applications, most existing systems, academic and commercial, are compromised in accuracy by changes in environmental illumination. In this paper, we present a novel solution for illumination invariant face recognition for indoor, cooperative-user applications. First, we present an active near infrared (NIR) imaging system that is able to produce face images of good condition regardless of visible lights in the environment. Second, we show that the resulting face images encode intrinsic information of the face, subject only to a monotonic transform in the gray tone; based on this, we use local binary pattern (LBP) features to compensate for the monotonic transform, thus deriving an illumination invariant face representation. Then, we present methods for face recognition using NIR images; statistical learning algorithms are used to extract most discriminative features from a large pool of invariant LBP features and construct a highly accurate face matching engine. Finally, we present a system that is able to achieve accurate and fast face recognition in practice, in which a method is provided to deal with specular reflections of active NIR lights on eyeglasses, a critical issue in active NIR image- based face recognition. Extensive, comparative results are provided to evaluate the imaging hardware, the face and eye detection algorithms, and the face recognition algorithms and systems, with respect to various factors, including illumination, eyeglasses, time lapse, and ethnic groups. Index Terms—Biometrics, face recognition, near infrared (NIR), illumination invariant, local binary pattern (LBP), statistical learning.

Half Quadratic Analysis for Mean Shift: with Extension to A Sequential Data Mode-Seeking Method

Abstract: Theoretical understanding and extension of mean shift procedure has received much attention recently. In this paper, we present a theoretical exploration and an algorithm development on mean shift. In the theory part, we point out that convex profile based mean shift can be justified from the viewpoint of half-quadratic (HQ) optimization. Such analysis facilitates the convergence study and uni-mode bandwidth selection for the latest variation, annealed mean shift. In the algorithm development part of this paper, we extend annealed mean shift inside our HQ framework to a novel method, namely adaptive mean shift (Ada-MS), to detect multiple data modes sequentially from an arbitrary starting point in linear running time. To validate the performance, we couple the investigation with two applications: image segmentation and color constancy. Extensive experiments show that the proposed method is time efficient and initialization invariant.

Tracking and recognition of multiple faces at distances

Abstract: Many applications require tracking and recognition of multiple faces at distances, such as in video surveillance. Such a task, dealing with noncooperative objects is more challenging than handling a single face and than tackling a cooperative user. The difficulties include mutual occlusions of multiple faces and arbitrary head poses. In this paper, we present a method for solving the problems and a real-time system implementation. An appearance model updating mechanism is developed via Gaussian Mixture Models to deal with tracking under head rotation and mutual occlusion. Face recognition based on video sequence is then performed to get the identity information. Through fusing the tracking and recognition information, the performance of them are both improved. A real-time system for multi-face tracking and recognition at distances is presented. The system can track multiple faces under head rotations, and deal with total occlusion effectively regardless of the motion trajectory. It is also able to recognize multi-persons simultaneously. Experimental results demonstrate promising performance of the system.

Structured ordinal features for appearance-based object representation

Abstract: In this paper, we propose a novel appearance-based representation, called Structured Ordinal Feature (SOF). SOF is a binary string encoded by combining eight ordinal blocks in a circle symmetrically. SOF is invariant to linear transformations on images and is flexible enough to represent different local structures of different complexity. We further extend SOF to Multi-scale Structured Ordinal Feature (MSOF) by concatenating binary strings of multi-scale SOFs at a fix position. In this way, MSOF encodes not only microstructure but also macrostructure of image patterns, thus provides a more powerful image representation. We also present an efficient algorithm for computing MSOF using integral images. Based on MSOF, statistical analysis and learning are performed to select most effective features and construct classifiers. The proposed method is evaluated with face recognition experiments, in which we achieve a high rank-1 recognition rate of 98.24% on FERET database.

Feature correlation filter for face recognition

Abstract: The correlation filters for pattern recognition, have been extensively studied in the areas of automatic target recognition(ATR) and biometrics. Whereas the conventional correlation filters perform directly on image pixels, in this paper, we propose a novel method, called "feature correlation filter (FCF)", by extending the concept of correlation filter to feature spaces. The FCF preserves the benefits of conventional correlation filters, i.e., shift-invariant, occlusion-insensitive, and closed-form solution, and also inherits virtues of the feature representations. Moreover, since the size of feature is often much smaller than the size of image, the FCF method can significantly reduce the storage requirement in recognition system. The comparative results on CMU-PIE and the FRGC2.0 database show that the proposed FCFs can achieve noteworthy performance improvement compared with their conventional counterpart.

Automated Feature Points Management for Video Mosaic Construction

Abstract: This paper presents a two level automatic feature points management method for constructing a seamless entire panorama from video sequence. In the first level, through fusing the number of tracked feature points and the estimated ratio of lost information of the mosaicing image, a feature point quantity management module is developed to select the key frames. In the second level, a feature points quality management technique is used to choose the key points for mosaicing. This module includes a coarse-to-fine method with two steps: (1) Feature points quality based key point subset creation; and (2) Multi-resolution based key point selection. The main contribution of the algorithm is that it is able to achieve robust and fast mosaicing result while maintain the most valuable information of the scene. Experiments are performed using video sequences under different conditions. The results show that the proposed algorithm could achieve robust and efficient video mosaic image.

Coarse-to-fine statistical shape model by Bayesian inference

Abstract: In this paper, we take a predefined geometry shape as a constraint for accurate shape alignment. A shape model is divided in two parts: fixed shape and active shape. The fixed shape is a user-predefined simple shape with only a few landmarks which can be easily and accurately located by machine or human. The active one is composed of many landmarks with complex shape contour. When searching an active shape, pose parameter is calculated by the fixed shape. Bayesian inference is introduced to make the whole shape more robust to local noise generated by the active shape, which leads to a compensation factor and a smooth factor for a coarse-to-fine shape search. This method provides a simple and stable means for online and offline shape analysis. Experiments on cheek and face contour demonstrate the effectiveness of our proposed approach.

Color constancy via convex kernel optimization

Abstract: This paper introduces a novel convex kernel based method for color constancy computation with explicit illuminant parameter estimation. A simple linear render model is adopted and the illuminants in a new scene that contains some of the color surfaces seen in the training image are sequentially estimated in a global optimization framework. The proposed method is fully data-driven and initialization invariant. Nonlinear color constancy can also be approximately solved in this kernel optimization framework with piecewise linear assumption. Extensive experiments on real-scene images validate the practical performance of our method.

Shape and Texture Based Deformable Models for Facial Image Analysis

Abstract: In this chapter, we introduce concepts and algorithms of shape and texture based deformable models, more speciflcally Active Shape Models (ASM), Active Appearance models (AAM) and Morphable Models, for facial image analysis. Such models, learned from training ex- amples, allow admissible deformations under statistical constraints on the shape and/or texture of the pattern of interests. As such, the de- formation is in accordance with the speciflc constraints on the pattern. Based on analysis of problems with the standard ASM and AAM, we further describe enhanced models and algorithms, namely Direct Ap- pearance Models (DAM) and Texture Constrained ASM (TC-ASM), for

Acquisition of Face Depth Information from Near Infrared Images

Abstract: This paper proposes a method for acquiring face depth information directly from near infrared (NIR) images, using statistical learning. To perform such learning, ground truth NIR images and range data are captured. A method of alignment between the two image modalities is proposed. By constructing the low dimensional face subspaces of NIR images and depth maps, the raw data are projected into respective subspaces. The mapping between the two subspaces is learned. The experiment substantiates the accuracy of the depth recovered and the economy of time and memory consumed.

Robust unsupervised clustering using generalized annealing m-estimator

Abstract: A new robust clustering algorithm, called generalized annealing M-estimator (GAM-estimator), is proposed. Initialized with multiple seeds, the GAM-estimator converges to several optimal cluster centers. Neither knowledge about the number of clusters nor scale is needed. The global optimal solution of clustering is achieved by minimization of an objective function. The algorithm is applied to unsupervised texture segmentation and texture-based defect detection .