Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition
Seong G. Kong,Jingu Heo,Faysal Boughorbel,Yue Zheng,Besma Abidi,Andreas Koschan,Mingzhong Yi,Mongi A. Abidi +7 more
TLDR
In this paper, an ellipse fitting method was used to detect eyeglass regions and replaced with eye template patterns to preserve the details useful for face recognition in the fused image.Abstract:
This paper describes a new software-based registration and fusion of visible and thermal infrared (IR) image data for face recognition in challenging operating environments that involve illumination variations. The combined use of visible and thermal IR imaging sensors offers a viable means for improving the performance of face recognition techniques based on a single imaging modality. Despite successes in indoor access control applications, imaging in the visible spectrum demonstrates difficulties in recognizing the faces in varying illumination conditions. Thermal IR sensors measure energy radiations from the object, which is less sensitive to illumination changes, and are even operable in darkness. However, thermal images do not provide high-resolution data. Data fusion of visible and thermal images can produce face images robust to illumination variations. However, thermal face images with eyeglasses may fail to provide useful information around the eyes since glass blocks a large portion of thermal energy. In this paper, eyeglass regions are detected using an ellipse fitting method, and replaced with eye template patterns to preserve the details useful for face recognition in the fused image. Software registration of images replaces a special-purpose imaging sensor assembly and produces co-registered image pairs at a reasonable cost for large-scale deployment. Face recognition techniques using visible, thermal IR, and data-fused visible-thermal images are compared using a commercial face recognition software (FaceIt®) and two visible-thermal face image databases (the NIST/Equinox and the UTK-IRIS databases). The proposed multiscale data-fusion technique improved the recognition accuracy under a wide range of illumination changes. Experimental results showed that the eyeglass replacement increased the number of correct first match subjects by 85% (NIST/Equinox) and 67% (UTK-IRIS).read more
Citations
More filters
Journal ArticleDOI
Infrared and visible image fusion methods and applications: A survey
TL;DR: This survey comprehensively survey the existing methods and applications for the fusion of infrared and visible images, which can serve as a reference for researchers inrared and visible image fusion and related fields.
Journal ArticleDOI
Infrared and visible image fusion via gradient transfer and total variation minimization
TL;DR: A novel fusion algorithm, named Gradient Transfer Fusion (GTF), based on gradient transfer and total variation (TV) minimization is proposed, which can keep both the thermal radiation and the appearance information in the source images.
Book
Template Matching Techniques in Computer Vision: Theory and Practice
TL;DR: This book and the accompanying website, focus on template matching, a subset of object recognition techniques of wide applicability, which has proved to be particularly effective for face recognition applications.
Journal ArticleDOI
Thermal cameras and applications: a survey
Rikke Gade,Thomas B. Moeslund +1 more
TL;DR: An overview of the current applications of thermal cameras is provided, and the nature of thermal radiation and the technology of thermal camera are described.
Proceedings ArticleDOI
Attribute-based people search in surveillance environments
TL;DR: A novel framework for searching for people in surveillance environments based on a parsing of human parts and their attributes, including facial hair, eyewear, clothing color, etc, which can be extracted using detectors learned from large amounts of training data is proposed.
References
More filters
Journal ArticleDOI
A method for registration of 3-D shapes
Paul J. Besl,H.D. McKay +1 more
TL;DR: In this paper, the authors describe a general-purpose representation-independent method for the accurate and computationally efficient registration of 3D shapes including free-form curves and surfaces, based on the iterative closest point (ICP) algorithm, which requires only a procedure to find the closest point on a geometric entity to a given point.
Book
Ten lectures on wavelets
TL;DR: This paper presents a meta-analyses of the wavelet transforms of Coxeter’s inequality and its applications to multiresolutional analysis and orthonormal bases.
Journal ArticleDOI
Eigenfaces for recognition
Matthew Turk,Alex Pentland +1 more
TL;DR: A near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals, and that is easy to implement using a neural network architecture.
Journal ArticleDOI
Ten Lectures on Wavelets
TL;DR: In this article, the regularity of compactly supported wavelets and symmetry of wavelet bases are discussed. But the authors focus on the orthonormal bases of wavelets, rather than the continuous wavelet transform.
Book
Numerical Recipes in C: The Art of Scientific Computing
TL;DR: Numerical Recipes: The Art of Scientific Computing as discussed by the authors is a complete text and reference book on scientific computing with over 100 new routines (now well over 300 in all), plus upgraded versions of many of the original routines, with many new topics presented at the same accessible level.