“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

A survey on deep learning based face recognition

Recently, near-infrared (NIR) images are increasingly being captured for recognizing faces in low-light/night-time conditions Matching these images against the controlled high-resolution visible facial images usually present in the database is a challenging task In surveillance scenarios, the NIR images can have very low resolution and also non-frontal pose which makes the problem even more challenging In this paper, we propose an orthogonal dictionary alignment approach for addressing this problem We also propose a re-ranking approach to further improve the recognition performance for each probe by combining the rank list given by the proposed algorithm with that given by another complementary feature/algorithm Finally, we have also collected our own database Heterogeneous face recognition across Pose and Resolution (HPR) that has facial images captured from two surveillance quality NIR cameras and one high-resolution visible camera, with significant variations in head pose and resolution Extensive experiments on the modified CASIA NIR-VIS 20 database, the Surveillance Camera face database, and our HPR database show the effectiveness of the proposed approaches and the collected database

Dictionary Alignment With Re-Ranking for Low-Resolution NIR-VIS Face Recognition

In recent years, researchers have proposed many deep learning (DL) methods for various tasks, and particularly face recognition (FR) made an enormous leap using these techniques. Deep FR systems benefit from the hierarchical architecture of the DL methods to learn discriminative face representation. Therefore, DL techniques significantly improve state-of-the-art performance on FR systems and encourage diverse and efficient real-world applications. In this paper, we present a comprehensive analysis of various FR systems that leverage the different types of DL techniques, and for the study, we summarize 171 recent contributions from this area. We discuss the papers related to different algorithms, architectures, loss functions, activation functions, datasets, challenges, improvement ideas, current and future trends of DL-based FR systems. We provide a detailed discussion of various DL methods to understand the current state-of-the-art, and then we discuss various activation and loss functions for the methods. Additionally, we summarize different datasets used widely for FR tasks and discuss challenges related to illumination, expression, pose variations, and occlusion. Finally, we discuss improvement ideas, current and future trends of FR tasks.

Recent Advances in Deep Learning Techniques for Face Recognition

Face re-identification challenge: Are face recognition models good enough?

Road safety is a major public health issue across the globe and over two-thirds of the road accidents occur at nighttime under low-light conditions or darkness. The state of the driver and her/his actions are the key factors impacting road safety. How can we monitor these in a cost-effective manner and in low-light conditions? RGB cameras present in smartphones perform poorly in low-lighting conditions due to lack of information captured. Hence, existing monitoring solutions rely upon specialized hardware such as infrared cameras or thermal cameras in low-light conditions, but are limited to only high-end vehicles owing to the cost of the hardware. We present InSight, a windshield-mounted smartphone-based system that can be retrofitted to the vehicle to monitor the state of the driver, specifically driver fatigue (based on frequent yawning and eye closure) and driver distraction (based on their direction of gaze). Challenges arise from designing an accurate, yet low-cost and non-intrusive system to continuously monitor the state of the driver. In this paper, we present two novel and practical approaches for continuous driver monitoring in low-light conditions: (i) Image synthesis: enabling monitoring in low-light conditions using just the smartphone RGB camera by synthesizing a thermal image from RGB with a Generative Adversarial Network, and (ii) Near-IR LED: using a low-cost near-IR (NIR) LED attachment to the smartphone, where the NIR LED acts as a light source to illuminate the driver's face, which is not visible to the human eyes, but can be captured by standard smartphone cameras without any specialized hardware. We show that the proposed techniques can capture the driver's face accurately in low-lighting conditions to monitor driver's state. Further, since NIR and thermal imagery is significantly different than RGB images, we present a systematic approach to generate labelled data, which is used to train existing computer vision models. We present an extensive evaluation of both the approaches with data collected from 15 drivers in controlled basement area and on real roads in low-light conditions. The proposed NIR LED setup has an accuracy (Fl-score) of 85% and 93.8% in detecting driver fatigue and distraction, respectively in low-light.

InSight: Monitoring the State of the Driver in Low-Light Using Smartphones

Advancing state of the art in face recognition and bridging the gap between laboratory and real-world scenarios require the availability of challenging databases. One of the challenging applications of face recognition is surveillance, where unconstrained video data is captured both in day and night time (visible and near infrared spectrum). These videos have multiple subjects in each frame, which are matched with good quality gallery images. Due to the lack of an existing database for such a cross spectral cross resolution video-to-still face recognition application, this is still an open research problem. This paper presents a video database that can be utilized to benchmark face recognition algorithms addressing cross spectral cross resolution matching. The proposed Cross-Spectral Cross-Resolution Video dataset (CSCRV) contains videos pertaining to 160 subjects with an open-set protocol. We present baseline results with two commercial matchers for two experimental scenarios, where we observe very low performance of both the matchers. It is our assertion that this dataset can help researchers develop robust face recognition algorithms to handle real world surveillance scenarios.

Cross-spectral cross-resolution video database for face recognition

Systems and techniques for monitoring driver state are described herein. In an example, a driver state monitoring system is adapted to receive a set of color images of a person, such as images of a driver of a vehicle with varying levels of illumination in the images. The driver state monitoring system may be further adapted to generate a set of synthesized thermal images from the set of color images. The driver state monitoring system may be further adapted to use a trained thermal image face detector to locate a human face in the synthesized thermal images. The driver state monitoring system may be further adapted to use a trained thermal image facial landmark predictor to locate facial landmarks in the synthesized thermal images. The driver state monitoring system may be further adapted to analyze the facial landmarks in the synthesized thermal images to determine facial feature movements.

Systems and methods for monitoring driver state

Existing face recognition algorithms achieve high recognition performance for frontal face images with good illumination and close proximity to the imaging device. However, most of the existing algorithms fail to perform equally well in surveillance scenarios, where videos are captured across varying resolutions and spectra. In surveillance settings, cameras are usually placed far away from the subjects, thereby resulting in variations across pose, illumination, occlusion, and resolution. Current video datasets used for face recognition are often captured in constrained environments, and thus fail to simulate the real world scenarios. In this paper, we present the FaceSurv database featuring 252 subjects in 460 videos. The proposed dataset contains over 142K face images, spread across videos captured in both visible and near-infrared spectra. Each video contains a group of individuals walking from 36ft towards the imaging device, offering a plethora of challenges common to surveillance settings. Benchmark experimental protocol and baseline results have been reported with state-of-the-art algorithms for face detection and recognition. It is our assertion that the availability of such a challenging database will facilitate the development of robust face recognition systems relevant to real world surveillance scenarios.

FaceSurv: A Benchmark Video Dataset for Face Detection and Recognition Across Spectra and Resolutions

One of the challenging applications in face recognition is video surveillance, where unconstrained low-resolution video data is captured both in day and night time (visible and near-infrared) with multiple subjects in frames, which are matched with high resolution gallery images. Due to the lack of an existing database for such a cross spectral cross resolution video-to-still face recognition application, this still remains an open research problem. The aim of this research is to come up with robust face recognition algorithms in surveillance scenarios. For this, we first present a video database which can be utilized to benchmark face recognition algorithms addressing crossspectral and cross-resolution matching. The proposed Cross-Spectral Cross-Resolution Video dataset version 2 (CSCRV-v2) contains 460 videos pertaining to 252 subjects with an open-set protocol. We then focus on the first step in the face recognition pipeline i.e., face detection and propose an algorithm FaceFinder. FaceFinder addresses shortcomings (like high false positive rate) of existing face detectors by making use of human body segmentation results of a trained Convolutional Neural Network model specifically designed for semantic segmentation. Experimental results on CSCRV-v2, are compared with that of two off-the-shelf face detection algorithms to show the efficacy of the proposed algorithm. We then focus on face recognition and present results with two commercial matchers for two experimental scenarios on the data. It is our assertion that our research will further facilitate the research community to develop robust face recognition algorithms to handle real world surveillance scenarios.

Sanchit Gupta

Papers

InSight: Monitoring the State of the Driver in Low-Light Using Smartphones

Cross-spectral cross-resolution video database for face recognition

Systems and methods for monitoring driver state

FaceSurv: A Benchmark Video Dataset for Face Detection and Recognition Across Spectra and Resolutions

Cross-spectral cross-resolution face recognition in videos