Local binary pattern (LBP) is a nonparametric descriptor, which efficiently summarizes the local structures of images. In recent years, it has aroused increasing interest in many areas of image processing and computer vision and has shown its effectiveness in a number of applications, in particular for facial image analysis, including tasks as diverse as face detection, face recognition, facial expression analysis, and demographic classification. This paper presents a comprehensive survey of LBP methodology, including several more recent variations. As a typical application of the LBP approach, LBP-based facial image analysis is extensively reviewed, while its successful extensions, which deal with various tasks of facial image analysis, are also highlighted.

/pdf/local-binary-patterns-and-its-application-to-facial-image-1kaldi7ot7.pdf

Local Binary Patterns and Its Application to Facial Image Analysis: A Survey

This paper introduces a mathematical model that incorporates the pertinent optical and physiological properties of skin reflections with the objective to increase our understanding of the algorithmic principles behind remote photoplethysmography (rPPG). The model is used to explain the different choices that were made in existing rPPG methods for pulse extraction. The understanding that comes from the model can be used to design robust or application-specific rPPG solutions. We illustrate this by designing an alternative rPPG method, where a projection plane orthogonal to the skin tone is used for pulse extraction. A large benchmark on the various discussed rPPG methods shows that their relative merits can indeed be understood from the proposed model.

/pdf/algorithmic-principles-of-remote-ppg-300xdjrz7d.pdf

Algorithmic Principles of Remote PPG

Automatic affect analysis has attracted great interest in various contexts including the recognition of action units and basic or non-basic emotions In spite of major efforts, there are several open questions on what the important cues to interpret facial expressions are and how to encode them In this paper, we review the progress across a range of affect recognition applications to shed light on these fundamental questions We analyse the state-of-the-art solutions by decomposing their pipelines into fundamental components, namely face registration, representation, dimensionality reduction and recognition We discuss the role of these components and highlight the models and new trends that are followed in their design Moreover, we provide a comprehensive analysis of facial representations by uncovering their advantages and limitations; we elaborate on the type of information they encode and discuss how they deal with the key challenges of illumination variations, registration errors, head-pose variations, occlusions, and identity bias This survey allows us to identify open issues and to define future directions for designing real-world affect recognition systems

Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition

Face anti-spoofing is crucial to prevent face recognition systems from a security breach. Previous deep learning approaches formulate face anti-spoofing as a binary classification problem. Many of them struggle to grasp adequate spoofing cues and generalize poorly. In this paper, we argue the importance of auxiliary supervision to guide the learning toward discriminative and generalizable cues. A CNN-RNN model is learned to estimate the face depth with pixel-wise supervision, and to estimate rPPG signals with sequence-wise supervision. The estimated depth and rPPG are fused to distinguish live vs. spoof faces. Further, we introduce a new face anti-spoofing database that covers a large range of illumination, subject, and pose variations. Experiments show that our model achieves the state-of-the-art results on both intra- and cross-database testing.

/pdf/learning-deep-models-for-face-anti-spoofing-binary-or-1ekom3p7fk.pdf

Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision

Photoplethysmography (PPG) is a noninvasive optical technique for detecting microvascular blood volume changes in tissues. Its ease of use, low cost and convenience make it an attractive area of research in the biomedical and clinical communities. Nevertheless, its single spot monitoring and the need to apply a PPG sensor directly to the skin limit its practicality in situations such as perfusion mapping and healing assessments or when free movement is required. The introduction of fast digital cameras into clinical imaging monitoring and diagnosis systems, the desire to reduce the physical restrictions, and the possible new insights that might come from perfusion imaging and mapping inspired the evolution of the conventional PPG technology to imaging PPG (IPPG). IPPG is a noncontact method that can detect heart-generated pulse waves by means of peripheral blood perfusion measurements. Since its inception, IPPG has attracted significant public interest and provided opportunities to improve personal healthcare. This study presents an overview of the wide range of IPPG systems currently being introduced along with examples of their application in various physiological assessments. We believe that the widespread acceptance of IPPG is happening, and it will dramatically accelerate the promotion of this healthcare model in the near future.

/pdf/photoplethysmography-revisited-from-contact-to-noncontact-2jla840amb.pdf

Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging

Remote photoplethysmography (rPPG) enables contactless monitoring of the blood volume pulse using a regular camera. Recent research focused on improved motion robustness, but the proposed blind source separation techniques (BSS) in RGB color space show limited success. We present an analysis of the motion problem, from which far superior chrominance-based methods emerge. For a population of 117 stationary subjects, we show our methods to perform in 92% good agreement (±1.96σ) with contact PPG, with RMSE and standard deviation both a factor of 2 better than BSS-based methods. In a fitness setting using a simple spectral peak detector, the obtained pulse-rate for modest motion (bike) improves from 79% to 98% correct, and for vigorous motion (stepping) from less than 11% to more than 48% correct. We expect the greatly improved robustness to considerably widen the application scope of the technology.

Robust Pulse Rate From Chrominance-Based rPPG

https://escholarship.org/content/qt66w2q2sj/qt66w2q2sj.pdf?t=plshl1

Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit — A pilot study

In this paper, we extensively investigate local features based facial expression recognition with face registration errors, which has never been addressed before. Our contributions are three fold. Firstly, we propose and experimentally study the histogram of oriented gradients (HOG) descriptors for facial representation. Secondly, we present facial representations based on local binary patterns (LBP) and local ternary patterns (LTP) extracted from overlapping local regions. Thirdly, we quantitatively study the impact of face registration errors on facial expression recognition using different facial representations. Overall LBP with overlapping gives the best performance (92.9% recognition rate on the Cohn-Kanade database), while maintaining a compact feature vector and best robustness against face registration errors.

Local features based facial expression recognition with face registration errors

A method of processing images of at least one living being, includes obtaining a sequence (19) of digital images taken at consecutive points in time. At least one measurement zone (26) comprising a plurality of image points is selected. For each measurement zone (26), a signal (28,30) representative of at least variations in a time- varying value of a combination of pixel values at at least a number of the image points for use in determining at least one of a presence and a frequency value of at least one peak in a spectrum of the signal (28,30) corresponding to a frequency of a periodic physiological phenomenon is obtained. The step (25) of selecting at least one measurement zone (26) includes analyzing information based on pixel data of a plurality of image parts in at least one of the images (19), each image part including at least one image point, and selecting each measurement zone (26) from contiguous parts determined to have similar characteristics.

Processing images of at least one living being

A method of facilitating obtaining a first signal, for analysis to characterize at least one periodic component, includes obtaining two second signals representative of intensities of electromagnetic radiation. The first signal is at least derivable from an output signal obtainable by applying a transformation to the second signals such that any value of the output signal is based on values from each respective second signal at corresponding points in time. The method further includes obtaining a value of a variable determining influences of components of respective second signals on the output signal when the signals corresponding to the second signals are captured and the transformation is applied, by (i) analyzing the first, second and/or the output signals to select a value of a parameter corresponding to a respective one of the variables; or (ii) calculating values of at least one time-varying factor corresponding to a respective one of the variables.

/pdf/method-and-system-for-obtaining-a-first-signal-for-analysis-4egyg0gbzl.pdf

Vincent Jeanne

Papers

Robust Pulse Rate From Chrominance-Based rPPG

Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit — A pilot study

Local features based facial expression recognition with face registration errors

Processing images of at least one living being

Method and system for obtaining a first signal for analysis to characterize at least one periodic component thereof