This paper introduces a mathematical model that incorporates the pertinent optical and physiological properties of skin reflections with the objective to increase our understanding of the algorithmic principles behind remote photoplethysmography (rPPG). The model is used to explain the different choices that were made in existing rPPG methods for pulse extraction. The understanding that comes from the model can be used to design robust or application-specific rPPG solutions. We illustrate this by designing an alternative rPPG method, where a projection plane orthogonal to the skin tone is used for pulse extraction. A large benchmark on the various discussed rPPG methods shows that their relative merits can indeed be understood from the proposed model.

/pdf/algorithmic-principles-of-remote-ppg-300xdjrz7d.pdf

Algorithmic Principles of Remote PPG

Photoplethysmography (PPG) is a noninvasive optical technique for detecting microvascular blood volume changes in tissues. Its ease of use, low cost and convenience make it an attractive area of research in the biomedical and clinical communities. Nevertheless, its single spot monitoring and the need to apply a PPG sensor directly to the skin limit its practicality in situations such as perfusion mapping and healing assessments or when free movement is required. The introduction of fast digital cameras into clinical imaging monitoring and diagnosis systems, the desire to reduce the physical restrictions, and the possible new insights that might come from perfusion imaging and mapping inspired the evolution of the conventional PPG technology to imaging PPG (IPPG). IPPG is a noncontact method that can detect heart-generated pulse waves by means of peripheral blood perfusion measurements. Since its inception, IPPG has attracted significant public interest and provided opportunities to improve personal healthcare. This study presents an overview of the wide range of IPPG systems currently being introduced along with examples of their application in various physiological assessments. We believe that the widespread acceptance of IPPG is happening, and it will dramatically accelerate the promotion of this healthcare model in the near future.

/pdf/photoplethysmography-revisited-from-contact-to-noncontact-2jla840amb.pdf

Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging

Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the heart rate (HR). While considerable progress has been made in the last few years, still many issues remain open. In particular, state of-the-art approaches are not robust enough to operate in natural conditions (e.g. in case of spontaneous movements, facial expressions, or illumination changes). Opposite to previous approaches that estimate the HR by processing all the skin pixels inside a fixed region of interest, we introduce a strategy to dynamically select face regions useful for robust HR estimation. Our approach, inspired by recent advances on matrix completion theory, allows us to predict the HR while simultaneously discover the best regions of the face to be used for estimation. Thorough experimental evaluation conducted on public benchmarks suggests that the proposed approach significantly outperforms state-of the-art HR estimation methods in naturalistic conditions.

/pdf/self-adaptive-matrix-completion-for-heart-rate-estimation-34gc3nht50.pdf

Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions

Current progress in wearable and implanted health monitoring technologies has strong potential to alter the future of healthcare services by enabling ubiquitous monitoring of patients. A typical health monitoring system consists of a network of wearable or implanted sensors that constantly monitor physiological parameters. Collected data are relayed using existing wireless communication protocols to a base station for additional processing. This article provides researchers with information to compare the existing low-power communication technologies that can potentially support the rapid development and deployment of WBAN systems, and mainly focuses on remote monitoring of elderly or chronically ill patients in residential environments.

A Survey on Wireless Body Area Networks for eHealthcare Systems in Residential Environments

Vital signs such as pulse rate and breathing rate are currently measured using contact probes. But, non-contact methods for measuring vital signs are desirable both in hospital settings (e.g. in NICU) and for ubiquitous in-situ health tracking (e.g. on mobile phone and computers with webcams). Recently, camera-based non-contact vital sign monitoring have been shown to be feasible. However, camera-based vital sign monitoring is challenging for people with darker skin tone, under low lighting conditions, and/or during movement of an individual in front of the camera. In this paper, we propose distancePPG, a new camera-based vital sign estimation algorithm which addresses these challenges. DistancePPG proposes a new method of combining skin-color change signals from different tracked regions of the face using a weighted average, where the weights depend on the blood perfusion and incident light intensity in the region, to improve the signal-to-noise ratio (SNR) of camera-based estimate. One of our key contributions is a new automatic method for determining the weights based only on the video recording of the subject. The gains in SNR of camera-based PPG estimated using distancePPG translate into reduction of the error in vital sign estimation, and thus expand the scope of camera-based vital sign monitoring to potentially challenging scenarios. Further, a dataset will be released, comprising of synchronized video recordings of face and pulse oximeter based ground truth recordings from the earlobe for people with different skin tones, under different lighting conditions and for various motion scenarios.

DistancePPG: Robust non-contact vital signs monitoring using a camera

/pdf/distanceppg-robust-non-contact-vital-signs-monitoring-using-4ht5mlqhhx.pdf

Blood carries oxygen and nutrients to the trillions of cells in our body to sustain vital life processes. Lack of blood perfusion can cause irreversible cell damage. Therefore, blood perfusion measurement has widespread clinical applications. In this paper, we develop PulseCam — a new camera-based, motion-robust, and highly sensitive blood perfusion imaging modality with 1 mm spatial resolution and 1 frame-per-second temporal resolution. Existing camera-only blood perfusion imaging modality suffers from two core challenges: (i) motion artifact, and (ii) small signal recovery in the presence of large surface reflection and measurement noise. PulseCam addresses these challenges by robustly combining the video recording from the camera with a pulse waveform measured using a conventional pulse oximeter to obtain reliable blood perfusion maps in the presence of motion artifacts and outliers in the video recordings. For video stabilization, we adopt a novel brightness-invariant optical flow algorithm that helps us reduce error in blood perfusion estimate below 10% in different motion scenarios compared to 20–30% error when using current approaches. PulseCam can detect subtle changes in blood perfusion below the skin with at least two times better sensitivity, three times better response time, and is significantly cheaper compared to infrared thermography. PulseCam can also detect venous or partial blood flow occlusion that is difficult to identify using existing modalities such as the perfusion index measured using a pulse oximeter. With the help of a pilot clinical study, we also demonstrate that PulseCam is robust and reliable in an operationally challenging surgery room setting. We anticipate that PulseCam will be used both at the bedside as well as a point-of-care blood perfusion imaging device to visualize and analyze blood perfusion in an easy-to-use and cost-effective manner.

/pdf/pulsecam-a-camera-based-motion-robust-and-highly-sensitive-4drjgj5n3u.pdf

PulseCam: a camera-based, motion-robust and highly sensitive blood perfusion imaging modality.

Measuring blood perfusion is important in medical care as an indicator of injury and disease. However, currently available devices to measure blood perfusion like laser Doppler flowmetry are bulky, expensive, and cumbersome to use. An alternative low-cost and portable camera-based blood perfusion measurement system has recently been proposed, but such camera-only system produces noisy low-resolution blood perfusion maps. In this paper, we propose a new multi-sensor modality, named PulseCam, for measuring blood perfusion by combining a traditional pulse oximeter with a video camera in a unique way to provide low noise and high-resolution blood perfusion maps. Our proposed multi-sensor modality improves per pixel signal to noise ratio of measured perfusion map by up to 3 dB and improves the spatial resolution by 2 — 3 times compared to best known camera-only methods. Blood perfusion measured in the palm using our PulseCam setup during a post-occlusive reactive hyperemia (PORH) test replicates standard PORH response curve measured using laser Doppler flowmetry device but with much lower cost and a portable setup making it suitable for further development as a clinical device.

PulseCam: High-resolution blood perfusion imaging using a camera and a pulse oximeter

Recent progress in deep generative models has improved the quality of neural vocoders in speech domain. However, generating a high-quality singing voice remains challenging due to a wider variety of musical expressions in pitch, loudness, and pronunciations. In this work, we propose a hierarchical diffusion model for singing voice neural vocoders. The proposed method consists of multiple diffusion models operating in different sampling rates; the model at the lowest sampling rate focuses on generating accurate low-frequency components such as pitch, and other models progressively generate the waveform at higher sampling rates on the basis of the data at the lower sampling rate and acoustic features. Experimental results show that the proposed method produces high-quality singing voices for multiple singers, outperforming state-of-the-art neural vocoders with a similar range of computational costs.

Mayank Kumar

Papers

DistancePPG: Robust non-contact vital signs monitoring using a camera

DistancePPG: Robust non-contact vital signs monitoring using a camera

PulseCam: a camera-based, motion-robust and highly sensitive blood perfusion imaging modality.

PulseCam: High-resolution blood perfusion imaging using a camera and a pulse oximeter

Hierarchical Diffusion Models for Singing Voice Neural Vocoder