Photoplethysmography and its application in clinical physiological measurement

We present a new algorithm for manifold learning and nonlinear dimensionality reduction. Based on a set of unorganized da-ta points sampled with noise from a parameterized manifold, the local geometry of the manifold is learned by constructing an approxi-mation for the tangent space at each point, and those tangent spaces are then aligned to give the global coordinates of the data pointswith respect to the underlying manifold. We also present an error analysis of our algorithm showing that reconstruction errors can bequite small in some cases. We illustrate our algorithm using curves and surfaces both in 2D/3D Euclidean spaces and higher dimension-al Euclidean spaces. We also address several theoretical and algorithmic issues for further research and improvements.

Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment

Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the heart rate (HR). While considerable progress has been made in the last few years, still many issues remain open. In particular, state of-the-art approaches are not robust enough to operate in natural conditions (e.g. in case of spontaneous movements, facial expressions, or illumination changes). Opposite to previous approaches that estimate the HR by processing all the skin pixels inside a fixed region of interest, we introduce a strategy to dynamically select face regions useful for robust HR estimation. Our approach, inspired by recent advances on matrix completion theory, allows us to predict the HR while simultaneously discover the best regions of the face to be used for estimation. Thorough experimental evaluation conducted on public benchmarks suggests that the proposed approach significantly outperforms state-of the-art HR estimation methods in naturalistic conditions.

/pdf/self-adaptive-matrix-completion-for-heart-rate-estimation-34gc3nht50.pdf

Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions

Sign language facilitates communication between hearing impaired peoples and the rest of the society. A number of sign language recognition (SLR) systems have been developed by researchers, but they are limited to isolated sign gestures only. In this paper, we propose a modified long short-term memory (LSTM) model for continuous sequences of gestures or continuous SLR that recognizes a sequence of connected gestures. It is based on splitting of continuous signs into sub-units and modeling them with neural networks. Thus, the consideration of a different combination of sub-units is not required during training. The proposed system has been tested with 942 signed sentences of Indian Sign Language (ISL). These sign sentences are recognized using 35 different sign words. The average accuracy of 72.3% and 89.5% has been recorded on signed sentences and isolated sign words, respectively.

A Modified LSTM Model for Continuous Sign Language Recognition Using Leap Motion

Continuous monitoring of respiration is essential for early detection of critical illness. Current methods require sensors attached to the body and/or are not robust to subject motion. Alternative camera-based solutions have been presented using motion vectors and remote photoplethysmography. In this work, we present a non-contact camera-based method to detect respiration, which can operate in both visible and dark lighting conditions by detecting the respiratory-induced colour differences of the skin. We make use of the close similarity between skin colour variations caused by the beating of the heart and those caused by respiration, leading to a much improved signal quality compared to single-channel approaches. Essentially, we propose to find the linear combination of colour channels which suppresses the distortions best in a frequency band including pulse rate, and subsequently we use this same linear combination to extract the respiratory signal in a lower frequency band. Evaluation results obtained from recordings on healthy subjects which perform challenging scenarios, including motion, show that respiration can be accurately detected over the entire range of respiratory frequencies, with a correlation coefficient of 0.96 in visible light and 0.98 in infrared, compared to 0.86 with the best-performing non-contact benchmark algorithm. Furthermore, evaluation on a set of videos recorded in a Neonatal Intensive Care Unit (NICU) shows that this technique looks promising as a future alternative to current contact-sensors showing a correlation coefficient of 0.87.

Robust respiration detection from remote photoplethysmography

Human pulse rate (PR) can be estimated in several ways, including measurement instruments that directly count the PR through contact- and noncontact-based approaches. Over the last decade, computer-vision-assisted noncontact-based PR estimation has evolved significantly. Such techniques can be adopted for clinical purposes to mitigate some of the limitations of contact-based techniques. However, existing vision-guided noncontact-based techniques have not been benchmarked with respect to a challenging dataset. In view of this, we present a systematic review of such techniques implemented over a uniform computing platform. We have simultaneously recorded the PR and video of 14 volunteers. Five sets of data have been recorded for every volunteer using five different experimental conditions by varying the distance from the camera and illumination condition. Pros and cons of the existing noncontact image- and video-based PR techniques have been discussed with respect to our dataset. Experimental evaluation suggests that image- or video-based PR estimation can be highly effective for nonclinical purposes, and some of these approaches are very promising toward developing clinical applications. The present review is the first in this field of contactless vision-guided PR estimation research.

Computer-Vision-Guided Human Pulse Rate Estimation: A Review

A sign language is generally composed of three main parts, namely manual signas that are gestures made by hand or fingers movements, non-manual signs such as facial expressions or body postures, and finger-spelling where words are spelt out using gestures by the signers to convey the meaning. In literature, researchers have proposed various Sign Language Recognition (SLR) systems by focusing only one part of the sign language. However, combination of different parts has not been explored much. In this paper, we present a framework to recognize manual signs and finger spellings using Leap motion sensor. In the first phase, Support Vector Machine (SVM) classifier has been used to differentiate between manual and finger spelling gestures. Next, two BLSTM-NN classifiers are used for the recognition of manual signs and finger-spelling gestures using sequence-classification and sequence-transcription based approaches, respectively. A dataset of 2240 sign gestures consisting of 28 isolated manual signs and 28 finger-spelling words, has been recorded involving 10 users. We have obtained an overall accuracy of 63.57% in real-time recognition of sign gestures.

Real-time recognition of sign language gestures and air-writing using leap motion

Designing of marker-based augmented reality learning environment for kids using convolutional neural network architecture

Signature recognition is identifying the signature’s owner, whereas verification is the process to find whether a signature is genuine or forged. Though, both are important in the field of forensic sciences, however, verification is more important to banks and credit card companies. In this paper, we have proposed a methodology to analyze 3D signatures captured using Leap motion sensor. We have extended existing 2D features into 3D from raw signatures and applied well-known classifiers for recognition as well as verification. We have shown that the 3rd dimension, which essentially represents instantaneous pressure during writing, can improve the accuracy of the biometric systems. We have created a large dataset containing more than 2000 signatures registered by 100 volunteers using the Leap motion interface. This has been made available online for the research community. Our analysis shows that, the proposed 3D extension is better than its original 2D version. Recognition and verification accuracy have increased by 6.8% and 9.5%, respectively using k-NN. Similarly, accuracy has increased by 9.9% (recognition) and 6.5% (verification) when HMM is used as the classifier. Similar results have been recorded on benchmark datasets. A comparison with 2D tablet-stylus interface has been carried out which also supports our claims. We believe, Leap motion can be an alternative to the existing biometric setups.

Analysis of 3D signatures recorded using leap motion sensor

Recognition of signature is a method of identification, whereas verification takes the decision about its genuineness. Though recognition and verification both play important role in forensic sciences, however, recognition is of special importance to the banking sectors. In this paper, we present a methodology to analyse 3D signatures captured using Leap motion sensor with the help of a new feature-set extracted using convex hull vertices enclosing the signature. We have used k-NN and HMM classifiers to classify signatures. Experiments carried out using our dataset as well as publicly available datasets reveal that the proposed feature-set can reduce the computational burden significantly as compared to existing features. It has been observed that a 10-fold computational gain can be achieved with non-noticeable loss in performance using the proposed feature-set as compared with the existing high-level features due to significant reduction in the feature vector size. On a large dataset of 1600 samples, two of the existing features take approximately 60s and 3s to recognise signatures using k-NN and HMM classifiers. However, features constructed using convex hull vertices take 1.9s and 0.4s, respectively. Our proposed system can be used in applications where recognition and verification need to be performed quickly on large datasets comprising with billions of samples.

Santosh Kumar Behera

Papers

Computer-Vision-Guided Human Pulse Rate Estimation: A Review

Real-time recognition of sign language gestures and air-writing using leap motion

Designing of marker-based augmented reality learning environment for kids using convolutional neural network architecture

Analysis of 3D signatures recorded using leap motion sensor

Fast recognition and verification of 3D air signatures using convex hulls