Human pose estimation has been actively studied for decades. While traditional approaches rely on 2d data like images or videos, the development of Time-of-Flight cameras and other depth sensors created new opportunities to advance the field. We give an overview of recent approaches that perform human motion analysis which includes depth-based and skeleton-based activity recognition, head pose estimation, facial feature detection, facial performance capture, hand pose estimation and hand gesture recognition. While the focus is on approaches using depth data, we also discuss traditional image based methods to provide a broad overview of recent developments in these areas.

A Survey on Human Motion Analysis from Depth Data

Emotional health plays very vital role to improve people’s quality of lives, especially for the elderly. Negative emotional states can lead to social or mental health problems. To cope with emotional health problems caused by negative emotions in daily life, we propose efficient facial expression recognition system to contribute in emotional healthcare system. Thus, facial expressions play a key role in our daily communications, and recent years have witnessed a great amount of research works for reliable facial expressions recognition (FER) systems. Therefore, facial expression evaluation or analysis from video information is very challenging and its accuracy depends on the extraction of robust features. In this paper, a unique feature extraction method is presented to extract distinguished features from the human face. For person independent expression recognition, depth video data is used as input to the system where in each frame, pixel intensities are distributed based on the distances to the camera. A novel robust feature extraction process is applied in this work which is named as local directional position pattern (LDPP). In LDPP, after extracting local directional strengths for each pixel such as applied in typical local directional pattern (LDP), top directional strength positions are considered in binary along with their strength sign bits. Considering top directional strength positions with strength signs in LDPP can differentiate edge pixels with bright as well as dark regions on their opposite sides by generating different patterns whereas typical LDP only considers directions representing the top strengths irrespective of their signs as well as position orders (i.e., directions with top strengths represent 1 and rest of them 0), which can generate the same patterns in this regard sometimes. Hence, LDP fails to distinguish edge pixels with opposite bright and dark regions in some cases which can be overcome by LDPP. Moreover, the LDPP capabilities are extended through principal component analysis (PCA) and generalized discriminant analysis (GDA) for better face characteristic illustration in expression. The proposed features are finally applied with deep belief network (DBN) for expression training and recognition.

Facial Expression Recognition Utilizing Local Direction-Based Robust Features and Deep Belief Network

In this work, we present a system for recognizing letters and finger-spelled words of the American sign language (ASL) in real-time. To this end, the system segments the hand and estimates the hand orientation from captured depth data. The letter classification is based on average neighborhood margin maximization and relies on the segmented depth data of the hands. For word recognition, the letter confidences are aggregated. Furthermore, the word recognition is used to improve the letter recognition by updating the training examples of the letter classifiers on-line.

/pdf/real-time-sign-language-letter-and-word-recognition-from-36z82u8ohp.pdf

Real-time sign language letter and word recognition from depth data

Cameras for 3D depth imaging, using either time-of-flight (ToF) or structured light sensors, have received a lot of attention recently and have been improved considerably over the last few years. The present techniques make full-range 3D data available at video frame rates, and thus pave the way for a much broader application of 3D vision systems. A series of workshops have closely followed the developments within ToF imaging over the years. Today, depth imaging workshops can be found at every major computer vision conference. The papers presented in this volume stem from a seminar on Time-of-Flight Imaging held at Schloss Dagstuhl in October 2012. They cover all aspects of ToF depth imaging, from sensors and basic foundations, to algorithms for low level processing, to important applications that exploit depth imaging. In addition, this book contains the proceedings of a workshop on Imaging New Modalities, which was held at the German Conference on Pattern Recognition in Saarbrcken, Germany, in September 2013. A state-of-the-art report on the Kinect sensor and its applications is followed by two reports on local and global ToF motion compensation and a novel depth capture system using a plenoptic multi-lens multi-focus camera sensor.

Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications

Sign languages represent an interesting niche for statistical machine translation that is typically hampered by the scarceness of suitable data, and most papers in this area apply only a few, well-known techniques that are not adapted to small-sized corpora. In this article, we analyze existing data collections and emphasize their quality and usability for statistical machine translation. We also offer findings in the proper preprocessing of a sign language corpus, by introducing sentence end markers, splitting compound words and handling parallel communication channels. Then, we focus on optimization procedures that are tailored to scarce resources, such as scaling factor optimization, alignment optimization and system combination. All methods are evaluated on two of the largest sign language corpora available.

Analysis, preparation, and optimization of statistical sign language machine translation

The SignSpeak project will be the first step to approach sign language recognition and translation at a scientific level already reached in similar research fields such as automatic speech recognition or statistical machine translation of spoken languages Deaf communities revolve around sign languages as they are their natural means of communication Although deaf, hard of hearing and hearing signers can communicate without problems amongst themselves, there is a serious challenge for the deaf community in trying to integrate into educational, social and work environments The overall goal of SignSpeak is to develop a new vision-based technology for recognizing and translating continuous sign language to text New knowledge about the nature of sign language structure from the perspective of machine recognition of continuous sign language will allow a subsequent breakthrough in the development of a new vision-based technology for continuous sign language recognition and translation Existing and new publicly available corpora will be used to evaluate the research progress throughout the whole project

/pdf/the-signspeak-project-bridging-the-gap-between-signers-and-afq26ghagn.pdf

M. Wheatley

Papers

The SignSpeak Project - Bridging the Gap Between Signers and Speakers

SignSpeak - Scientific understanding and vision-based technological development for continuous sign language recognition and translation