Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

This paper studies an approach to gait based human identification via similarity learning by deep convolutional neural networks (CNNs). With a pretty small group of labeled multi-view human walking videos, we can train deep networks to recognize the most discriminative changes of gait patterns which suggest the change of human identity. To the best of our knowledge, this is the first work based on deep CNNs for gait recognition in the literature. Here, we provide an extensive empirical evaluation in terms of various scenarios, namely, cross-view and cross-walking-condition, with different preprocessing approaches and network architectures. The method is first evaluated on the challenging CASIA-B dataset in terms of cross-view gait recognition. Experimental results show that it outperforms the previous state-of-the-art methods by a significant margin. In particular, our method shows advantages when the cross-view angle is large, i.e., no less than 36 degree. And the average recognition rate can reach 94 percent, much better than the previous best result (less than 65 percent). The method is further evaluated on the OU-ISIR gait dataset to test its generalization ability to larger data. OU-ISIR is currently the largest dataset available in the literature for gait recognition, with 4,007 subjects. On this dataset, the average accuracy of our method under identical view conditions is above 98 percent, and the one for cross-view scenarios is above 91 percent. Finally, the method also performs the best on the USF gait dataset, whose gait sequences are imaged in a real outdoor scene. These results show great potential of this method for practical applications.

A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs

The new developments in deep generative networks have significantly improve the quality and efficiency in generating realistically-looking fake face videos. In this work, we describe a new method to expose fake face videos generated with deep neural network models. Our method is based on detection of eye blinking in the videos, which is a physiological signal that is not well presented in the synthesized fake videos. Our method is evaluated over benchmarks of eye-blinking detection datasets and shows promising performance on detecting videos generated with DNN based software DeepFake.

In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking

Audio- and video-based biometric person authentication

This review article surveys extensively the current progresses made toward video-based human activity recognition Three aspects for human activity recognition are addressed including core technology, human activity recognition systems, and applications from low-level to high-level representation In the core technology, three critical processing stages are thoroughly discussed mainly: human object segmentation, feature extraction and representation, activity detection and classification algorithms In the human activity recognition systems, three main types are mentioned, including single person activity recognition, multiple people interaction and crowd behavior, and abnormal activity recognition Finally the domains of applications are discussed in detail, specifically, on surveillance environments, entertainment environments and healthcare systems Our survey, which aims to provide a comprehensive state-of-the-art review of the field, also addresses several challenges associated with these systems and applications Moreover, in this survey, various applications are discussed in great detail, specifically, a survey on the applications in healthcare monitoring systems

https://www.mdpi.com/2073-431X/2/2/88/pdf

A Review on Video-Based Human Activity Recognition

We present a new method for viewpoint independent gait biometrics. The system relies on a single camera, does not require camera calibration, and works with a wide range of camera views. This is achieved by a formulation where the gait is self-calibrating. These properties make the proposed method particularly suitable for identification by gait, where the advantages of completely unobtrusiveness, remoteness, and covertness of the biometric system preclude the availability of camera information and specific walking directions. The approach has been assessed for feature extraction and recognition capabilities on the SOTON gait database and then evaluated on a multiview database to establish recognition capability with respect to view invariance. Moreover, tests on the multiview CASIA-B database, composed of more than 2270 video sequences with 65 different subjects walking freely along different walking directions, have been performed. The obtained results show that human identification by gait can be achieved without any knowledge of internal or external camera parameters with a mean correct classification rate of 73.6% across all views using purely dynamic gait features. The performance of the proposed method is particularly encouraging for application in surveillance scenarios.

/pdf/self-calibrating-view-invariant-gait-biometrics-yqg3gcrpvq.pdf

Self-Calibrating View-Invariant Gait Biometrics

We present a new method for front-view gait biometrics which uses a single non-calibrated camera and extracts unique signatures from descriptors of a silhouette's deformation. The proposed approach is particularly suitable for identification by gait in the real world, where the advantages of completely unobtrusiveness, remoteness and covertness of the biometric system preclude the availability of camera information and where the CCTV images usually present subjects from an upper front-view. Tests on three different gait databases with subjects walking towards the camera have been performed. The obtained results, with mean CCR of 96.3%, show that gait recognition of individuals observed the front can be achieved without any knowledge of camera parameters. Moreover, the method has been applied to three different walking directions and the results have been compared with the algorithms found in literature. The performance of the proposed system is particularly encouraging for its appliance in surveillance scenarios.

/pdf/front-view-gait-recognition-3z5q4f18b8.pdf

Front-view Gait Recognition

Many studies have confirmed that gait analysis can be used as a new biometrics. In this research, gait analysis is deployed for people identification in multi-camera surveillance scenarios. We present a new method for viewpoint independent markerless gait analysis that does not require camera calibration and works with a wide range of walking directions. These properties make the proposed method particularly suitable for gait identification in real surveillance scenarios where people and their behaviour need to be tracked across a set of cameras. Tests on 300 synthetic and real video sequences, with subjects walking freely along different walking directions, have been performed. Since the choice of the cameras' characteristics is a key-point for the development of a smart surveillance system, the performance of the proposed approach is measured with respect to different video properties: spatial resolution, frame-rate, data compression and image quality. The obtained results show that markerless gait analysis can be achieved without any knowledge of camera's position and subject's pose. The extracted gait parameters allow recognition of people walking from different views with a mean recognition rate of 92.2% and confirm that gait can be effectively used for subjects' identification in a multi-camera surveillance scenario.

/pdf/performance-analysis-for-automated-gait-extraction-and-4k3d2aare7.pdf

Performance analysis for automated gait extraction and recognition in multi-camera surveillance

A markerless computer vision technique specifically designed to track natural elements on the human body surface is presented. The method implements the estimate of translation, rotation, and scaling by means of a maximum likelihood approach carried out in the Gauss-Laguerre transform domain. The approach is particularly suitable for human movement analysis in clinical contexts, where kinematics is at present performed by means of marker-based systems. Specific drawbacks of these latter systems, such as the burden of time for marker placement and the intrinsic intrusive nature, would be removed by the proposed method. Experimental results in terms of tracking performance are obtained by analyzing video sequences capturing the execution of the sit-to-stand task in two groups of young and elderly volunteers. The results are compared with clinical studies that used marker-based systems, and are particularly encouraging for a future extension of the approach to other motor tasks and to predict scores obtained from the physical performance batteries that are widely and regularly used by clinicians and physical therapists.

Markerless Human Motion Analysis in Gauss–Laguerre Transform Domain: An Application to Sit-To-Stand in Young and Elderly People

Restoration of upper limb movements in subjects recovering from stroke is an essential keystone in rehabilitative practices. Rehabilitation of arm movements, in fact, is usually a far more difficult one as compared to that of lower extremities. For these reasons, researchers are developing new methods and technologies so that the rehabilitative process could be more accurate, rapid and easily accepted by the patient. This paper introduces the proof of concept for a new non-invasive FES-assisted rehabilitation system for the upper limb, called smartFES (sFES), where the electrical stimulation is controlled by a biologically inspired neural inverse dynamics model, fed by the kinematic information associated with the execution of a planar goal-oriented movement. More specifically, this work details two steps of the proposed system: an ad hoc markerless motion analysis algorithm for the estimation of kinematics, and a neural controller that drives a synthetic arm. The vision of the entire system is to acquire kinematics from the analysis of video sequences during planar arm movements and to use it together with a neural inverse dynamics model able to provide the patient with the electrical stimulation patterns needed to perform the movement with the assisted limb. The markerless motion tracking system aims at localizing and monitoring the arm movement by tracking its silhouette. It uses a specifically designed motion estimation method, that we named Neural Snakes, which predicts the arm contour deformation as a first step for a silhouette extraction algorithm. The starting and ending points of the arm movement feed an Artificial Neural Controller, enclosing the muscular Hill's model, which solves the inverse dynamics to obtain the FES patterns needed to move a simulated arm from the starting point to the desired point. Both position error with respect to the requested arm trajectory and comparison between curvature factors have been calculated in order to determine the accuracy of the system. The proposed method has been tested on real data acquired during the execution of planar goal-oriented arm movements. Main results concern the capability of the system to accurately recreate the movement task by providing a synthetic arm model with the stimulation patterns estimated by the inverse dynamics model. In the simulation of movements with a length of ± 20 cm, the model has shown an unbiased angular error, and a mean (absolute) position error of about 1.5 cm, thus confirming the ability of the system to reliably drive the model to the desired targets. Moreover, the curvature factors of the factual human movements and of the reconstructed ones are similar, thus encouraging future developments of the system in terms of reproducibility of the desired movements. A novel FES-assisted rehabilitation system for the upper limb is presented and two parts of it have been designed and tested. The system includes a markerless motion estimation algorithm, and a biologically inspired neural controller that drives a biomechanical arm model and provides the stimulation patterns that, in a future development, could be used to drive a smart Functional Electrical Stimulation system (sFES). The system is envisioned to help in the rehabilitation of post stroke hemiparetic patients, by assisting the movement of the paretic upper limb, once trained with a set of movements performed by the therapist or in virtual reality. Future work will include the application and testing of the stimulation patterns in real conditions.

/pdf/a-neural-tracking-and-motor-control-approach-to-improve-kyezmeauhp.pdf

Michela Goffredo

Papers

Self-Calibrating View-Invariant Gait Biometrics

Front-view Gait Recognition

Performance analysis for automated gait extraction and recognition in multi-camera surveillance

Markerless Human Motion Analysis in Gauss–Laguerre Transform Domain: An Application to Sit-To-Stand in Young and Elderly People

A neural tracking and motor control approach to improve rehabilitation of upper limb movements.