The main goal of this work is to design an automated solution based on RGB-D data for quantitative analysis, perceptible evaluation and comparison of handball player’s performance. To that end, we introduced a new RGB-D dataset that can be used for an objective comparison and evaluation of handball player’s performance during throws. We filmed 62 handball players (44 beginners and 18 experts), who performed the same type of action, using a Kinect V2 sensor that provides RGB data, depth data and skeletons. Moreover, using skeleton data simulating 3D joint connections, we examined the main angles responsible for throwing performance in order to analyze individual skills of handball players (beginners against model and experts) relatively to throw actions. The comparison was performed statically (using only one frame) as well as dynamically during the entire throwing action. In particular, given the temporal sequence of 25 joints of each handball player, we adopted the dynamic time warping technique to compare the throwing motion between two athletes. The obtained results were found to be promising. Thus, the suggested markless solution would help handball coaches to optimize beginners’ movements during throwing actions.

Skeleton-based comparison of throwing motion for handball players

Video understanding requires abundant semantic information. Substantial progress has been made on deep learning models in the image, text, and audio domains, and notable efforts have been recently dedicated to the design of deep networks in the video domain. We discuss the state-of-the-art convolutional neural network (CNN) and its pipelines for the exploration of video features, various fusion strategies, and their performances; we also discuss the limitations of CNN for long-term motion cues and the use of sequential learning models such as long short-term memory to overcome these limitations. In addition, we address various multi-model approaches for extracting important cues and score fusion techniques from hybrid deep learning frameworks. Then, we highlight future plans in this domain, recent trends, and substantial challenges for video understanding. This survey’s objectives are to study the plethora of approaches that have been developed for solving video understanding problems, to comprehensively study spatiotemporal cues, to explore the various models that are available for solving these problems and to identify the most promising approaches.

A study on deep learning spatiotemporal models and feature extraction techniques for video understanding

Presently in most of the real world applications like video surveillance systems, human activities are captured and retained as multimodal information for authorized permitted actions. However the degree of accuracy in recognition of such actions greatly depends on many factors, including occlusion, illumination factor, cluttered environment, and so on. In this work we propose the correlation of temporal difference frame (CTDF) algorithm which captures the local maxima’s of every small movement and its neighboring information. Temporal difference obtained between frames, block size defined to obtain the surround information and finally, the comparison of one to all points between identified frames greatly increase the accuracy. The algorithm takes in the raw video input of the standard UT interaction and BIT interaction datasets. Features extracted using the proposed algorithm is passed through variants of SVM which gives state of art results, 95.83% accuracy for UT Interaction and an accuracy of 90.4% for BIT interaction dataset.

Action recognition using Correlation of Temporal Difference Frame (CTDF)—an algorithmic approach

Human action recognition from video is an important research area in the field of computer vision. It is an integral part of surveillance systems, human–computer interactions and various real-world applications. This paper presents a method to automatically identify view invariant human activity from input video stream using Motion History Image (MHI) and Histogram of Directional Derivative features (HODD). The proposed system uses Multi View Human Action Video (MuHAVi) dataset for training and testing and the Support Vector Machine (SVM) classifier for classification.

Multi View Human Action Recognition Using HODD

This paper presents a method for action recognition based on trajectory representation. The motion information for action recognition is obtained by tracking the key points through the video sequence using a standard KLT tracker. We propose a new 3D spatio-temporal descriptor based on histogram of directional derivative (3D-HODD) to describe the volume extracted around the trajectories. Our descriptor describes the local object appearance within the volume effectively and distinctively. The final descriptor constructed by combining the shape of trajectories (motion information) with 3D-HODD (appearance information). A multiclass support vector machine has been used to classify the human activities. The proposed framework for recognition of human action has been extensively validated on the benchmark datasets, with a focus that this methodology is robust and attains more precise human action recognition rate as compared to current methodologies available.

Three-dimensional spatio-temporal trajectory descriptor for human action recognition

In this paper, we have introduced a novel approach for recognition of activities of daily living (ADL). These activities are the ones that the human beings perform in daily life. At the object level, we used computational color model for efficient object segmentation and tracking to handle dynamic background change in indoor environment. To make it computationally efficient, cosine of the angle between the expected image color vector and current image color vector is used. At feature level, we have presented a linear predictive coding of histogram of directional derivative as a spatio-temporal descriptor. Our proposed descriptor describes the local object shape and appearance within cuboids effectively and distinctively. A multiclass support vector machine has been used to classify the human activities. The proposed framework for recognition of indoor human activity has been extensively validated on the benchmark of ADL datasets, with a focus that this methodology is robust and attains more precise human activity recognition rate as compared to current methodologies available.

Recognition of vision-based activities of daily living using linear predictive coding of histogram of directional derivative

In this paper, we introduced a novel local spatio-temporal feature descriptor for human action recognition. Our proposed descriptor is based on the Histogram of Directional Derivative (HODD). Histogram of gradient (HOG) has been widely used to generate the descriptor for local features for human action recognition. The distinctiveness of descriptor is essential to match similar action and differentiate different action. However most of the techniques in literature for HOG is based on orientation quantization leading to the reduction of the distinctiveness. Our proposed descriptor describes the local object shape and appearance within cuboid effectively and distinctively.

Histogram of directional derivative based spatio-temporal descriptor for human action recognition

Many times during the hospital visits it is common for a patient to receive medications using an IV saline, which is capable of passing the various medications like antibiotics, steroids fluids. IV saline is very useful in such cases where patient is suffering from weakness or dehydration from illness however various risks are involved in this treatment due to human mistakes in monitoring the IV bag, such risk may lead to dangerous situations like air embolism, reversal of blood to empty saline bag. This type of mistake may lead to the serious condition of patient or even may cause death of person. So in advance we proposed a system which will monitor the IV Bag, also it will alert the medical staff on duty whenever bag gets empty and needs to change the bag also added benefits of this system is it will continuously monitor the heart rate, SPO2 readings and temperature of the patient so whenever the readings will goes below the preset critical value it will provide an alert system using a buzzer and also an alert message will be sent to the medical staffs mobile or computer also it uses a cam module for live patient health condition monitoring.

Sidharth B. Bhorge

Papers

Recognition of vision-based activities of daily living using linear predictive coding of histogram of directional derivative

Histogram of directional derivative based spatio-temporal descriptor for human action recognition

Three-dimensional spatio-temporal trajectory descriptor for human action recognition

IoT Based IV Pole Monitoring System