In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly’s daily life by monitoring their cognitive and physical function through daily activities. This survey focuses on critical role of machine learning in developing HAR applications based on inertial sensors in conjunction with physiological and environmental sensors.

Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey

Research in offline Arabic handwriting recognition has increased considerably in the past few years. This is evident from the numerous research results published recently in major journals and conferences in the area of handwriting recognition. Features and classifications techniques utilized in recent research work have diversified noticeably compared to the past. Moreover, more efforts have been diverted, in last few years, to construct different databases for Arabic handwriting recognition. This article provides a comprehensive survey of recent developments in Arabic handwriting recognition. The article starts with a summary of the characteristics of Arabic text, followed by a general model for an Arabic text recognition system. Then the used databases for Arabic text recognition are discussed. Research works on preprocessing phase, like text representation, baseline detection, line, word, character, and subcharacter segmentation algorithms, are presented. Different feature extraction techniques used in Arabic handwriting recognition are identified and discussed. Different classification approaches, like HMM, ANN, SVM, k-NN, syntactical methods, etc., are discussed in the context of Arabic handwriting recognition. Works on Arabic lexicon construction and spell checking are presented in the postprocessing phase. Several summary tables of published research work are provided for used Arabic text databases and reported results on Arabic character, word, numerals, and text recognition. These tables summarize the features, classifiers, data, and reported recognition accuracy for each technique. Finally, we discuss some future research directions in Arabic handwriting recognition.

Offline arabic handwritten text recognition: A Survey

Human Activity Recognition (HAR) is a field that infers human activities from raw time-series signals acquired through embedded sensors of smartphones and wearable devices. It has gained much attraction in various smart home environments, especially to continuously monitor human behaviors in ambient assisted living to provide elderly care and rehabilitation. The system follows various operation modules such as data acquisition, pre-processing to eliminate noise and distortions, feature extraction, feature selection, and classification. Recently, various state-of-the-art techniques have proposed feature extraction and selection techniques classified using traditional Machine learning classifiers. However, most of the techniques use rustic feature extraction processes that are incapable of recognizing complex activities. With the emergence and advancement of high computational resources, Deep Learning techniques are widely used in various HAR systems to retrieve features and classification efficiently. Thus, this review paper focuses on providing profound concise of deep learning techniques used in smartphone and wearable sensor-based recognition systems. The proposed techniques are categorized into conventional and hybrid deep learning models described with its uniqueness, merits, and limitations. The paper also discusses various benchmark datasets used in existing techniques. Finally, the paper lists certain challenges and issues that require future research and improvements.

Human Activity Recognition With Smartphone and Wearable Sensors Using Deep Learning Techniques: A Review

Skin cancer is caused due to unusual development of skin cells and deadly type cancer. Early diagnosis is very significant and can avoid some categories of skin cancers, such as melanoma and focal cell carcinoma. The recognition and the classification of skin malignant growth in the beginning time is expensive and challenging. The deep learning architectures such as recurrent networks and convolutional neural networks (ConvNets) are developed in the past, which are proven appropriate for non-handcrafted extraction of complex features. To additional expand the efficiency of the ConvNet models, a cascaded ensembled network that uses an integration of ConvNet and handcrafted features based multi-layer perceptron is proposed in this work. This offered model utilizes the convolutional neural network model to mine non-handcrafted image features and colour moments and texture features as handcrafted features. It is demonstrated that accuracy of ensembled deep learning model is improved to 98.3% from 85.3% of convolutional neural network model.

https://ieeexplore.ieee.org/ielx7/6287639/9668973/09706441.pdf

Dermatologist-Level Classification of Skin Cancer Using Cascaded Ensembling of Convolutional Neural Network and Handcrafted Features Based Deep Neural Network

After lung cancer, breast cancer is the second leading cause of death in women. If breast cancer is detected early, mortality rates in women can be reduced. Because manual breast cancer diagnosis takes a long time, an automated system is required for early cancer detection. This paper proposes a new framework for breast cancer classification from ultrasound images that employs deep learning and the fusion of the best selected features. The proposed framework is divided into five major steps: (i) data augmentation is performed to increase the size of the original dataset for better learning of Convolutional Neural Network (CNN) models; (ii) a pre-trained DarkNet-53 model is considered and the output layer is modified based on the augmented dataset classes; (iii) the modified model is trained using transfer learning and features are extracted from the global average pooling layer; (iv) the best features are selected using two improved optimization algorithms known as reformed differential evaluation (RDE) and reformed gray wolf (RGW); and (v) the best selected features are fused using a new probability-based serial approach and classified using machine learning algorithms. The experiment was conducted on an augmented Breast Ultrasound Images (BUSI) dataset, and the best accuracy was 99.1%. When compared with recent techniques, the proposed framework outperforms them.

/pdf/breast-cancer-classification-from-ultrasound-images-using-2krct9jy.pdf

Breast Cancer Classification from Ultrasound Images Using Probability-Based Optimal Deep Learning Feature Fusion

Automatic recognition of human activities using wearable sensors remains a challenging problem due to high variability in inter-person gait and movements Moreover, finding the best on-body location for a wearable sensor is also critical though it provides valuable context information that can be used for accurate recognition This article addresses the problem of classifying motion signals generated by multiple wearable sensors for the recognition of human activity and localisation of the wearable sensors Unlike existing methods that used the raw accelerometer and gyroscope signals for extracting time and frequency-based features for activity inference, we propose to create frequency images for the raw signals and show this representation to be more robust The frequency image sequences are generated from the accelerometer and gyroscope signals from seven different body parts These frequency images serve as the input to our proposed two-stream Convolutional Neural Networks (CNN) for predicting the human activity and the location of the sensor generating the activity signal We show that the complementary information collected by both accelerometer and gyroscope sensors can be leveraged to develop an effective classifier that can accurately predict the performed human activity We evaluate the performance of the proposed method using the cross-subjects approach and show that it achieves an impressive F1-score of 090 on a publicly available real-world human activity dataset This performance is superior to that reported by another state-of-the-art method on the same dataset Moreover, we also experimented with the datasets from different body locations to predict the best position for the underlying task We show that shin and waist are the best places on the body for placing sensors and this could help other researchers to collect higher quality activity data We plan to publicly release the generated frequency images from all sensor positions and activities and our implementation code with the publication

/pdf/deep-human-activity-recognition-with-localisation-of-9nflwahcba.pdf

Deep Human Activity Recognition With Localisation of Wearable Sensors

Skin is one of the vital and well-known sensory organs in human physiology and due to various reasons, the abnormality in skin arises. Skin Melanoma (SM) is one of the medical crisis in humans and the untreated SM will cause various abnormalities, such as skin irritation, spreading the cancerous cells through the blood stream, etc. Efficient assessment of the SM is essential to identify the severity of the disease and hence the proposed work implemented a Convolutional-Neural-Network (CNN) based approach to support the automated SM examination. This work employed the VGG-SegNet scheme to extract the SM section from the Digital-Dermoscpy-Image (DDI). After the extraction, a relative assessment between the segmented SM and the Ground-Truth (GT) is executed and the essential performance indices are then computed. The proposed scheme is tested and validated using the benchmark ISIC2016 database and the average result attained with the proposed study helped to achieve a better values of Jaccard-Index, Dice, and Accuracy for the DDI with and without the artifacts. These results confirm that, proposed technique is significant in evaluating the clinical grader DDI.

Extraction of Abnormal Skin Lesion from Dermoscopy Image using VGG-SegNet

The brain abnormality is one of the major sicknesses in human’s health and the untreated brain defect will cause major illness. Ischemic stroke is one of the major medical emergencies and the timely diagnosis and treatment will save the patient from serious sickness. The proposed research employs the U-Net scheme to extort the Ischemic-Stoke-Lesion (ISL) from the brain MRI slices of ISLES2015 database. In this work, a pre-trained U-Net encoder-decoder system is employed to extort the ISL fragment from the chosen test image. After the extraction, a relative assessment is performed with the ground-truth available along with consequent test image. In this work, 20 patients’ images (20 patient x 25 slices = 500 images) are adopted for the assessment and the general result achieved with the executed methodology helped to achieve a better value of Jaccard (>90%), Dice (>95%) and Accuracy (>98%) on the considered image dataset.

U-Net Supported Segmentation of Ischemic-Stroke-Lesion from Brain MRI Slices

This paper addresses the problem of classifying motion signals acquired via wearable sensors for the recognition of human activity. Automatic and accurate classification of motion signals is important in facilitating the development of an effective automated health monitoring system for the elderlies. Thus, we gathered hip motion signals from two different waist mounted sensors and for each individual sensor, we converted the motion signal into spectral image sequence. We use these images as inputs to independently train two Convolutional Neural Networks (CNN), one for each of the generated image sequences from the two sensors. The outputs of the trained CNNs are then fused together to predict the final class of the human activity. We evaluate the performance of the proposed method using the cross-subjects testing approach. Our method achieves recognition accuracy (F1 score) of 0.87 on a publicly available real-world human activity dataset. This performance is superior to that reported by another state-of-the-art method on the same dataset.

/pdf/deep-human-activity-recognition-using-wearable-sensors-3dlkiz2de4.pdf

Deep human activity recognition using wearable sensors

We present a closed-loop unsupervised clustering method for motion vectors extracted from highly dynamic video scenes. Motion vectors are assigned to nonconvex homogeneous clusters characterizing direction, size and shape of regions with multiple independent activities. The proposed method is based on support vector clustering. Cluster labels are propagated over time via incremental learning. The proposed method uses a kernel function that maps the input motion vectors into a high-dimensional space to produce nonconvex clusters. We improve the mapping effectiveness by quantifying feature similarities via a blend of position and orientation affinities. We use the Quasiconformal Kernel Transformation to boost the discrimination of outliers. The temporal propagation of the clusters’ identities is achieved via incremental learning based on the concept of feature obsolescence to deal with appearing and disappearing features. Moreover, we design an online clustering performance prediction algorithm used as a feedback that refines the cluster model at each frame in an unsupervised manner. We evaluate the proposed method on synthetic data sets and real-world crowded videos and show that our solution outperforms state-of-the-art approaches.

/pdf/support-vector-motion-clustering-4tv0frqwyk.pdf

Isah A. Lawal

Papers

Deep Human Activity Recognition With Localisation of Wearable Sensors

Extraction of Abnormal Skin Lesion from Dermoscopy Image using VGG-SegNet

U-Net Supported Segmentation of Ischemic-Stroke-Lesion from Brain MRI Slices

Deep human activity recognition using wearable sensors

Support Vector Motion Clustering