Sensor-based and vision-based human activity recognition: A comprehensive survey

The recent proliferation of fake portrait videos poses direct threats on society, law, and privacy [1]. Believing the fake video of a politician, distributing fake pornographic content of celebrities, fabricating impersonated fake videos as evidence in courts are just a few real world consequences of deep fakes. We present a novel approach to detect synthetic content in portrait videos, as a preventive solution for the emerging threat of deep fakes. In other words, we introduce a deep fake detector. We observe that detectors blindly utilizing deep learning are not effective in catching fake content, as generative models produce formidably realistic results. Our key assertion follows that biological signals hidden in portrait videos can be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content. To prove and exploit this assertion, we first engage several signal transformations for the pairwise separation problem, achieving 99.39% accuracy. Second, we utilize those findings to formulate a generalized classifier for fake content, by analyzing proposed signal transformations and corresponding feature sets. Third, we generate novel signal maps and employ a CNN to improve our traditional classifier for detecting synthetic content. Lastly, we release an "in the wild" dataset of fake portrait videos that we collected as a part of our evaluation process. We evaluate FakeCatcher on several datasets, resulting with 96%, 94.65%, 91.50%, and 91.07% accuracies, on Face Forensics [2], Face Forensics++ [3], CelebDF [4], and on our new Deep Fakes Dataset respectively. In addition, our approach produces a significantly superior detection rate against baselines, and does not depend on the source, generator, or properties of the fake content. We also analyze signals from various facial regions, under image distortions, with varying segment durations, from different generators, against unseen datasets, and under several dimensionality reduction techniques.

FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

Epilepsy is a health problem that seriously affects the quality of humans for many years. Therefore, it is important to accurately analyze and recognize epilepsy based on EEG signals, and for a long time, researchers have attempted to extract new features from the signals for epilepsy recognition. However, it is very difficult to select useful features from a large number of them in this diagnostic application. As the development of artificial intelligence progresses, unsupervised feature learning based on the deep learning model can obtain features that can better describe identified objects from unlabeled data. In this paper, the deep convolution network and autoencoders-based model, named as AE-CDNN, is constructed in order to perform unsupervised feature learning from EEG in epilepsy. We extract features by AE-CDNN model and classify the features based on two public EEG data sets. Experimental results showed that the classification results of features obtained by AE-CDNN are more optimal than features obtained by principal component analysis and sparse random projection. Using several common classifiers to classify features obtained by AE-CDNN model results in high accuracy and not inferior to the research results from most recent studies. The results also showed that the features of AE-CDNN model are clear, effective, and easy to learn. These features can speed up the convergence and reduce the training times of classifiers. Therefore, the AE-CDNN model can be effectively applied to feature extraction of EEG in epilepsy.

Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals

Achieving the goal of detecting seizure activity automatically using electroencephalogram (EEG) signals is of great importance and significance for the treatment of epileptic seizures. To realize this aim, a newly-developed time-frequency analytical algorithm, namely local mean decomposition (LMD), is employed in the presented study. LMD is able to decompose an arbitrary signal into a series of product functions (PFs). Primarily, the raw EEG signal is decomposed into several PFs, and then the temporal statistical and non-linear features of the first five PFs are calculated. The features of each PF are fed into five classifiers, including back propagation neural network (BPNN), K-nearest neighbor (KNN), linear discriminant analysis (LDA), un-optimized support vector machine (SVM) and SVM optimized by genetic algorithm (GA-SVM), for five classification cases, respectively. Confluent features of all PFs and raw EEG are further passed into the high-performance GA-SVM for the same classification tasks. Experimental results on the international public Bonn epilepsy EEG dataset show that the average classification accuracy of the presented approach are equal to or higher than 98.10% in all the five cases, and this indicates the effectiveness of the proposed approach for automated seizure detection.

LMD Based Features for the Automatic Seizure Detection of EEG Signals Using SVM

Human emotion recognition plays a vital role in psychology, psycho-physiology and human machine interface (HMI) design. Electroencephalogram (EEG) reflects the internal emotional state changes of the subject compared to other conventional methods (face recognition, gestures, speech, etc). In this work, EEG signals are collected using 62 channels from 20 subjects in the age group of 21~39 years for determining discrete emotions. Audio-visual stimuli (video clips) is used for inducing five different emotions (happy, surprise, fear, disgust, neutral). EEG signals are preprocessed through Butterworth 4th order filter with a cut off frequency of 0.5 Hz-60 Hz and smoothened using Surface Laplacian filter. EEG signals are framed into a short time duration of 5s and two statistical features (spectral centroid and spectral entropy) in four frequency bands namely alpha (8 Hz-16 Hz), beta (16 Hz-32 Hz), gamma (32 Hz-60 Hz) and alpha to gamma (8 Hz-60 Hz) are extracted using Fast Fourier Transform (FFT). These features are mapped into the corresponding emotions using two simple classifiers such as K Nearest Neighbor(KNN) and Probabilistic Neural Network (PNN). In this work, KNN outperforms PNN by offering the maximum mean classification accuracy of 91.33 % on beta band. This experimental results indicates the short time duration of EEG signals is highly essential for detecting the emotional state changes of the subjects.

Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT)

In this paper, a novel hierarchical multi-class SVM (H-MSVM) with extreme learning machine (ELM) as kernel is proposed to classify electroencephalogram (EEG) signals for epileptic seizure detection A clinical EEG benchmark dataset having five classes, obtained from Department of Epileptology, Medical Center, University of Bonn, Germany, is considered in this work for validating the clinical utilities Wavelet transform-based features such as statistical values, largest Lyapunov exponent, and approximate entropy are extracted and considered as input to the classifier In general, SVM provides better classification accuracy, but takes more time for classification and also there is scope for a new multi-classification scheme In order to mitigate the problem of SVM, a novel multi-classification scheme based on hierarchical approach, with ELM kernel, is proposed Experiments have been conducted using holdout and cross-validation methods on the entire dataset Metrics namely classification accuracy, sensitivity, specificity, and execution time are computed to analyze the performance of the proposed work The results show that the proposed H-MSVM with ELM kernel is efficient in terms of better classification accuracy at a lesser execution time when compared to ANN, various multi-class SVMs, and other research works which use the same clinical dataset

Hierarchical multi-class SVM with ELM kernel for epileptic EEG signal classification

In this paper, a new epileptic seizure detection method using fuzzy-rules-based sub-band specific features and layered directed acyclic graph support vector machine (LDAG-SVM) is proposed for classification of electroencephalogram (EEG) signals. Wavelet transformation is used to decompose the input EEG signals into various sub-bands. The nonlinear features, namely approximate entropy, largest Lyapunov exponent and correlation dimension, are extracted from each sub-band. In this proposed work, sub-band specific feature subset that is reduced in size and capable of discriminating samples is selected by employing fuzzy rules. For classification purpose, a new LDAG-SVM is used for detecting epileptic seizure. Every sub-band has its own characteristics. If appropriate features which characterize the specific sub-band are selected, then the classification accuracy is improved and computational complexity is reduced. The important advantage of the fuzzy logic is its close relation to human thinking. Due to the lengthy record and intra-professional variability, automation of epileptic detection is inevitable. Fuzzy rules are the natural choice of employing human expertise to build machine learning system. Performances of the proposed methods are evaluated using two different benchmark EEG datasets, namely Bonn and CHB-MIT. The performance measures such as classification accuracy, sensitivity, specificity, execution time and receiver operating characteristics are used to measure and analyze the performances of the proposed classifier. The proposed LDAG-SVM with fuzzy-rules-based selected sub-band specific features provides better performance in terms of improved classification accuracy with reduced execution time compared to existing methods.

Epileptic seizure detection using fuzzy-rules-based sub-band specific features and layered multi-class SVM

Electroencephalographms (EEGs) are records of brain electrical activity. It is an indispensable tool for diagnosing neurological diseases, such as epilepsy. Wavelet transform (WT) is an effective tool for analysis of non-stationary signal, such as EEGs. Wavelet analysis is used to decompose the EEG into delta, theta, alpha, beta, and gamma sub-bands. Lyapunov exponent is used to quantify the nonlinear chaotic dynamics of the signal‥ Furthermore, the distinct states of brain activity had different chaotic dynamics quantified by nonlinear invariant measures such as Lyapunov exponents. The probabilistic neural network (PNN) and radial basis function neural network were tested and also their performance of classification rate was evaluated using benchmark dataset. Decision making was performed in two stages: feature extraction by computing the Lyapunov exponents, Wavelet Coefficients and classification using the classifiers trained on the extracted features. Our research demonstrated that the Lyapunov exponents and Wavelet Coefficients are the features which well represent the EEG signals and the multi-class SVM and PNN trained on these features achieved high classification accuracies such as 96% and 94%.

Lyapunov features based EEG signal classification by multi-class SVM

(OELM) is proposed for the classification of EEG signals with the emphasis on epileptic seizure detection. OELM is an effective learning algorithm of single-hidden layer feed-forward neural networks. It requires setting the number of hidden neurons and the activation function. Adjustment in the input weights and hidden layer's biases are not needed during the implementation of the algorithm, and only one optimal solution is produced. This makes the OELM a valuable tool for the applications that need small response time and provide a good accuracy. The features such as energy, entropy, maximum value, minimum value, mean value and standard deviation of wavelet coefficients are used to represent the time frequency distribution of the EEG signals in each sub-band of the Wavelet Transformation. We have compared the proposed classifier with other traditional classifiers by evaluating it with the benchmark EEG dataset. It is found that the performance of the proposed OELM with Wavelet based statistical features is better in terms of training time and classification accuracy. An accuracy of 94% for classifying the epileptic EEG signals is achieved and needs less training time compared with SVM.

An Optimized Extreme Learning Machine for Epileptic Seizure Detection

In this paper, we have proposed a novel wavelet based CSI feature and a novel Adaptive Multi-Class Support Vector Machine (SVM) for the multi-class electroencephalogram (EEG) signals classification with the emphasis on epileptic seizure detection. The purpose was to determine an optimum classification scheme for this problem and also to infer clues about the extracted features. Wavelets have played an important role in biomedical signal processing for its ability to capture localized spatial-frequency information of EEG signal. CSI is used to develop a normalized index which state the maximum difference between the seizure and non-seizure states between the frequency range of 1-30Hz. The adaptive MSVM works well for high dimensional, multi-class data streams. Decision making was performed in two stages: feature extraction by computing the Combined Seizure Index and classification using the classifiers trained on the extracted features. We have compared the adaptive MSVM with the benchmark EEG dataset. Our experimental results show that the adaptive MSVM with wavelet based features which will represent the EEG signals and the classification methods trained on these features achieved high classification accuracies with better false rate and sensitivity.

A. S. Muthanantha Murugavel

Papers

Hierarchical multi-class SVM with ELM kernel for epileptic EEG signal classification

Epileptic seizure detection using fuzzy-rules-based sub-band specific features and layered multi-class SVM

Lyapunov features based EEG signal classification by multi-class SVM

An Optimized Extreme Learning Machine for Epileptic Seizure Detection

Combined Seizure Index with Adaptive Multi-Class SVM for epileptic EEG classification