Emotional state recognition of a speaker is a difficult task for machine learning algorithms which plays an important role in the field of speech emotion recognition (SER). SER plays a significant role in many real-time applications such as human behavior assessment, human-robot interaction, virtual reality, and emergency centers to analyze the emotional state of speakers. Previous research in this field is mostly focused on handcrafted features and traditional convolutional neural network (CNN) models used to extract high-level features from speech spectrograms to increase the recognition accuracy and overall model cost complexity. In contrast, we introduce a novel framework for SER using a key sequence segment selection based on redial based function network (RBFN) similarity measurement in clusters. The selected sequence is converted into a spectrogram by applying the STFT algorithm and passed into the CNN model to extract the discriminative and salient features from the speech spectrogram. Furthermore, we normalize the CNN features to ensure precise recognition performance and feed them to the deep bi-directional long short-term memory (BiLSTM) to learn the temporal information for recognizing the final state of emotion. In the proposed technique, we process the key segments instead of the whole utterance to reduce the computational complexity of the overall model and normalize the CNN features before their actual processing, so that it can easily recognize the Spatio-temporal information. The proposed system is evaluated over different standard dataset including IEMOCAP, EMO-DB, and RAVDESS to improve the recognition accuracy and reduce the processing time of the model, respectively. The robustness and effectiveness of the suggested SER model is proved from the experimentations when compared to state-of-the-art SER methods with an achieve up to 72.25%, 85.57%, and 77.02% accuracy over IEMOCAP, EMO-DB, and RAVDESS dataset, respectively.

/pdf/clustering-based-speech-emotion-recognition-by-incorporating-3wrsb2ojjx.pdf

Clustering-Based Speech Emotion Recognition by Incorporating Learned Features and Deep BiLSTM

Speech is the most dominant source of communication among humans, and it is an efficient way for human–computer interaction (HCI) to exchange information. Nowadays, speech emotion recognition (SER) is an active research area that plays a crucial role in real-time applications. In this era, the SER system has lacked real-time speech processing. To address this problem, we propose an end-to-end real-time SER model that is based on a one-dimensional dilated convolutional neural network (DCNN). Our model used a multi-learning strategy to parallel extract spatial salient emotional features and learn long term contextual dependencies from the speech signals. We used residual blocks with a skip connection (RBSC) module , in order to find a correlation, the emotional cues, and the sequence learning (Seq_L) module , to learn the long term contextual dependencies in the input features. Furthermore, we used a fusion layer to concatenate these learned features for the final emotion recognition task. Our model structure is quite simple, and it is capable of automatically learning salient discriminative features from the speech signals. We evaluated our model using benchmark IEMOCAP and EMO-DB datasets and obtained a high recognition accuracy, which were 73% and 90%, respectively. The experimental results indicated the significance and the efficiency of our proposed model have shown excessive assistance with the implementation of a real-time SER system. Hence, our model is capable of processing original speech signals for the emotion recognition that utilizes lightweight dilated CNN architecture that implements the multi-learning trick (MLT) approach.

MLT-DNet: Speech emotion recognition using 1D dilated CNN based on multi-learning trick approach

With the rapid development of Internet of Things (IoT) technology, location-based services have been widely applied in the construction of smart cities. Satellite-based location services have been utilized in outdoor environments, but they are not suitable for indoor technology due to the absence of global positioning system (GPS) signal. Therefore, many indoor localization technologies and systems have emerged by utilizing many other signals. In particular, fingerprinting localization has recently garnered attention because its promising performance. In this work, we aim to study recent indoor localization technologies and systems based on various fingerprints, which use machine learning and intelligent algorithms. We also present the architecture of intelligent localization. The development of indoor localization technology should have the ability of self-adaptation and self-learning in the future. And the architecture shows how to make localization become more “smart” by advanced techniques. The state-of-the-art localization systems’ working principles are summarized and compared in terms of their localization accuracy, latency, energy consumption, complexity, and robustness. We also discuss the challenges of existing indoor localization technologies, potential solutions to these challenges, and possible improvement measures.

Indoor Intelligent Fingerprint-Based Localization: Principles, Approaches and Challenges

Speech emotion recognition using fusion of three multi-task learning-based classifiers: HSF-DNN, MS-CNN and LLD-RNN

Artificial intelligence (AI) and machine learning (ML) are employed to make systems smarter. Today, the speech emotion recognition (SER) system evaluates the emotional state of the speaker by investigating his/her speech signal. Emotion recognition is a challenging task for a machine. In addition, making it smarter so that the emotions are efficiently recognized by AI is equally challenging. The speech signal is quite hard to examine using signal processing methods because it consists of different frequencies and features that vary according to emotions, such as anger, fear, sadness, happiness, boredom, disgust, and surprise. Even though different algorithms are being developed for the SER, the success rates are very low according to the languages, the emotions, and the databases. In this paper, we propose a new lightweight effective SER model that has a low computational complexity and a high recognition accuracy. The suggested method uses the convolutional neural network (CNN) approach to learn the deep frequency features by using a plain rectangular filter with a modified pooling strategy that have more discriminative power for the SER. The proposed CNN model was trained on the extracted frequency features from the speech data and was then tested to predict the emotions. The proposed SER model was evaluated over two benchmarks, which included the interactive emotional dyadic motion capture (IEMOCAP) and the berlin emotional speech database (EMO-DB) speech datasets, and it obtained 77.01% and 92.02% recognition results. The experimental results demonstrated that the proposed CNN-based SER system can achieve a better recognition performance than the state-of-the-art SER systems.

https://www.mdpi.com/1424-8220/20/18/5212/pdf

Deep-Net: A Lightweight CNN-Based Speech Emotion Recognition System Using Deep Frequency Features

Speech emotion recognition is a vital and challenging task that the feature extraction plays a significant role in the SER performance. With the development of deep learning, we put our eyes on the structure of end-to-end and authenticate the algorithm that is extraordinary effective. In this paper, we introduce a novel architecture ADRNN (dilated CNN with residual block and BiLSTM based on the attention mechanism) to apply for the speech emotion recognition which can take advantage of the strengths of diverse networks and overcome the shortcomings of utilizing alone, and are evaluated in the popular IEMOCAP database and Berlin EMODB corpus. Dilated CNN can assist the model to acquire more receptive fields than using the pooling layer. Then, the skip connection can keep more historic info from the shallow layer and BiLSTM layer are adopted to learn long-term dependencies from the learned local features. And we utilize the attention mechanism to enhance further extraction of speech features. Furthermore, we improve the loss function to apply softmax together with the center loss that achieves better classification performance. As emotional dialogues are transformed of the spectrograms, we pick up the values of the 3-D Log-Mel spectrums from raw signals and put them into our proposed algorithm and obtain a notable performance to get the 74.96% unweighted accuracy in the speaker-dependent and the 69.32% unweighted accuracy in the speaker-independent experiment. It is better than the 64.74% from previous state-of-the-art methods in the spontaneous emotional speech of the IEMOCAP database. In addition, we propose the networks that achieve recognition accuracies of 90.78% and 85.39% on Berlin EMODB of speaker-dependent and speaker-independent experiment respectively, which are better than the accuracy of 88.30% and 82.82% obtained by previous work. For validating the robustness and generalization, we also make an experiment for cross-corpus between above databases and get the preferable 63.84% recognition accuracy in final.

/pdf/speech-emotion-recognition-from-3d-log-mel-spectrograms-with-fpm0b8dg8e.pdf

Speech Emotion Recognition From 3D Log-Mel Spectrograms With Deep Learning Network

In the indoor environment, due to weak receiver signals, environmental noise, multipath interference, and non-line-of-sight propagation, the traditional positioning algorithms based on received signal strength indication (RSSI) have many problems, such as inaccurate positioning results, great dependence on the signal propagation path loss model, and high time and labor costs. This paper studied the wireless indoor positioning algorithm based on neural network. A weighted median-Gaussian filtering method is proposed to preprocess RSSI and establish a location fingerprint database. An indoor positioning algorithm based on an improved fast clustering algorithm combined with a Levenberg–Marquardt (LM) algorithm is proposed. The improved clustering algorithm is used to design the network structure, initialize the number of radial basis function (RBF) neurons, find the local density peak as the cluster center to achieve rapid clustering of samples, and adjust the parameters of the kernel function of the hidden layer neurons. And the LM algorithm is used for numerical optimization. In order to verify the performance of the algorithm, positioning experiments are performed in the library. The error rate was reduced by 26.2% compared with the RBF network. The positioning results data confirm the effectiveness and applicability of the proposed algorithm.

/pdf/indoor-positioning-of-rbf-neural-network-based-on-improved-4uiscecd07.pdf

Indoor Positioning of RBF Neural Network Based on Improved Fast Clustering Algorithm Combined With LM Algorithm

The traditional 2D panoramic view has the problem of small imaging range, low utilization of original image and serious stretching of the foreground. This paper designs a 3D panoramic viewing system. The real-time performance of the panoramic system requires a large number of parallel image transformations. The general-purpose processor CPU is incapable. This paper designs an embedded GPU solution. The core computing module is based on NXP's i.MX6Q application processor and builds an embedded Linux system. Using V4L2 to capture images to reduce memory and CPU usage, and use OpenGL ES shaders to process image transformations in parallel at high speed. Because it is in a 3D environment, it is impossible to cover all images through a still viewpoint like a 2D look-around. In order to roam the 3D scene, this paper proposes to design a virtual camera system. Using the virtual camera to roam the 3D environment, realize the virtual viewpoint in the 3D scene, traverse each scene in the grid, eliminate the dead angle of the field of view, and finally realize the 3D panoramic view, and verified on the embedded hardware platform, which can clearly roam every image on the surface. And recorded the video of 3-D looping. Experiments show that the 3D panoramic viewing system proposed in this paper has excellent performance and good effects. It has a good real-time performance on the image display, and the response speed is increased by 5%. The main operation is undertaken by the GPU and only takes a small number of CPU resources. The proposed virtual camera system can better realize the 3D panoramic view.

/pdf/embedded-gpu-3d-panoramic-viewing-system-based-on-virtual-29dfqqpgz1.pdf

Embedded GPU 3D Panoramic Viewing System Based on Virtual Camera Roaming 3D Environment

A new method of P300 potential detection is given in the paper. Firstly, by creating the signal differential profile, a single electrode choose method of P300 is given. Secondly, aiming at the shortcoming of traditional smoothing method, adaptive filtering method is applied to the signal preprocessing of P300 potential for the first time in this paper. This method takes the average signal of the original data as the reference signal and can effectively eliminate all kinds of noise components mixed in the P300 potential. Furthermore, different from traditional time-domain feature extraction method of P300 potential, the paper proposes a feature extraction method based on short-time Fourier transform, which fully extracts the frequency-domain features of P300 potential while retaining the time-domain features. Finally, a convolutional neural network classifier is designed and obtains an average classification accuracy of 89.81% in the different average times of reference signals which was about 5% higher than traditional methods.

P300 Detection with Adaptive Filtering and EEG Spectrogram Graph

Introduction Digital health interventions are an effective way to treat depression, but it is still largely unclear how patients’ individual symptoms evolve dynamically during such treatments. Data-driven forecasts of depressive symptoms would allow to greatly improve the personalisation of treatments. In current forecasting approaches, models are often trained on an entire population, resulting in a general model that works overall, but does not translate well to each individual in clinically heterogeneous, real-world populations. Model fairness across patient subgroups is also frequently overlooked. Personalised models tailored to the individual patient may therefore be promising. Methods We investigate different personalisation strategies using transfer learning, subgroup models, as well as subject-dependent standardisation on a newly-collected, longitudinal dataset of depression patients undergoing treatment with a digital intervention (N=65 patients recruited). Both passive mobile sensor data as well as ecological momentary assessments were available for modelling. We evaluated the models’ ability to predict symptoms of depression (Patient Health Questionnaire-2; PHQ-2) at the end of each day, and to forecast symptoms of the next day. Results In our experiments, we achieve a best mean-absolute-error (MAE) of 0.801 (25% improvement) for predicting PHQ-2 values at the end of the day with subject-dependent standardisation compared to a non-personalised baseline (MAE=1.062). For one day ahead-forecasting, we can improve the baseline of 1.539 by 12% to a MAE of 1.349 using a transfer learning approach with shared common layers. In addition, personalisation leads to fairer models at group-level. Discussion Our results suggest that personalisation using subject-dependent standardisation and transfer learning can improve predictions and forecasts, respectively, of depressive symptoms in participants of a digital depression intervention. We discuss technical and clinical limitations of this approach, avenues for future investigations, and how personalised machine learning architectures may be implemented to improve existing digital interventions for depression.

Tianhao Yan

Papers

Speech Emotion Recognition From 3D Log-Mel Spectrograms With Deep Learning Network

Indoor Positioning of RBF Neural Network Based on Improved Fast Clustering Algorithm Combined With LM Algorithm

Embedded GPU 3D Panoramic Viewing System Based on Virtual Camera Roaming 3D Environment

P300 Detection with Adaptive Filtering and EEG Spectrogram Graph

Personalised depression forecasting using mobile sensor data and ecological momentary assessment