An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

With the growing volume of online information, recommender systems have been an effective strategy to overcome information overload. The utility of recommender systems cannot be overstated, given their widespread adoption in many web applications, along with their potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also to the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. The field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning-based recommender systems. More concretely, we provide and devise a taxonomy of deep learning-based recommendation models, along with a comprehensive summary of the state of the art. Finally, we expand on current trends and provide new perspectives pertaining to this new and exciting development of the field.

Deep Learning Based Recommender System: A Survey and New Perspectives

Context Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain-computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About [Formula: see text] of the studies used convolutional neural networks (CNNs), while [Formula: see text] used recurrent neural networks (RNNs), most often with a total of 3-10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was [Formula: see text] across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

https://iopscience.iop.org/article/10.1088/1741-2552/ab260c/pdf

Deep learning-based electroencephalography analysis: a systematic review.

With the ever-growing volume of online information, recommender systems have been an effective strategy to overcome such information overload. The utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. Evidently, the field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the field.

Deep Learning based Recommender System: A Survey and New Perspectives.

Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and comprehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. For each category, we present we also present the advantages and limitations and discuss the computational complexity of the techniques in real application domains. Finally, we outline open issues in research and challenges faced while adopting these techniques.

Deep Learning for Anomaly Detection: A Survey.

There is a growing interest in identifying inorganic material affinity classes for peptide sequences due to the development of bionanotechnology and its wide applications. In particular, a selective model capable of learning cross-material affinity patterns can help us design peptide sequences with desired binding selectivity for one inorganic material over another. However, as a newly emerging topic, there are several distinct challenges of it that limit the performance of many existing peptide sequence classification algorithms. In this paper, we propose a novel framework to identify affinity classes for peptide sequences across inorganic materials. After enlarging our dataset by simulating peptide sequences, we use a context learning based method to obtain the vector representation of each amino acid and each peptide sequence. By analyzing the structure and affinity class of each peptide sequence, we are able to capture the semantics of amino acids and peptide sequences in a vector space. At the last step we train our classifier based on these vector features and the heuristic rules. The construction of our models gives us the potential to overcome the challenges of this task and the empirical results show the effectiveness of our models.

Identifying inorganic material affinity classes for peptide sequences based on context learning

FAT-RE: A faster dependency-free model for relation extraction

W-CTC: a Connectionist Temporal Classification Loss with Wild Cards

Missing data, which commonly appears in multivariate time series, has been widely recognized as a key challenge in time series analysis. Many commonly used imputation methods either ignore the temporal dependencies of time series data, or do not adequately utilize the relationships among variables. State-ofthe-art methods on time series imputation are built on Recurrent Neural Networks (RNNs), which utilize the historical information to estimate current values sequentially. However, RNNs rely heavily on the output of nearby timestamps, which may lead to important information lost for long sequences. Moreover, individual variables typically present different dynamics and missingness patterns, which is neglected by the global RNN hidden states. In this paper, we propose an imputation framework to learn both global and local dependencies of multivariate time series, as well as a multi-dimensional self-attention to learn capture distant correlations across both time and feature. Extensive experiments show that the proposed framework outperforms the state-of-the-art methods in the imputation task, and benefits the downstream task.

GLIMA: Global and Local Time Series Imputation with Multi-directional Attention Learning

Stochastic gradient descent (SGD) augmented with various momentum variants (e.g. heavy ball momentum (SHB) and Nesterov's accelerated gradient (NAG)) has been the default optimizer for many learning tasks. Tuning the optimizer's hyperparameters is arguably the most time-consuming part of model training. Many new momentum variants, despite their empirical advantage over classical SHB/NAG, introduce even more hyperparameters to tune. Automating the tedious and error-prone tuning is essential for AutoML. This paper focuses on how to efficiently tune a large class of multistage momentum variants to improve generalization. We use the general formulation of quasi-hyperbolic momentum (QHM) and extend "constant and drop'', the widespread learning rate α scheduler where α is set large initially and then dropped every few epochs, to other hyperparameters (e.g. batch size b, momentum parameter β, instant discount factor ν). Multistage QHM is a unified framework which covers a large family of momentum variants as its special cases (e.g. vanilla SGD/SHB/NAG). Existing works mainly focus on scheduling α's decay, while multistage QHM allows additional varying hyperparameters such as b, β, and ν, and demonstrates better generalization ability than only tuning α. Our tuning strategies have rigorous justifications rather than a blind trial-and-error. We theoretically prove why our tuning strategies could improve generalization. We also show the convergence of multistage QHM for general nonconvex objective functions. Our strategies simplify the tuning process and beat competitive optimizers in test accuracy empirically.

Guangxu Xun

Papers

Identifying inorganic material affinity classes for peptide sequences based on context learning

FAT-RE: A faster dependency-free model for relation extraction

W-CTC: a Connectionist Temporal Classification Loss with Wild Cards

GLIMA: Global and Local Time Series Imputation with Multi-directional Attention Learning

A Stagewise Hyperparameter Scheduler to Improve Generalization