The PASCAL Visual Object Classes Challenge

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.

Deep learning with convolutional neural networks for EEG decoding and visualization.

Hyperspectral remote sensing technology has advanced significantly in the past two decades. Current sensors onboard airborne and spaceborne platforms cover large areas of the Earth surface with unprecedented spectral, spatial, and temporal resolutions. These characteristics enable a myriad of applications requiring fine identification of materials or estimation of physical parameters. Very often, these applications rely on sophisticated and complex data analysis methods. The sources of difficulties are, namely, the high dimensionality and size of the hyperspectral data, the spectral mixing (linear and nonlinear), and the degradation mechanisms associated to the measurement process such as noise and atmospheric effects. This paper presents a tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing. In all topics, we describe the state-of-the-art, provide illustrative examples, and point to future challenges and research directions.

/pdf/hyperspectral-remote-sensing-data-analysis-and-future-1cxk3zgg98.pdf

Hyperspectral Remote Sensing Data Analysis and Future Challenges

For object detection, the two-stage approach (e.g., Faster R-CNN) has been achieving the highest accuracy, whereas the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, in this paper, we propose a novel single-shot based detector, called RefineDet, that achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. RefineDet consists of two inter-connected modules, namely, the anchor refinement module and the object detection module. Specifically, the former aims to (1) filter out negative anchors to reduce search space for the classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes the refined anchors as the input from the former to further improve the regression accuracy and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multitask loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO demonstrate that RefineDet achieves state-of-the-art detection accuracy with high efficiency. Code is available at https://github.com/sfzhang15/RefineDet.

/pdf/single-shot-refinement-neural-network-for-object-detection-j6fdptx7kt.pdf

Single-Shot Refinement Neural Network for Object Detection

Objective Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. Approach In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). Main results We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Significance Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

https://iopscience.iop.org/article/10.1088/1741-2552/aace8c/pdf

EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces

In this letter, we present a kernel-based nonlinear version of the adaptive subspace detector (ASD) that implicitly detects signals of interest in a high-dimensional (possibly infinite) feature space associated with a particular nonlinear mapping. In order to address the high dimensionality of the feature space, ASD is first implicitly formulated in the feature space, which is then converted into an expression in terms of kernel functions via the kernel trick property of the Mercer kernels. Experimental results based on simulated data and real hyperspectral imagery show that the proposed kernel-based ASD outperforms the conventional ASD and a nonlinear anomaly detector so called the kernel RX-algorithm.

Kernel adaptive subspace detector for hyperspectral imagery

Human-autonomy sensor fusion is an emerging technology with a wide range of applications, including object detection/recognition, surveillance, collaborative control, and prosthetics. For object detection, humans and computer-vision-based systems employ different strategies to locate targets, likely providing complementary information. However, little effort has been made in combining the outputs of multiple autonomous detectors and multiple human-generated responses. This paper presents a method for integrating several sources of human- and autonomy-generated information for rapid object detection tasks. Human electroencephalography (EEG) and button-press responses from rapid serial visual presentation (RSVP) experiments are fused with outputs from trained object detection algorithms. Three fusion methods—Bayesian, Dempster-Shafer, and Dynamic Dempster-Shafer—are implemented for comparison. Results demonstrate that fusion of these human classifiers with computer-vision-based detectors improves object detection accuracy over purely computer-vision-based detection (5% relative increase in mean average precision) and the best individual computer vision algorithm (28% relative increase in mean average precision). Computer vision fused with button press response and/or the XDAWN + Bayesian Linear Discriminant Analysis neural classifier provides considerable improvement, while computer vision fused with other neural classifiers provides little or no improvement. Of the three fusion methods, Dynamic Dempster-Shafer Theory (DDST) Fusion exhibits the greatest performance in this application.

Human-autonomy sensor fusion for rapid object detection

In this paper, a novel framework of sparse kernel learning for support vector data description (SVDD) based anomaly detection is presented. By introducing 0-1 control variables to original features in the input space, sparse feature selection for anomaly detection is modeled as a mixed integer programming problem. Due to the prohibitively high computational complexity, it is relaxed into a quadratically constrained linear programming (QCLP) problem. The QCLP problem can then be practically solved by using an iterative optimization method, in which multiple subsets of features are iteratively found as opposed to a single subset. However, when a nonlinear kernel such as Gaussian radial basis function kernel, associated with an infinite-dimensional reproducing kernel Hilbert space (RKHS) is used in the QCLP-based iterative optimization, it is impractical to find optimal subsets of features due to a large number of possible combinations of the original features. To tackle this issue, a feature map called the empirical kernel map, which maps data points in the input space into a finite space called the empirical kernel feature space (EKFS), is used in the proposed work. The QCLP-based iterative optimization problem is solved in the EKFS instead of in the input space or the RKHS. This is possible because the geometrical properties of the EKFS and the corresponding RKHS remain the same. Now, an explicit nonlinear exploitation of the data in a finite EKFS is achievable, which results in optimal feature ranking. Comprehensive experimental results on three hyperspectral images and several machine learning datasets show that our proposed method can provide improved performance over the current state-of-the-art techniques.

Sparse kernel learning-based feature selection for anomaly detection

We propose adaptive anomaly detectors that find materials whose spectral characteristics are substantially different from those of the neighboring materials.The target spectral vectors are assumed to have different statistical characteristics from the background vectors. In order to detect anomalies we use a dual rectangular window that separates the local area into two regions-- the inner window region (IWR) and outer window region (OWR). The statistical spectral differences between the IWR and OWR is exploited by generating subspace projection vectors onto which the IWR and OWR vectors are projected. Anomalies are detected if the projection separation between the IWR and OWR vectors is greater than a predefined threshold. Four different methods are used to produce the subspace projection vectors. The four proposed anomaly detectors have been applied to HYDICE (HYperspectral Digital Imagery Collection Experiment)images and the detection performance for each method has been evaluated.

Dual-window-based anomaly detection for hyperspectral imagery

In this paper a non-linear matched filter is introduced for target detection in hyperspectral imagery which is implemented by using the ideas in kernel-based learning theory. The proposed non-linear matched filter exploits the notion that performing matched filtering in a non-linear feature space of high dimensionality increases the probability of detection. Defining matched filter in a kernel feature space is equivalent to a non-linear matched filter in the original input space which allows the higher order correlation between the spectral bands to be exploited. It is also shown that the non-linear matched filter can easily be implemented using the ideas of kernel functions. The kernel version of the non-linear matched filter is implemented and simulation results are shown to outperform the linear version.

Heesung Kwon

Papers

Kernel adaptive subspace detector for hyperspectral imagery

Human-autonomy sensor fusion for rapid object detection

Sparse kernel learning-based feature selection for anomaly detection

Dual-window-based anomaly detection for hyperspectral imagery

Hyperspectral Target Detection Using Kernel Spectral Matched Filter