The PASCAL Visual Object Classes Challenge

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.

Deep learning with convolutional neural networks for EEG decoding and visualization.

Hyperspectral remote sensing technology has advanced significantly in the past two decades. Current sensors onboard airborne and spaceborne platforms cover large areas of the Earth surface with unprecedented spectral, spatial, and temporal resolutions. These characteristics enable a myriad of applications requiring fine identification of materials or estimation of physical parameters. Very often, these applications rely on sophisticated and complex data analysis methods. The sources of difficulties are, namely, the high dimensionality and size of the hyperspectral data, the spectral mixing (linear and nonlinear), and the degradation mechanisms associated to the measurement process such as noise and atmospheric effects. This paper presents a tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing. In all topics, we describe the state-of-the-art, provide illustrative examples, and point to future challenges and research directions.

/pdf/hyperspectral-remote-sensing-data-analysis-and-future-1cxk3zgg98.pdf

Hyperspectral Remote Sensing Data Analysis and Future Challenges

For object detection, the two-stage approach (e.g., Faster R-CNN) has been achieving the highest accuracy, whereas the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, in this paper, we propose a novel single-shot based detector, called RefineDet, that achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. RefineDet consists of two inter-connected modules, namely, the anchor refinement module and the object detection module. Specifically, the former aims to (1) filter out negative anchors to reduce search space for the classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes the refined anchors as the input from the former to further improve the regression accuracy and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multitask loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO demonstrate that RefineDet achieves state-of-the-art detection accuracy with high efficiency. Code is available at https://github.com/sfzhang15/RefineDet.

/pdf/single-shot-refinement-neural-network-for-object-detection-j6fdptx7kt.pdf

Single-Shot Refinement Neural Network for Object Detection

Objective Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. Approach In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). Main results We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Significance Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

https://iopscience.iop.org/article/10.1088/1741-2552/aace8c/pdf

EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces

In this paper, an algorithm to randomly select feature subspaces for hyperspectral image classification using the principle of coalition game theory (CGT) is presented. The feature selection algorithms associated with nonlinear kernel-based support vector machines (SVM) are either NP-hard or greedy and hence, not very optimal. To deal with this problem, a metric based on the principles of CGT called Shapely value (SV) and a sampling approximation is used to determine the contributions of individual features toward the classification task. Feature subsets are randomly drawn from a probability distribution function (pdf) generated using normalized SVs of the individual features. These feature subsets are then used to build kernels corresponding to individual weak classifiers in the sparse kernel-based ensemble learning (SKEL) framework. By weighting the kernels optimally and sparsely, a small number of useful subsets of features are selected which improve the generalization performance of the ensemble classifier. The algorithm is applied on real hyperspectral datasets, and the results are presented in the paper.

Coalition Game Theory-Based Feature Subspace Selection for Hyperspectral Classification

In this paper, we present a kernel-based nonlinear version of the adaptive subspace detector (ASD) that detects signals of interest in a high dimensional (possibly infinite) feature space associated with a certain nonlinear mapping. In order to address the high dimensionality of the feature space, ASD is first implicitly formulated in the feature space which is then converted into an expression in terms of kernel functions via the kernel trick of the Mercer kernels. The proposed kernel-based ASD (KASD) exploits the nonlinear correlations between the spectral bands that is ignored by the conventional ASD. Experimental results based on the given hyperspectral image show that the proposed KASD outperforms the conventional ASD.

Kernel adaptive subspace detector for hyperspectral target detection

In this paper, we present a kernel-based nonlinear version of canonical correlation analysis (CCA), 
so called kernel canonical correlation analysis (KCCA), for 
hyperspectral anomaly detection applications. CCA only measures linear dependency 
between two sets of signal vectors (target and background) ignoring higher order correlations 
crucial for distinguishing between man-made objects and background clutter. 
In order to exploit nonlinear correlations we implicitly map the two 
sets of data into a high dimensional feature space where correlations of nonlinear features 
extracted from the original data are exploited by a kernel function. 
A generalized eigenproblem is then formulated for KCCA. In this paper, both CCA and KCCA are applied 
to real hyperspectral images and detection performance of CCA and KCCA are compared to the 
well-known RX anomaly detection algorithm.

Kernel canonical correlation analysis for hyperspectral anomaly detection

Pixel-wise classification in remote sensing identifies entities in large-scale satellite-based images at the pixel level. Few fully annotated large-scale datasets for pixel-wise classification exist due to the challenges of annotating individual pixels. Training data scarcity inevitably ensues from the annotation challenge, leading to overfitting classifiers and degraded classification performance. The lack of annotated pixels also necessarily results in few hard examples of various entities critical for generating a robust classification hyperplane. To overcome the problem of the data scarcity and lack of hard examples in training, we introduce a two-step hard example generation (HEG) approach that first generates hard example candidates and then mines actual hard examples. In the first step, a generator that creates hard example candidates is learned via the adversarial learning framework by fooling a discriminator and a pixel-wise classification model at the same time. In the second step, mining is performed to build a fixed number of hard examples from a large pool of real and artificially generated examples. To evaluate the effectiveness of the proposed HEG approach, we design a nine-layer fully convolutional network suitable for pixel-wise classification. Experiments show that using generated hard examples from the proposed HEG approach improves the pixel-wise classification model’s accuracy on red tide detection and hyperspectral image classification tasks.

/pdf/generating-hard-examples-for-pixel-wise-classification-4r8ixz8cja.pdf

Generating Hard Examples for Pixel-Wise Classification

In this paper, a Generalized Kernel-based Ensemble Learning (GKEL) algorithm for hyperspectral classification problems is presented. The proposed algorithm generalizes the Sparse Kernel-based Ensemble Learning (SKEL) technique, developed previously by the authors. SKEL optimally and sparsely weights and aggregates an ensemble of individual SVM classifiers which independently conduct learning within their corresponding randomly selected spectral feature sub-space using a Gaussian kernel. This ensemble decision is fully optimal, if the dimensionality of the randomly selected feature subspaces and the initial number of the sub-classifiers are determined optimally and is sub-optimal, otherwise. This suboptimality issue is addressed by taking a bottom-up approach. Individual sub-classifiers are added one-by-one optimally to the ensemble until the ensemble converges. The feature subspace of each individual classifier is optimally selected. The ensemble is modeled as a Quadratically-Constrained Linear Programming (QCLP) problem and optimized by combining Multiple Kernel Learning (MKL) with a greedy, non-linear integer programming method for non-monotonic sparse feature sub-space selection. Hyperspectral image data as well as multivariate data are used to verify the performance improvement of the proposed GKEL algorithm over SKEL in detecting difficult targets.

Heesung Kwon

Papers

Coalition Game Theory-Based Feature Subspace Selection for Hyperspectral Classification

Kernel adaptive subspace detector for hyperspectral target detection

Kernel canonical correlation analysis for hyperspectral anomaly detection

Generating Hard Examples for Pixel-Wise Classification

Generalized optimal Kernel-Based Ensemble Learning for hyperspectral classification problems