The PASCAL Visual Object Classes Challenge

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.

Deep learning with convolutional neural networks for EEG decoding and visualization.

Hyperspectral remote sensing technology has advanced significantly in the past two decades. Current sensors onboard airborne and spaceborne platforms cover large areas of the Earth surface with unprecedented spectral, spatial, and temporal resolutions. These characteristics enable a myriad of applications requiring fine identification of materials or estimation of physical parameters. Very often, these applications rely on sophisticated and complex data analysis methods. The sources of difficulties are, namely, the high dimensionality and size of the hyperspectral data, the spectral mixing (linear and nonlinear), and the degradation mechanisms associated to the measurement process such as noise and atmospheric effects. This paper presents a tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing. In all topics, we describe the state-of-the-art, provide illustrative examples, and point to future challenges and research directions.

/pdf/hyperspectral-remote-sensing-data-analysis-and-future-1cxk3zgg98.pdf

Hyperspectral Remote Sensing Data Analysis and Future Challenges

For object detection, the two-stage approach (e.g., Faster R-CNN) has been achieving the highest accuracy, whereas the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, in this paper, we propose a novel single-shot based detector, called RefineDet, that achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. RefineDet consists of two inter-connected modules, namely, the anchor refinement module and the object detection module. Specifically, the former aims to (1) filter out negative anchors to reduce search space for the classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes the refined anchors as the input from the former to further improve the regression accuracy and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multitask loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO demonstrate that RefineDet achieves state-of-the-art detection accuracy with high efficiency. Code is available at https://github.com/sfzhang15/RefineDet.

/pdf/single-shot-refinement-neural-network-for-object-detection-j6fdptx7kt.pdf

Single-Shot Refinement Neural Network for Object Detection

Objective Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. Approach In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). Main results We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Significance Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

https://iopscience.iop.org/article/10.1088/1741-2552/aace8c/pdf

EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces

Object detection is increasingly used onboard Unmanned Aerial Vehicles (UAV) for various applications; however, the machine learning (ML) models for UAV-based detection are often validated using data curated for tasks unrelated to the UAV application. This is a concern because training neural networks on large-scale benchmarks have shown excellent capability in generic object detection tasks, yet conventional training approaches can lead to large inference errors for UAV-based images. Such errors arise due to differences in imaging conditions between images from UAVs and images in training. To overcome this problem, we characterize boundary conditions of ML models, beyond which the models exhibit rapid degradation in detection accuracy. Our work is focused on understanding the impact of different UAV-based imaging conditions on detection performance by using synthetic data generated using a game engine. Properties of the game engine are exploited to populate the synthetic datasets with realistic and annotated images. Specifically, it enables the fine control of various parameters, such as camera position, view angle, illumination conditions, and object pose. Using the synthetic datasets, we analyze detection accuracy in different imaging conditions as a function of the above parameters. We use three well-known neural network models with different model complexity in our work. In our experiment, we observe and quantify the following: 1) how detection accuracy drops as the camera moves toward the nadir-view region; 2) how detection accuracy varies depending on different object poses, and 3) the degree to which the robustness of the models changes as illumination conditions vary.

Validation of object detection in UAV-based images using synthetic data

Target classification is a crucial component in automatic target recognition systems, yet one of the most difficult to develop due to the high level of variability in target signatures. Classification in low-dimensional manifold space is a promising approach since the manifold learning algorithm embeds the target chips into a low-dimensional space using key class features, and therefore is effective in the presence of noise and when the training and testing data exhibit variations due to differences in target range, aspect angles or other factors. This work develops an approach using support vector machine (SVM) classification in a nonlinear manifold space learned from real target imagery, outperforming classification in the image space. The proposed approach is very robust with respect to the dimensionality of the embedding as well as to the parameter settings, demonstrating the practicality of this approach for automatic target recognition applications.

Robust classification using support vector machine in low-dimensional manifold space for automatic target recognition

In this paper, we present a segmentation based wavelet coding scheme, in which an image is segmented into two regions: stationary areas (background) and the areas containing edge information (foreground). These regions are then encoded independently using two dedicated encoders that are optimized for each region. A 2-D edge operator is used for segmenting the image. We use the embedded zerotree wavelet (EZW) algorithm for encoding the background due to its good performance on stationary areas. The foreground area is, however, encoded using a predictive residual vector quantizer (PRVQ). Experimental results show that the proposed technique improves the quality of the reconstructed images, both numerically (in terms of mean square error) and perceptually when compared to EZW at the same bit rate.

Segmentation based wavelet coding of digital images

Target detection techniques play an important role in automatic target recognition (ATR) systems because overall ATR performance depends closely on detection results. A number of detection techniques based on infrared (IR) images have been developed using a variety of pattern recognition approaches. However, target detection based on a single IR sensor is often hampered by adverse weather conditions or countermeasures, resulting in unacceptably high false alarm rates. Multiple imaging sensors in different spectral ranges, such as visible and infrared bands, are used here to reduce such adverse effects. The imaging data from the different sensors are jointly processed to exploit the spatial characteristics of the objects. Four local features are used to exploit the local characteristics of the images generated from each sensor. A confidence image is created via feature-based fusion that combines the features to obtain potential target locations. Experimental results using two test sequences are provided to demonstrate the viability of the proposed technique.

Multisensor target detection using adaptive feature-based fusion

Compression of SAR imagery for battlefield digitization is discussed in this paper THe images are first processed to separate out possible target areas These target areas are compressed losslessly to avoid any degradation of the images The background information which is usually necessary to establish context, is compressed using a hybrid vector quantization algorithm An adaptive variable rate residual vector quantizer is use to compress the residual signal generated by a neural network predictor The vector quantizer codebooks are optimized for entropy coding using an entropy-constrained algorithm to further improve the coding performance This constrained vector-quantizer combination performs extremely well as suggested by the experimental results© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering Downloading of the abstract is permitted for personal use only

Heesung Kwon

Papers

Validation of object detection in UAV-based images using synthetic data

Robust classification using support vector machine in low-dimensional manifold space for automatic target recognition

Segmentation based wavelet coding of digital images

Multisensor target detection using adaptive feature-based fusion

Compression of SAR imagery using adaptive residual vector quantization