Hyperspectral image classification using CNN: Application to industrial food packaging

Road detection technology plays an essential role in a variety of applications, such as urban planning, map updating, traffic monitoring and automatic vehicle navigation. Recently, there has been much development in detecting roads in high-resolution (HR) satellite images based on semantic segmentation. However, the objects being segmented in such images are of small size, and not all the information in the images is equally important when making a decision. This paper proposes a novel approach to road detection based on semantic segmentation and edge detection. Our approach aims to combine these two techniques to improve road detection, and it produces sharp-pixel segmentation maps, using the segmented masks to generate road edges. In addition, some well-known architectures, such as SegNet, used multi-scale features without refinement; thus, using attention blocks in the encoder to predict fine segmentation masks resulted in finer edges. A combination of weighted cross-entropy loss and the focal Tversky loss as the loss function is also used to deal with the highly imbalanced dataset. We conducted various experiments on two datasets describing real-world datasets covering the three largest regions in Saudi Arabia and Massachusetts. The results demonstrated that the proposed method of encoding HR feature maps effectively predicts sharp segmentation masks to facilitate accurate edge detection, even against a harsh and complicated background.

https://www.mdpi.com/2072-4292/14/3/613/pdf?version=1643524435

Semantic Segmentation and Edge Detection - Approach to Road Detection in Very High Resolution Satellite Images

Convolutional neural network (CNN) has exhibited enormous potentials in hyperspectral image (HSI) classification owing to excellent locally modeling ability. Although excellent performance of CNN-based methods has been witnessed, there still have some limitations of their internal network backbone. On the one hand, modeling long-distance context dependencies is an inborn defect, which leads to receptive field limitation and insufficient feature capture in HSI. On the other hand, CNN-based methods usually need various sample distribution to train and cannot infer dynamically, which may not capture the inherent changes of HSI data well. To overcome the abovementioned issues, we propose a novel local transformer with spatial partition restore network (SPRLT-Net) for HSI classification. First, local transformer is introduced to obtain the spatial attention weights dynamically by measuring the similarity between related pixel pairs. Second, a spatial partition restore (SPR) module is designed to split the input patch into several overlapping continuous subpatches as sequential. With the obtained attention weights at hand, the SPR module restores the sequential to the original patch. Finally, a fully connected layer is used for classification. SPRLT-Net can capture global context dependencies, and the dynamical attention weights can adapt the inherent changes of HSI spatial pixels. Experimental results based on spatially disjoint samples and randomly selected samples of five benchmark datasets demonstrate that SPRLT-Net outperforms the other state-of-the-art methods in terms of classification accuracy, generalization performance, and computational complexity.

https://ieeexplore.ieee.org/ielx7/4609443/4609444/09772356.pdf

Local Transformer With Spatial Partition Restore for Hyperspectral Image Classification

The development of remote sensing images in recent years has made it possible to identify materials in inaccessible environments and study natural materials on a large scale. But hyperspectral images (HSIs) are a rich source of information with their unique features in various applications. However, several problems reduce the accuracy of HSI classification; for example, the extracted features are not effective, noise, the correlation of bands, and most importantly, the limited labeled samples. To improve accuracy in the case of limited training samples, we propose a multiscale dual-branch residual spectral&#x2013;spatial network with attention to the HSI classification model named MDBRSSN in this article. First, due to the correlation and redundancy between HSI bands, a principal component analysis operation is applied to preprocess the raw HSI data. Then, in MDBRSSN, a dual-branch structure is designed to extract the useful spectral&#x2013;spatial features of HSI. The advanced feature, multiscale abstract information extracted by the convolution neural network, is applied to image processing, which can improve complex hyperspectral data classification accuracy. In addition, the attention mechanisms applied separately to each branch enable MDBRSSN to optimize and refine the extracted feature maps. Such an MDBRSSN framework can learn and fuse deeper hierarchical spectral&#x2013;spatial features with fewer training samples. The purpose of designing the MDBRSSN model is to have high classification accuracy compared to state-of-the-art methods when the training samples are limited, which is proved by the results of the experiments in this article on four datasets. In Salinas, Pavia University, Indian Pines, and Houston 2013, the proposed model obtained 99.64&#x0025;, 98.93&#x0025;, 98.17&#x0025;, and 96.57&#x0025; overall accuracy using only 1&#x0025;, 1&#x0025;, 5&#x0025;, and 5&#x0025; of labeled data for training, respectively, which are much better compared to the state-of-the-art methods. 

Multiscale Dual-Branch Residual Spectral–Spatial Network With Attention for Hyperspectral Image Classification

The convolutional neural network (CNN) method has been widely used in the classification of hyperspectral images (HSIs). However, the efficiency and accuracy of the HSI classification are inevitably degraded when small samples are available. This study proposes a multidimensional CNN model named MDAN, which is constructed with an attention mechanism, to achieve an ideal classification performance of CNN within the framework of few-shot learning. In this model, a three-dimensional (3D) convolutional layer is carried out for obtaining spatial–spectral features from the 3D volumetric data of HSI. Subsequently, the two-dimensional (2D) and one-dimensional (1D) convolutional layers further learn spatial and spectral features efficiently at an abstract level. Based on the most widely used convolutional block attention module (CBAM), this study investigates a convolutional block self-attention module (CBSM) to improve accuracy by changing the connection ways of attention blocks. The CBSM model is used with the 2D convolutional layer for better performance of HSI classification purposes. The MDAN model is applied for classification applications using HSI, and its performance is evaluated by comparing the results with the support vector machine (SVM), 2D CNN, 3D CNN, 3D–2D–1D CNN, and CBAM. The findings of this study indicate that classification results from the MADN model show overall classification accuracies of 97.34%, 96.43%, and 92.23% for Salinas, WHU-Hi-HanChuan, and Pavia University datasets, respectively, when only 1% HSI data were used for training. The training and testing times of the MDAN model are close to those of the 3D–2D–1D CNN, which has the highest efficiency among all comparative CNN models. The attention model CBSM is introduced into MDAN, which achieves an overall accuracy of about 1% higher than that of the CBAM model. The performance of the two proposed methods is superior to the other models in terms of both efficiency and accuracy. The results show that the combination of multidimensional CNNs and attention mechanisms has the best ability for small-sample problems in HSI classification.

https://www.mdpi.com/2072-4292/14/3/785/pdf?version=1644456367

An Investigation of a Multidimensional CNN Combined with an Attention Mechanism Model to Resolve Small-Sample Problems in Hyperspectral Image Classification

Convolutional neural networks provide an ideal solution for hyperspectral image (HSI) classification. However, the classification effect is not satisfactory when limited training samples are available. Focused on "small sample" hyperspectral classification, we proposed a novel 3D-2D-convolutional neural network (CNN) model named AD-HybridSN (Attention-Dense-HybridSN). In our proposed model, a dense block was used to reuse shallow features and aimed at better exploiting hierarchical spatial-spectral features. Subsequent depth separable convolutional layers were used to discriminate the spatial information. Further refinement of spatial-spectral features was realized by the channel attention method and spatial attention method, which were performed behind every 3D convolutional layer and every 2D convolutional layer, respectively. Experiment results indicate that our proposed model can learn more discriminative spatial-spectral features using very few training data. In Indian Pines, Salinas and the University of Pavia, AD-HybridSN obtain 97.02%, 99.59% and 98.32% overall accuracy using only 5%, 1% and 1% labeled data for training, respectively, which are far better than all the contrast models.

/pdf/spatial-spectral-feature-refinement-for-hyperspectral-image-23e3s24ccu.pdf

Spatial-Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN.

The efficacy of reactive oxygen species-mediated therapy is generally limited by hypoxia and overexpressed glutathione (GSH) in the tumor microenvironment (TME). To address these issues, herein, a smart Mn3O4/OCN-PpIX@BSA nanoplatform is rationally developed to enhance the combinational therapeutic efficacy of chemodynamic therapy (CDT) and sonodynamic therapy (SDT) through TME modulation. For constructing the catalytic nanoplatform (Mn3O4/OCN-PpIX@BSA), Mn3O4 nanoparticles were grown in situ on oxidized g-C3N4 (OCN) nanosheets, and the as-prepared Mn3O4/OCN nano-hybrids were then successively loaded with protoporphyrin (PpIX) and coated with bovine serum albumin (BSA). The catalase-like Mn3O4 nanoparticles are able to effectively catalyze the overexpressed endogenous H2O2 to produce O2, which could relieve hypoxia and improve the therapeutic effect of combinational CDT/SDT. The decomposition of Mn3O4 by GSH enables the release of Mn2+ ions, which not only facilitates good T1/T2 dual-modal magnetic resonance imaging for tumor localization but also results in the depletion of GSH and the Mn2+-driven Fenton-like reaction, thus further amplifying the oxidative stress and achieving improved therapeutic efficacy. It is worth noting that the Mn3O4/OCN-PpIX@BSA nanocomposites exhibit minimal toxicity to normal tissues at therapeutic doses. These positive findings provide a new strategy for the convenient construction of TME-regulating smart theranostic nanoagents to improve the therapeutic outcomes towards malignant tumors effectively.

Modulation of hypoxia and redox in the solid tumor microenvironment with a catalytic nanoplatform to enhance combinational chemodynamic/sonodynamic therapy.

Fengyuan Wei

Papers

Spatial-Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN.

Modulation of hypoxia and redox in the solid tumor microenvironment with a catalytic nanoplatform to enhance combinational chemodynamic/sonodynamic therapy.