Integrating Convolutional Neural Network and Multiresolution Segmentation for Land Cover and Land Use Mapping Using Satellite Imagery
15 Jun 2021-Applied Sciences (Multidisciplinary Digital Publishing Institute)-Vol. 11, Iss: 12, pp 5551
TL;DR: The accuracies revealed the efficiency of the CNN–MRS model for land cover map production in large areas and was compared quantitatively with state-of-the-art CNN model results and related works.
Abstract: Depletion of natural resources, population growth, urban migration, and expanding drought conditions are some of the reasons why environmental monitoring programs are required and regularly produced and updated. Additionally, the usage of artificial intelligence in the geospatial field of Earth observation (EO) and regional land monitoring missions is a challenging issue. In this study, land cover and land use mapping was performed using the proposed CNN–MRS model. The CNN–MRS model consisted of two main steps: CNN-based land cover classification and enhancing the classification with spatial filter and multiresolution segmentation (MRS). Different band numbers of Sentinel-2A imagery and multiple patch sizes (32 × 32, 64 × 64, and 128 × 128 pixels) were used in the first experiment. The algorithms were evaluated in terms of overall accuracy, precision, recall, F1-score, and kappa coefficient. The highest overall accuracy was obtained with the proposed approach as 97.31% in Istanbul test site area and 98.44% in Kocaeli test site area. The accuracies revealed the efficiency of the CNN–MRS model for land cover map production in large areas. The McNemar test measured the significance of the models used. In the second experiment, with the Zurich Summer dataset, the overall accuracy of the proposed approach was obtained as 92.03%. The results are compared quantitatively with state-of-the-art CNN model results and related works.
Citations
More filters
TL;DR: A novel Superpixel-based Attention Graph Neural Network (SAGNN) for semantic segmentation of high spatial resolution aerial images and the accuracy of the model on the Potsdam and Vaihingen public datasets exceeds all benchmark approaches.
Abstract: Semantic segmentation is one of the significant tasks in understanding aerial images with high spatial resolution. Recently, Graph Neural Network (GNN) and attention mechanism have achieved excellent performance in semantic segmentation tasks in general images and been applied to aerial images. In this paper, we propose a novel Superpixel-based Attention Graph Neural Network (SAGNN) for semantic segmentation of high spatial resolution aerial images. A K-Nearest Neighbor (KNN) graph is constructed from our network for each image, where each node corresponds to a superpixel in the image and is associated with a hidden representation vector. On this basis, the initialization of the hidden representation vector is the appearance feature extracted by a unary Convolutional Neural Network (CNN) from the image. Moreover, relying on the attention mechanism and recursive functions, each node can update its hidden representation according to the current state and the incoming information from its neighbors. The final representation of each node is used to predict the semantic class of each superpixel. The attention mechanism enables graph nodes to differentially aggregate neighbor information, which can extract higher-quality features. Furthermore, the superpixels not only save computational resources, but also maintain object boundary to achieve more accurate predictions. The accuracy of our model on the Potsdam and Vaihingen public datasets exceeds all benchmark approaches, reaching 90.23% and 89.32%, respectively.
8 citations
24 Sep 2021
TL;DR: In this study, the classification of point clouds obtained by aerial photogrammetry and Light Detection and Ranging (LiDAR) technology belonging to the same region is performed by using machine learning.
Abstract: With the development of photogrammetry technologies, point clouds have found a wide range of use in academic and commercial areas. This situation has made it essential to extract information from point clouds. In particular, artificial intelligence applications have been used to extract information from point clouds to complex structures. Point cloud classification is also one of the leading areas where these applications are used. In this study, the classification of point clouds obtained by aerial photogrammetry and Light Detection and Ranging (LiDAR) technology belonging to the same region is performed by using machine learning. For this purpose, nine popular machine learning methods have been used. Geometric features obtained from point clouds were used for the feature spaces created for classification. Color information is also added to these in the photogrammetric point cloud. According to the LiDAR point cloud results, the highest overall accuracies were obtained as 0.96 with the Multilayer Perceptron (MLP) method. The lowest overall accuracies were obtained as 0.50 with the AdaBoost method. The method with the highest overall accuracy was achieved with the MLP (0.90) method. The lowest overall accuracy method is the GNB method with 0.25 overall accuracy.
5 citations
TL;DR: In this paper , object detection using deep learning from aerial or terrestrial images has become a popular research area, and many different disciplines use deep learning algorithms for various purposes, including object detection from aerial images.
Abstract: Many different disciplines use deep Learning algorithms for various purposes. In recent years, object detection by deep learning from aerial or terrestrial images has become a popular research area. In this study, object detection application was performed by training the YOLOv2 and YOLOv3 algorithms in the Google Colaboratory cloud service with the help of Python software language with the DOTA dataset consisting of aerial photographs. 43 aerial photographs containing 9 class objects were used for evaluation. Accuracy analyzes of these two algorithms were made according to Recall, Precision and F-score for 9 classes, and the results were compared accordingly. YOLOv2 gave better results in 5 out of 9 classes, while YOLOv3 gave better results in recognizing small objects. While YOLOv2 can detect objects in an average photograph in 43 seconds, YOLOv3 has achieved superior performance in terms of time by detecting objects in an average of 2.5 seconds.
3 citations
TL;DR: The SegUNet3D as discussed by the authors is an ensemble approach based on the combination of U-Net and SegNet algorithms for semantic segmentation of 3D mobile point clouds with spherical projection, which is able to improve the mIoU metric by 15.9% in SemanticPOSS dataset and up to 5.4% in RELLIS-3D dataset.
Abstract: Mobile light detection and ranging (LiDAR) sensor point clouds are used in many fields such as road network management, architecture and urban planning, and 3D High Definition (HD) city maps for autonomous vehicles. Semantic segmentation of mobile point clouds is critical for these tasks. In this study, we present a robust and effective deep learning-based point cloud semantic segmentation method. Semantic segmentation is applied to range images produced from point cloud with spherical projection. Irregular 3D mobile point clouds are transformed into regular form by projecting the clouds onto the plane to generate 2D representation of the point cloud. This representation is fed to the proposed network that produces semantic segmentation. The local geometric feature vector is calculated for each point. Optimum parameter experiments were also performed to obtain the best results for semantic segmentation. The proposed technique, called SegUNet3D, is an ensemble approach based on the combination of U-Net and SegNet algorithms. SegUNet3D algorithm has been compared with five different segmentation algorithms on two challenging datasets. SemanticPOSS dataset includes the urban area, whereas RELLIS-3D includes the off-road environment. As a result of the study, it was demonstrated that the proposed approach is superior to other methods in terms of mean Intersection over Union (mIoU) in both datasets. The proposed method was able to improve the mIoU metric by up to 15.9% in the SemanticPOSS dataset and up to 5.4% in the RELLIS-3D dataset.
3 citations
TL;DR: In this article , the authors performed an in-depth analysis of convolutional neural networks (CNN) in combination with geographic object-based image analysis (GEOBIA) for mapping volcanic and glacial landforms.
Abstract: Abstract Rapid detection and mapping of landforms are crucially important to improve our understanding of past and presently active processes across the earth, especially, in complex and dynamic volcanoes. Traditional landform modeling approaches are labor-intensive and time-consuming. In recent years, landform mapping has increasingly been digitized. This study conducted an in-depth analysis of convolutional neural networks (CNN) in combination with geographic object-based image analysis (GEOBIA), for mapping volcanic and glacial landforms. Sentinel-2 image, as well as predisposing variables (DEM and its derivatives, e.g., slope, aspect, curvature and flow accumulation), were segmented using a multi-resolution segmentation algorithm, and relevant features were selected to define segmentation scales for each landform category. A set of object-based features was developed based on spectral (e.g., brightness), geometrical (e.g., shape index), and textural (grey level co-occurrence matrix) information. The landform modelling networks were then trained and tested based on labelled objects generated using GEOBIA and ground control points. Our results show that an integrated approach of GEOBIA and CNN achieved an ACC of 0.9685, 0.9780, 0.9614, 0.9767, 0.9675, 0.9718, 0.9600, and 0.9778 for dacite lava, caldera, andesite lava, volcanic cone, volcanic tuff, glacial circus, glacial valley, and suspended valley, respectively. The quantitative evaluation shows the highest performance (Accuracy > 0.9600 and cross-validation accuracy > 0.9400) for volcanic and glacial landforms and; therefore, is recommended for regional and large-scale landform mapping. Our results and the provided automatic workflow emphasize the potential of integrated GEOBIA and CNN for fast and efficient landform mapping as a first step in the earth’s surface management.
3 citations
References
More filters
Proceedings Article•
01 Jan 2015TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
49,914 citations
TL;DR: In this article, the authors present a procedure for having two or more judges independently categorize a sample of units and determine the degree, significance, and significance of the units. But they do not discuss the extent to which these judgments are reproducible, i.e., reliable.
Abstract: CONSIDER Table 1. It represents in its formal characteristics a situation which arises in the clinical-social-personality areas of psychology, where it frequently occurs that the only useful level of measurement obtainable is nominal scaling (Stevens, 1951, pp. 2526), i.e. placement in a set of k unordered categories. Because the categorizing of the units is a consequence of some complex judgment process performed by a &dquo;two-legged meter&dquo; (Stevens, 1958), it becomes important to determine the extent to which these judgments are reproducible, i.e., reliable. The procedure which suggests itself is that of having two (or more) judges independently categorize a sample of units and determine the degree, significance, and
34,965 citations
TL;DR: Quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures, including FCN and DeconvNet.
Abstract: We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/ .
13,468 citations
TL;DR: In this article, the relationship between various linear combinations of red and photographic infrared radiances and vegetation parameters is investigated, showing that red-IR combinations to be more significant than green-red combinations.
8,537 citations
TL;DR: This paper reviews the necessary considerations and available techniques for assessing the accuracy of remotely sensed data including the classification system, the sampling scheme, the sample size, spatial autocorrelation, and the assessment techniques.
6,747 citations