Central to the looming paradigm shift toward data-intensive science, machine-learning techniques are becoming increasingly important. In particular, deep learning has proven to be both a major breakthrough and an extremely powerful tool in many fields. Shall we embrace deep learning as the key to everything? Or should we resist a black-box solution? These are controversial issues within the remote-sensing community. In this article, we analyze the challenges of using deep learning for remote-sensing data analysis, review recent advances, and provide resources we hope will make deep learning in remote sensing seem ridiculously simple. More importantly, we encourage remote-sensing scientists to bring their expertise into deep learning and use it as an implicit general model to tackle unprecedented, large-scale, influential challenges, such as climate change and urbanization.

/pdf/deep-learning-in-remote-sensing-a-comprehensive-review-and-3o9ov8r19z.pdf

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Deep-learning (DL) algorithms, which learn the representative and discriminative features in a hierarchical manner from the data, have recently become a hotspot in the machine-learning area and have been introduced into the geoscience and remote sensing (RS) community for RS big data analysis. Considering the low-level features (e.g., spectral and texture) as the bottom level, the output feature representation from the top level of the network can be directly fed into a subsequent classifier for pixel-based classification. As a matter of fact, by carefully addressing the practical demands in RS applications and designing the input?output levels of the whole network, we have found that DL is actually everywhere in RS data analysis: from the traditional topics of image preprocessing, pixel-based classification, and target recognition, to the recent challenging tasks of high-level semantic feature extraction and RS scene understanding.

Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Its development in the past two decades can be regarded as an epitome of computer vision history. If we think of today's object detection as a technical aesthetics under the power of deep learning, then turning back the clock 20 years we would witness the wisdom of cold weapon era. This paper extensively reviews 400+ papers of object detection in the light of its technical evolution, spanning over a quarter-century's time (from the 1990s to 2019). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed up techniques, and the recent state of the art detection methods. This paper also reviews some important detection applications, such as pedestrian detection, face detection, text detection, etc, and makes an in-deep analysis of their challenges as well as technical improvements in recent years.

Object Detection in 20 Years: A Survey

Substantial efforts have been devoted more recently to presenting various methods for object detection in optical remote sensing images. However, the current survey of datasets and deep learning based methods for object detection in optical remote sensing images is not adequate. Moreover, most of the existing datasets have some shortcomings, for example, the numbers of images and object categories are small scale, and the image diversity and variations are insufficient. These limitations greatly affect the development of deep learning based object detection methods. In the paper, we provide a comprehensive review of the recent deep learning based object detection progress in both the computer vision and earth observation communities. Then, we propose a large-scale, publicly available benchmark for object DetectIon in Optical Remote sensing images, which we name as DIOR. The dataset contains 23,463 images and 192,472 instances, covering 20 object classes. The proposed DIOR dataset (1) is large-scale on the object categories, on the object instance number, and on the total image number; (2) has a large range of object size variations, not only in terms of spatial resolutions, but also in the aspect of inter- and intra-class size variability across objects; (3) holds big variations as the images are obtained with different imaging conditions, weathers, seasons, and image quality; and (4) has high inter-class similarity and intra-class diversity. The proposed benchmark can help the researchers to develop and validate their data-driven methods. Finally, we evaluate several state-of-the-art approaches on our DIOR dataset to establish a baseline for future research.

Object detection in optical remote sensing images: A survey and a new benchmark

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.

/pdf/deep-learning-in-remote-sensing-a-review-3epn5qlsfr.pdf

Deep learning in remote sensing: a review

Automatic oil tank detection plays a very important role for remote sensing image processing. To accomplish the task, a hierarchical oil tank detector with deep surrounding features is proposed in this paper. The surrounding features extracted by the deep learning model aim at making the oil tanks more easily to recognize, since the appearance of oil tanks is a circle and this information is not enough to separate targets from the complex background. The proposed method is divided into three modules: 1) candidate selection; 2) feature extraction; and 3) classification. First, a modified ellipse and line segment detector (ELSD) based on gradient orientation is used to select candidates in the image. Afterward, the feature combing local and surrounding information together is extracted to represent the target. Histogram of oriented gradients (HOG) which can reliably capture the shape information is extracted to characterize the local patch. For the surrounding area, the convolutional neural network (CNN) trained in ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) contest is applied as a blackbox feature extractor to extract rich surrounding feature. Then, the linear support vector machine (SVM) is utilized as the classifier to give the final output. Experimental results indicate that the proposed method is robust under different complex backgrounds and has high detection rate with low false alarm.

A Hierarchical Oil Tank Detector With Deep Surrounding Features for High-Resolution Optical Satellite Imagery

In this paper, we propose a simple but effective algorithm, Quadratic Constrained Energy Minimization (QCEM) detector for hyperspectral image target detection. QCEM is a nonlinear version of classical Constrained Energy Minimization (CEM) detector, and it exploits the nonlinear characteristics of data by adding quadratic term on CEM model. Experimental results on one real hyperspectral images and one synthetic image suggest our method significantly improves the performance of the original CEM detection algorithm.

Quadratic Constrained Energy Minimization for hyperspectral target detection

The invention discloses a visible remote-sensing image sea-land segmentation method based on image segmentation and supervised learning, and the method comprises five steps: 1, reading data through a computer; 2, carrying out the segmentation of an image; 3, extracting statistical characteristics of a segmented image region; 4, training a linear SVM classifier; 5, employing a trained linear SVM for sea-land judgment, and obtaining a final sea-land segmentation result graph The method irons out the defects in the prior art, well solves a problem of sea-land segmentation in a visible remote-sensing image, and obtains a better segmentation result Therefore, the method can be used in the sea-land segmentation of a visible remote-sensing image ship detection flow, is good in application prospect and is high in application value

Visible remote-sensing image sea-land segmentation method based on image segmentation and supervised learning

This paper presents a novel technique for 3D browsing of wide‐angle fisheye images using view‐dependent perspective correction (VDPC). First, the fisheye imaging model with interior orientation parameters (IOPs) is established. Thereafter, a VDPC model for wide‐angle fisheye images is proposed that adaptively selects correction planes for different areas of the image format. Finally, the wide‐angle fisheye image is re‐projected to obtain the visual effect of browsing in hemispherical space, using the VDPC model and IOPs of the fisheye camera calibrated using the ideal projection ellipse constraint. The proposed technique is tested on several downloaded internet images with unknown IOPs. Results show that the proposed VDPC model achieves a more uniform perspective correction of fisheye images in different areas, and preserves the detailed information with greater flexibility compared with the traditional perspective projection conversion (PPC) technique. The proposed algorithm generates a corrected image of 512 × 512 pixels resolution at a speed of 58 fps when run on a pure central processing unit (CPU) processor. With an ordinary graphics processing unit (GPU) processor, a corrected image of 1024 × 1024 pixels resolution can be generated at 60 fps. Therefore, smooth 3D visualisation of a fisheye image can be realised on a computer using the proposed algorithm, which may benefit applications such as panorama surveillance, robot navigation, etc.

Jun Wu

Papers

A Hierarchical Oil Tank Detector With Deep Surrounding Features for High-Resolution Optical Satellite Imagery

Quadratic Constrained Energy Minimization for hyperspectral target detection

Visible remote-sensing image sea-land segmentation method based on image segmentation and supervised learning

3D Browsing of Wide‐Angle Fisheye Images Under View‐Dependent Perspective Correction

High-precision calibration of wide-angle fisheye lens with radial distortion projection ellipse constraint (RDPEC)