Object detection in optical remote sensing images, being a fundamental but challenging problem in the field of aerial and satellite image analysis, plays an important role for a wide range of applications and is receiving significant attention in recent years. While enormous methods exist, a deep review of the literature concerning generic object detection is still lacking. This paper aims to provide a review of the recent progress in this field. Different from several previously published surveys that focus on a specific object class such as building and road, we concentrate on more generic object categories including, but are not limited to, road, building, tree, vehicle, ship, airport, urban-area. Covering about 270 publications we survey (1) template matching-based object detection methods, (2) knowledge-based object detection methods, (3) object-based image analysis (OBIA)-based object detection methods, (4) machine learning-based object detection methods, and (5) five publicly available datasets and three standard evaluation metrics. We also discuss the challenges of current studies and propose two promising research directions, namely deep learning-based feature representation and weakly supervised learning-based geospatial object detection. It is our hope that this survey will be beneficial for the researchers to have better understanding of this research field.

A survey on object detection in optical remote sensing images

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Its development in the past two decades can be regarded as an epitome of computer vision history. If we think of today's object detection as a technical aesthetics under the power of deep learning, then turning back the clock 20 years we would witness the wisdom of cold weapon era. This paper extensively reviews 400+ papers of object detection in the light of its technical evolution, spanning over a quarter-century's time (from the 1990s to 2019). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed up techniques, and the recent state of the art detection methods. This paper also reviews some important detection applications, such as pedestrian detection, face detection, text detection, etc, and makes an in-deep analysis of their challenges as well as technical improvements in recent years.

Object Detection in 20 Years: A Survey

Substantial efforts have been devoted more recently to presenting various methods for object detection in optical remote sensing images. However, the current survey of datasets and deep learning based methods for object detection in optical remote sensing images is not adequate. Moreover, most of the existing datasets have some shortcomings, for example, the numbers of images and object categories are small scale, and the image diversity and variations are insufficient. These limitations greatly affect the development of deep learning based object detection methods. In the paper, we provide a comprehensive review of the recent deep learning based object detection progress in both the computer vision and earth observation communities. Then, we propose a large-scale, publicly available benchmark for object DetectIon in Optical Remote sensing images, which we name as DIOR. The dataset contains 23,463 images and 192,472 instances, covering 20 object classes. The proposed DIOR dataset (1) is large-scale on the object categories, on the object instance number, and on the total image number; (2) has a large range of object size variations, not only in terms of spatial resolutions, but also in the aspect of inter- and intra-class size variability across objects; (3) holds big variations as the images are obtained with different imaging conditions, weathers, seasons, and image quality; and (4) has high inter-class similarity and intra-class diversity. The proposed benchmark can help the researchers to develop and validate their data-driven methods. Finally, we evaluate several state-of-the-art approaches on our DIOR dataset to establish a baseline for future research.

Object detection in optical remote sensing images: A survey and a new benchmark

Earth-observing remote sensing data, including aerial photography and satellite imagery, offer a snapshot of the world from which we can learn about the state of natural resources and the built environment. The components of energy systems that are visible from above can be automatically assessed with these remote sensing data when processed with machine learning methods. Here, we focus on the information gap in distributed solar photovoltaic (PV) arrays, of which there is limited public data on solar PV deployments at small geographic scales. We created a dataset of solar PV arrays to initiate and develop the process of automatically identifying solar PV locations using remote sensing imagery. This dataset contains the geospatial coordinates and border vertices for over 19,000 solar panels across 601 high-resolution images from four cities in California. Dataset applications include training object detection and other machine learning algorithms that use remote sensing imagery, developing specific algorithms for predictive detection of distributed PV systems, estimating installed PV capacity, and analysis of the socioeconomic correlates of PV deployment. Machine-accessible metadata file describing the reported data (ISA-Tab format)

/pdf/distributed-solar-photovoltaic-array-location-and-extent-46upvu2u5n.pdf

Distributed solar photovoltaic array location and extent dataset for remote sensing object identification.

https://hal.archives-ouvertes.fr/hal-01122605/document

Vehicle detection in aerial imagery

When failures occur in machine object recognition algorithms, researchers may have limited information on the root causes of the failure. For example, did the algorithm fail to detect a target due to occlusion, shadow, contrast, or some other known computer vision shortcoming? The Overhead Imagery Research Data Set (OIRDS) project will help advance the state of the art in image processing and computer vision by providing an open-access, annotated overhead imagery library that will allow researchers to break down algorithm performance by image and target attributes. The OIRDS project has produced a data set with almost 1,000 labeled images suitable for developing automated vehicle detection algorithms. These images contain approximately 1,800 labeled targets. For each target, the OIRDS provides over 30 annotations and over 60 statistics that describe the target within the context of the image.

Overhead imagery research data set — an annotated data library & tools to aid in the development of computer vision algorithms

Current machine learning algorithms identify statistical regularities in complex data sets and are regularly used across a range of application domains, but they lack the robustness and generalizability associated with human learning. If machine learning techniques could enable computers to learn from fewer examples, transfer knowledge between tasks, and adapt to changing contexts and environments, the results would have very broad scientific and societal impacts. Increased processing and memory resources have enabled larger, more capable learning models, but there is growing recognition that even greater computing resources would not be sufficient to yield algorithms capable of learning from a few examples and generalizing beyond initial training sets. This paper presents perspectives on feature selection, representation schemes and interpretability, transfer learning, continuous learning, and learning and adaptation in time-varying contexts and environments, five key areas that are essential for advancing machine learning capabilities. Appropriate learning tasks that require these capabilities can demonstrate the strengths of novel machine learning approaches that could address these challenges.

/pdf/future-directions-in-machine-learning-aduo7vw8ed.pdf

Future Directions in Machine Learning

After nearly half a century of computer vision research, application-specific systems are common but the goal of developing a robust, general-purpose computer vision system remains out of reach. Rather than focus on the strengths and weaknesses of current computer vision approaches, this paper will enumerate and investigate the challenges that must be overcome before this goal can be achieved. Key challenges include handling variations in environment or acquisition parameters such as lighting, view angle, distance, and image quality; recognizing naturally occurring as well as intentionally deceptive variations in object appearance; providing robust general-purpose image segmentation and co-registration; generating 3D representations from 2D images; developing useful object representations; providing required knowledge that is not represented in the image itself; and managing computational complexity. Each of these challenges, along with their relevance to solving the vision problem, will be discussed. Understanding these challenges as a whole may provide insight into underlying mechanisms that will provide the backbone of a robust general-purpose computer vision system.

Current challenges in automating visual perception

The National Geospatial-Intelligence Agency (NGA) is faced with the difficult task of extracting geospatial intelligence information on complex, time-sensitive targets from a growing volume of images. Neuroscience promises to provide basic research findings that could translate into tools, training, and procedures capable of enhancing the current analysts' performance and productivity, as well as leading toward tools for automated image analysis. The Neuroscience-Enabled Geospatial Intelligence (NEGI) advanced research program has been developed to identify useful neuroscience research results and guide translational neuroscience research to provide a new generation of geospatial analysis tools. In addition, the capability of current models of the human vision system to perform basic geospatial analysis tasks has been assessed, with encouraging results.

Carsten Oertel

Papers

Overhead imagery research data set — an annotated data library & tools to aid in the development of computer vision algorithms

Future Directions in Machine Learning

Current challenges in automating visual perception

Geospatial intelligence and the neuroscience of human vision