This paper presents a database containing 'ground truth' segmentations produced by humans for images of a wide variety of natural scenes. We define an error measure which quantifies the consistency between segmentations of differing granularities and find that different human segmentations of the same image are highly consistent. Use of this dataset is demonstrated in two applications: (1) evaluating the performance of segmentation algorithms and (2) measuring probability distributions associated with Gestalt grouping factors as well as statistics of image region properties.

/pdf/a-database-of-human-segmented-natural-images-and-its-2ve1vk356s.pdf

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Central to the looming paradigm shift toward data-intensive science, machine-learning techniques are becoming increasingly important. In particular, deep learning has proven to be both a major breakthrough and an extremely powerful tool in many fields. Shall we embrace deep learning as the key to everything? Or should we resist a black-box solution? These are controversial issues within the remote-sensing community. In this article, we analyze the challenges of using deep learning for remote-sensing data analysis, review recent advances, and provide resources we hope will make deep learning in remote sensing seem ridiculously simple. More importantly, we encourage remote-sensing scientists to bring their expertise into deep learning and use it as an implicit general model to tackle unprecedented, large-scale, influential challenges, such as climate change and urbanization.

/pdf/deep-learning-in-remote-sensing-a-comprehensive-review-and-3o9ov8r19z.pdf

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

If a physical object has a smooth or piecewise smooth boundary, its images obtained by cameras in varying positions undergo smooth apparent deformations. These deformations are locally well approximated by affine transforms of the image plane. In consequence the solid object recognition problem has often been led back to the computation of affine invariant image local features. Such invariant features could be obtained by normalization methods, but no fully affine normalization method exists for the time being. Even scale invariance is dealt with rigorously only by the scale-invariant feature transform (SIFT) method. By simulating zooms out and normalizing translation and rotation, SIFT is invariant to four out of the six parameters of an affine transform. The method proposed in this paper, affine-SIFT (ASIFT), simulates all image views obtainable by varying the two camera axis orientation parameters, namely, the latitude and the longitude angles, left over by the SIFT method. Then it covers the other four parameters by using the SIFT method itself. The resulting method will be mathematically proved to be fully affine invariant. Against any prognosis, simulating all views depending on the two camera orientation parameters is feasible with no dramatic computational load. A two-resolution scheme further reduces the ASIFT complexity to about twice that of SIFT. A new notion, the transition tilt, measuring the amount of distortion from one view to another, is introduced. While an absolute tilt from a frontal to a slanted view exceeding 6 is rare, much higher transition tilts are common when two slanted views of an object are compared (see Figure hightransitiontiltsillustration). The attainable transition tilt is measured for each affine image comparison method. The new method permits one to reliably identify features that have undergone transition tilts of large magnitude, up to 36 and higher. This fact is substantiated by many experiments which show that ASIFT significantly outperforms the state-of-the-art methods SIFT, maximally stable extremal region (MSER), Harris-affine, and Hessian-affine.

ASIFT: A New Framework for Fully Affine Invariant Image Comparison

The scale-invariant feature transform (SIFT) algorithm and its many variants are widely used in computer vision and in remote sensing to match features between images or to localize and recognize objects. However, mostly because of speckle noise, it does not perform well on synthetic aperture radar (SAR) images. In this paper, we introduce a SIFT-like algorithm specifically dedicated to SAR imaging, which is named SAR-SIFT. The algorithm includes both the detection of keypoints and the computation of local descriptors. A new gradient definition, yielding an orientation and a magnitude that are robust to speckle noise, is first introduced. It is then used to adapt several steps of the SIFT algorithm to SAR images. We study the improvement brought by this new algorithm, as compared with existing approaches. We present an application of SAR-SIFT to the registration of SAR images in different configurations, particularly with different incidence angles.

/pdf/sar-sift-a-sift-like-algorithm-for-sar-images-4qtsvlpaq7.pdf

Sar-sift: a sift-like algorithm for sar images

The Copernicus Sentinel-2 program now provides multispectral images at a global scale with a high revisit rate. In this paper we explore the usage of convolutional neural networks for urban change detection using such multispectral images. We first present the new change detection dataset that was used for training the proposed networks, which will be openly available to serve as a benchmark. The Onera Satellite Change Detection (OSCD) dataset is composed of pairs of multispectral aerial images, and the changes were manually annotated at pixel level. We then propose two architectures to detect changes, Siamese and Early Fusion, and compare the impact of using different numbers of spectral channels as inputs. These architectures are trained from scratch using the provided dataset.

https://hal.archives-ouvertes.fr/hal-01899024/document

Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks

We propose an automatic video inpainting algorithm which relies on the optimization of a global, patch-based functional. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects, and moving background. Furthermore, we achieve this in an order of magnitude less execution time with respect to the state-of-the-art. We are also able to achieve good quality results on high-definition videos. Finally, we provide specific algorithmic details to make implementation of our algorithm as easy as possible. The resulting algorithm requires no segmentation or manual input other than the definition of the inpainting mask and can deal with a wider variety of situations than is handled by previous work.

https://hal.archives-ouvertes.fr/hal-00937795/document

Video Inpainting of Complex Scenes

Satellite images with high spatial resolution raise many challenging issues in image understanding and pattern recognition. First, they allow measurement of small objects maybe up to 0.5 m, and both texture and geometrical structures emerge simultaneously. Second, objects in the same type of scenes might appear at different scales and orientations. Consequently, image indexing methods should combine the structure and texture information of images and comply with some invariant properties. This paper contributes to the indexing of high-resolution satellite images. We suggest a satellite image indexing method relying on topographic maps (Caselles et al., 1999) and a shape-based image indexing scheme (Xia et al., 2009). The proposed approach contains both the textural and structural information of satellite images and is also robust to changes in scale, orientation and contrast. Experimental analysis on a real satellite image database confirms the efficiency of the approach.

/pdf/structural-high-resolution-satellite-image-indexing-3m3vcevak9.pdf

Structural high-resolution satellite image indexing

This paper explores the mathematical and algorithmic properties of two sample-based texture models: random phase noise (RPN) and asymptotic discrete spot noise (ADSN). These models permit to synthesize random phase textures. They arguably derive from linearized versions of two early Julesz texture discrimination theories. The ensuing mathematical analysis shows that, contrarily to some statements in the literature, RPN and ADSN are different stochastic processes. Nevertheless, numerous experiments also suggest that the textures obtained by these algorithms from identical samples are perceptually similar. The relevance of this study is enhanced by three technical contributions providing solutions to obstacles that prevented the use of RPN or ADSN to emulate textures. First, RPN and ADSN algorithms are extended to color images. Second, a preprocessing is proposed to avoid artifacts due to the nonperiodicity of real-world texture samples. Finally, the method is extended to synthesize textures with arbitrary size from a given sample.

/pdf/random-phase-textures-theory-and-synthesis-1z8w82aqxr.pdf

Yann Gousseau

Papers

Sar-sift: a sift-like algorithm for sar images

Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks

Video Inpainting of Complex Scenes

Structural high-resolution satellite image indexing

Random Phase Textures: Theory and Synthesis