scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data

TL;DR: This paper proposes a modular, scalable, metric free single shot change detection/registration method that exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection.
Abstract: In order to exploit the currently continuous streams of massive, multi-temporal, high-resolution remote sensing datasets there is an emerging need to address efficiently the image registration and change detection challenges. To this end, in this paper we propose a modular, scalable, metric free single shot change detection/registration method. The approach exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection. The deformation space is discretized, while efficient linear programming and duality principles are used to optimize a joint solution space where local consistency is imposed on the deformation and the detection space. Promising results on large scale experiments demonstrate the extreme potentials of our method.

Summary (3 min read)

1. Introduction

  • The current generation of space-borne and airborne sensors are generating nearly continuous streams of massive multi-temporal high resolution remote sensing data.
  • Most remote sensing and GIS software still employ semi-automated registration procedures when it comes to very large, multispectral, very high resolution satellite data [17, 7].
  • Among the various recently proposed methods, those based on Markov Random Fields [8, 24, 1], kernels [2, 26] and neural networks [22, 23] have gained important attention.
  • Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20].

2.1. MRF formulation

  • The authors have designed and built an MRF model over two different graphs of the same dimensions.
  • The interaction between the two graphs is performed by the similarity cost which connect the registration with the change detection terms.
  • Each graph is superimposed on the image [9] and therefore every node of the graph acts and depends on a subset of pixels in the vicinity (depending on the chosen interpolation strategy).
  • In particular, the dimensions of the graph are related to the image dimensions forming a trade off between accuracy and computational complexity.
  • Ech and the authors couple the two different graphs to one.

2.2. The Registration Energy Term

  • The goal of image registration is to define a transformation map T which will project the source image to the target image.
  • The energy formulation for the registration comprises of a similarity cost (that seeks to satisfy the equation 2) and a smoothness penalty on the deformation domain.
  • The similarity cost depends on the presence of changes and will be subsequently defined.
  • The smoothness term penalises neighbouring nodes that have different displacement labels, depending on the distance of the labelled displacements.

2.3. The Change Detection Energy Term

  • The goal of the change detection term is to estimate the changed and unchanged image regions.
  • The authors employ two different labels in order to address the change detection problem lcp ∈ [0, 1].
  • The energy formulation for the change detection corresponds to a smoothness term which penalizes neighbouring nodes with different change labels.

2.4. Coupling the Energy Terms

  • The coupling between change detection and registration is achieved through the interconnection between the two graphs.
  • These two terms are integrated as in (equation 5) which simply uses a fixed cost in the presence of changes and the image matching cost in their absence.
  • With a slight abuse of notation the authors consider a node with an index p ∈ G (they recall that the two graphs are identical) corresponding to the same node throughout the two graphs (Greg, Gch).
  • In such a setting, optimizing an objective function seeking similarity correspondences is not meaningful and deformation vectors should be the outcome of the smoothness constraint on the displacement space.
  • Let us consider that this value is known and that it is independent from the image displacements, so the authors can distinguish the regions that have been changed.

2.5. Optimization

  • There are several techniques for the minimization of an MRF model which can be generally summarised into those based on the message passing and those on graph cut methods.
  • The first category is related to the linear programming relaxation [14].
  • The optimization of the implementation is performed by FastPD which is based on the dual theorem of linear programming [15, 16].

3. Implementation

  • Concerning the image, iteratively different levels of Gaussian image pyramids are used.
  • In all their experiments, 2 image and 3 grid levels were found adequate for the very high resolution satellite data.
  • Regarding the label space, a search for possible displacements along 8 directions (x, y and diagonal axes) is performed, while the change labels are always two and correspond to change or no change description.
  • Depending on the parameter label factor the values of registration labels change towards the optimal ones.
  • One of the problems in traditional change detection techniques, is that change in intensities does not directly mean semantic change.

4.1. Dataset

  • The developed framework was applied to several pairs of multispectal VHR images from different satellite sensors (i.e., Quickbird and WorldView-2).
  • The multi-temporal dataset covers approximately a 9 km2 region in the East Prefecture of Attica in Greece.
  • The dataset is quite challenging both due to its size and the pictured complexity derived from the different acquisition angles.
  • For the quantitative evaluation the ground truth was manually collected and annotated after an attentive and laborious photointerpretation done by an expert.

4.2. Experimental Results

  • Regarding the evaluation for the man-made change detection task, experimental results after the application of the developed method are shown in Figure 3 and Figure 4.
  • In particular, in Figure 3 the detected changes are shown with a red color while the ground truth polygons are shown with green.
  • The behaviour of the developed method can be further observed in Figure 5, where certain examples with True Positives, False Negatives and False Positives cases are presented.
  • The selected metric affects, also, the computational time significantly.

5. Conclusions

  • Developed and validated a novel framework which address concurrently the registration and change detection tasks in very high resolution multispectral multitemporal optical satellite data.the authors.
  • The developed method is modular, scalable and metric free.
  • The formulation exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection.
  • The framework was optimized for the detection of changes related to man-made objects in urban and peri-urban environments.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

HAL Id: hal-01264072
https://hal.archives-ouvertes.fr/hal-01264072
Submitted on 16 Feb 2016
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Simultaneous Registration and Change Detection in
Multitemporal, Very High Resolution Remote Sensing
Data
Maria Vakalopoulou, Konstantinos Karatzalos, Nikos Komodakis, Nikos
Paragios
To cite this version:
Maria Vakalopoulou, Konstantinos Karatzalos, Nikos Komodakis, Nikos Paragios. Simultaneous Reg-
istration and Change Detection in Multitemporal, Very High Resolution Remote Sensing Data. 2015
IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jun 2015,
Boston, United States. pp.61-69, �10.1109/CVPRW.2015.7301384�. �hal-01264072�

Simultaneous Registration and Change Detection in Multitemporal, Very High
Resolution Remote Sensing Data
Maria Vakalopoulou
1,2
, Konstantinos Karatzalos
1
, Nikos Komodakis
2
, Nikos Paragios
3
1
Remote Sensing Laboratory, Natio-
2
Ecole des Ponts ParisTech,
3
Center for Visual Computing
nal Technical University of Athens University Paris-Est Ecole Centrale de Paris
mariavak@mail.ntua.gr nikos.komodakis@enpc.fr nikos.paragios@ecp.fr
karank@central.ntua.gr
Abstract
In order to exploit the currently continuous streams of
massive, multi-temporal, high-resolution remote sensing
datasets there is an emerging need to address efficiently the
image registration and change detection challenges. To this
end, in this paper we propose a modular, scalable, metric
free single shot change detection/registration method. The
approach exploits a decomposed interconnected graphical
model formulation where registration similarity constraints
are relaxed in the presence of change detection. The defor-
mation space is discretized, while efficient linear program-
ming and duality principles are used to optimize a joint so-
lution space where local consistency is imposed on the de-
formation and the detection space. Promising results on
large scale experiments demonstrate the extreme potentials
of our method.
1. Introduction
The current generation of space-borne and airborne sen-
sors are generating nearly continuous streams of massive
multi-temporal high resolution remote sensing data. How-
ever, in order to efficiently exploit these datasets their ac-
curate co-registration is the first indispensable processing
step. Despite the fact that image registration is among the
most studied problems in computer vision, most remote
sensing and GIS software still employ semi-automated reg-
istration procedures when it comes to very large, multispec-
tral, very high resolution satellite data [17, 7]. This is, how-
ever, far from a cost-effective solution especially if we con-
sider huge multi-temporal datasets that require accurate co-
registration [13, 12].
In addition, the primary goal of the analysis of multi-
temporal datasets is the detection of changes between differ-
ent land cover types [3, 11]. In particular, change detection
Unregistered multi-temporal satellite data
July 2006 July 2011
Registration & Change Detection Framework
E = E
reg
+ E
chng
Registration & Change Detection Framework
E
reg,ch
= V
reg,ch
+ V
pg,ch
+ V
pg,reg
Registered Data Detected Changes
Figure 1. The developed framework addresses simultaneous the
registration and change detection tasks.
of man-made objects is still an emerging challenge due to
the significant importance for various engineering and en-
vironmental applications [18, 5, 10, 25, 3]. Apart from Na-
tional and local government applications like the update of
cadastral and other GIS databases, companies like Google
and Microsoft are seeking to include extensively up-to-date
2D and 3D urban models in their products (e.g., Microsoft
Virtual Earth and Google Earth).
1

Change detection, however, from multi-temporal earth
observation data is not a trivial task and still remains a chal-
lenge. Among the various recently proposed methods, those
based on Markov Random Fields [8, 24, 1], kernels [2, 26]
and neural networks [22, 23] have gained important atten-
tion. Focusing on man-made object change detection [4, 20]
in urban and peri-urban regions, several approaches have
been proposed based on very high resolution optical and
radar data [19, 23, 6, 20]. However, these change detec-
tion techniques assume and require accurately co-registered
data in order to perform pixel-by-pixel or region-by-region
multi-temporal data fusion, correlation or any change anal-
ysis.
In this paper, we propose a simultaneous registration and
change detection approach motivated by the fact that on one
hand, the registration of very high resolution data seems to
be optimally addressed through deformation grids and pow-
erful discrete optimization [12], while on the other hand, the
desired changes are located in the regions for which corre-
spondences between the unregistered multi-temporal data
can not be established (Figure 1).
To this end, we have designed, developed and evaluated
a modular, scalable, metric-free single shot change detec-
tion/registration method. The approach exploits a decom-
posed interconnected graphical model formulation where
registration similarity constraints are relaxed in the pres-
ence of change detection. We employ a discretized, grid-
based deformation space. State-of-the-art linear program-
ming and duality principles have been employed to opti-
mize the joint solution space where local consistency is im-
posed on the deformation and the detection space. The un-
supervised framework has been designed to handle and pro-
cess large very high resolution multispectral remote sens-
ing data, while optimized for man-made object change de-
tection in urban and peri-urban regions. The developed
method has been validated through large scale experiments
on several multi-temporal very high resolution optical satel-
lite datasets.
The main contributions of the developed method are (i)
the novel, single and modular joint registration and change
detection framework, (ii) the metric-free formulation which
allows numerous and change-specific implementations, (iii)
the low computational complexity which allows near real-
time performance with gpu implementation. It should be
mentioned that the detected changes can not be directly em-
ployed for the update of, e.g., a geospatial database, since
the developed unsupervised framework does not include
any prior information about the geometry of the man-made
objects.
G
reg
G
ch
V
reg,ch
qp
Figure 2. Each graph contains a smoothness term which imposes
the necessary homogeneity within the graph. The interaction be-
tween the two graphs is performed by the similarity cost which
connect the registration with the change detection terms.
2. Methodology
2.1. MRF formulation
We have designed and built an MRF model over two dif-
ferent graphs of the same dimensions. The first deformable
graph corresponds to the registration term (G
reg
) and the
second one to the change detection term (G
ch
). Each graph
contains a smoothness term which impose the necessary ho-
mogeneity within the graph. The interaction between the
two graphs is performed by the similarity cost which con-
nect the registration with the change detection terms.
Each graph is superimposed on the image [9] and there-
fore every node of the graph acts and depends on a subset of
pixels in the vicinity (depending on the chosen interpolation
strategy). With such a manner every pixel can participate
through a certain weight to the graph related to its distance
from the nodes. The computational complexity is therefore
lower as graph’s dimensions are smaller than the unregis-
tered raw images. In particular, the dimensions of the graph
are related to the image dimensions forming a trade off be-
tween accuracy and computational complexity. In such a
setting the deformation of a pixel is defined through an in-
terpolation of the displacement of the proximal graph nodes
as follows:
T (x) = x +
X
pG
η(||x p||)d
p
(1)
where d
p
is a displacement value, η(.) is the projection
function, p is a control point and x is a pixel in the image.
After the optimization, the optimal labels will be pro-
jected to the image pixels using the same projection func-
tion η(.). Once the similarity criterion has been defined, the
next step consists of imposing certain continuity on the de-
formation space which is discussed in the next subsection.
That way, we formulate an energy function E
reg,ch
=
E
reg
+ E
ch
and we couple the two different graphs to one.

The label for each node p belonging to the graph G, will
be l
p
= [l
c
, l
reg
] where l
c
are the labels for the change de-
tection, l
c
{0, 1} and l
reg
are the labels for the registra-
tion l
reg
where = [d
1
, . . . , d
n
] corresponding to all
possible displacements. Concluding the label space can be
summarized as L = {0, 1} × .
2.2. The Registration Energy Term
Let us denote a pair of images where A is the source
image and V is the target image defined on a domain . The
goal of image registration is to define a transformation map
T which will project the source image to the target image.
V (x) = A T (x) (2)
Let us consider a discrete set of labels L
reg
= [1, . . . , n],
and a set of discrete displacements = [d
1
, . . . , d
n
]. We
seek to assign a label l
reg
p
to each grid node p, where each
label corresponds to a discrete displacement d
l
reg
p
.
The energy formulation for the registration comprises of
a similarity cost (that seeks to satisfy the equation 2) and a
smoothness penalty on the deformation domain. The sim-
ilarity cost depends on the presence of changes and will
be subsequently defined. The smoothness term penalises
neighbouring nodes that have different displacement labels,
depending on the distance of the labelled displacements.
V
pq ,reg
(l
reg
p
, l
reg
q
) = ||d
l
reg
p
d
l
reg
q
|| (3)
where p and q are neighbouring nodes.
2.3. The Change Detection Energy Term
The goal of the change detection term is to estimate the
changed and unchanged image regions. We employ two dif-
ferent labels in order to address the change detection prob-
lem l
c
p
[0, 1]. The energy formulation for the change de-
tection corresponds to a smoothness term which penalizes
neighbouring nodes with different change labels.
V
pq ,ch
(l
c
p
, l
c
q
) = ||l
c
p
l
c
q
|| (4)
2.4. Coupling the Energy Terms
The coupling between change detection and registration
is achieved through the interconnection between the two
graphs. Assuming a pair of corresponding nodes of the two
graphs, one would expect that in the absence of change the
similarity cost should be satisfied and in that case the poten-
tial will be:
V
reg,ch
(l
reg
p
, l
c
p
) = (1 l
c
p
) ·
Z
ˆη(||x p||)
ρ(V (x), A(x + d
l
reg
p
))dx + l
c
p
· C
(5)
where we simply take all pixels in the vicinity of the graph
node and project them back to the grid node with a weight
that is proportional to the distance. In the presence of
change, we use a fixed cost C. These two terms are in-
tegrated as in (equation 5) which simply uses a fixed cost
in the presence of changes and the image matching cost in
their absence.
With a slight abuse of notation we consider a node with
an index p G (we recall that the two graphs are identical)
corresponding to the same node throughout the two graphs
(G
reg
, G
ch
).We can now integrate all terms to a single en-
ergy which detect changes, establishes correspondences and
impose smoothness in the change detection and the defor-
mation map as follows:
E
reg,ch
(l
c
, l
reg
) = w
1
·
X
pG
V
reg,ch
(l
reg
p
, l
c
p
)+
w
2
·
X
pG
reg
X
q N (p)
V
pq ,reg
(l
reg
p
, l
reg
q
)+
w
3
·
X
pG
ch
X
q N (p)
V
pq ,ch
(l
c
p
, l
c
q
)
(6)
where V
reg,ch
(l
reg
p
, l
c
p
) represents the coupling term for
each node at each label, V
pq ,reg
(l
reg
p
, l
reg
q
) the pairwise or
binary term for the registration and V
pq ,ch
(l
c
p
, l
c
q
) the pair-
wise for the change detection.
In particular, a similarity function ρ(.) is used in order
to compare the two images, while a constant value C is
used in order to define the changes. In such a setting, opti-
mizing an objective function seeking similarity correspon-
dences is not meaningful and deformation vectors should be
the outcome of the smoothness constraint on the displace-
ment space. However, the areas of change are unknown and
is one of the objective of the optimization process. Without
loss of generality we can assume that the matching cost cor-
responding to change can correspond to a value that can be
determined from the distribution of these costs on the en-
tire domain (it is metric dependent). Let us consider that
this value is known and that it is independent from the im-
age displacements, so we can distinguish the regions that
have been changed. The advantage of the methodology is
that by solving the two problems simultaneously first we
have less false changes caused by the misregistration of the
images and then the changed regions do not affect the en-
tire displacement map, as we do not calculate the displace-
ment there and their final displacement is caused by the un-
changed neighbour regions.
Finally, the pairwise costs for both terms have been de-
scribed in equation 3 and 4.
2.5. Optimization
There are several techniques for the minimization of an
MRF model which can be generally summarised into those

based on the message passing and those on graph cut meth-
ods. The first category is related to the linear programming
relaxation [14]. The optimization of the implementation is
performed by FastPD which is based on the dual theorem of
linear programming [15, 16].
3. Implementation
The minimization of the MRF energy is performed by a
multi-scale framework. Concerning the image, iteratively
different levels of Gaussian image pyramids are used. Con-
cerning the grid, in a similar way we consider different lev-
els of it, beginning with a sparser grid. Having very large re-
mote sensing imagery the multi-scale approach diminishes
the computational complexity without losing in terms of ac-
curacy. The different levels of the images and the grid with
the consistency of nodes in the grid are defined by the user.
In all our experiments, 2 image and 3 grid levels were found
adequate for the very high resolution satellite data.
Regarding the label space, a search for possible displace-
ments along 8 directions (x, y and diagonal axes) is per-
formed, while the change labels are always two and corre-
spond to change or no change description. The number of
the registration labels is the same at each level. Depending
on the parameter label factor the values of registration la-
bels change towards the optimal ones. The source image is
deformed according to the optimal labels and it is updated
for the next level. The value 0.8 was employed for updat-
ing the registration labels. Last but not least, the maximum
displacement was set smaller than 0.4 times the distance of
two consecutive nodes in order to preserve the right dis-
placement of every node.
In addition, several methods block matching methods
can be employed. Semantic changes in multitemporal im-
agery affect the local intensities and also change the struc-
ture of the region. One of the problems in traditional change
detection techniques, is that change in intensities does not
directly mean semantic change. This was crucial since the
focus, here, was on urban and peri-urban regions and man-
made object changes. The optimal displacement using the
SADG function are calculated using the weighted sum of
the difference between the pair and the gradient inner prod-
uct. On the other hand any other similarity measure as mu-
tual information, normalized cross correlation, ratio corre-
lation can be used. In Section 4 we have tested different
similarity functions. For the SADG metric and focused on
man-made object changes, the fixed cost C was set to 100.
In particular, higher C values result to less changes. The
parameter is not so sensitive since values between 90 to 120
lead to comparable results.
Last but not least, the number of iterations per level was
set to 10, the regularization parameter for the registration
task to 35 and for the change detection one to 3.5. The
function used for the projection from pixels to nodes and
reverse was the Cubic B-splines.
4. Experimental Results and Evaluation
4.1. Dataset
The developed framework was applied to several pairs
of multispectal VHR images from different satellite sen-
sors (i.e., Quickbird and WorldView-2). The multi-temporal
dataset covers approximately a 9 km
2
region in the East
Prefecture of Attica in Greece. All datasets were acquired
between the years of 2006 and 2011. The dataset is quite
challenging both due to its size and the pictured complex-
ity derived from the different acquisition angles. For the
quantitative evaluation the ground truth was manually col-
lected and annotated after an attentive and laborious photo-
interpretation done by an expert.
4.2. Experimental Results
Extensive experiments were performed over several im-
ages pairs and based on several similarity metrics namely
the Sum of Absolute Differences (SAD), the Sum of
Square Differences (SSD), the Normalized Cross Correla-
tion (NCC), the Normalized Mutual Information (NMI), the
Correlation Ratio (CR), the Sum of Gradient Inner Products
(GRAD), the Normalized Correlation Coefficient plus Sum
of Gradient Inner Products (CCGIP), the Hellinger Distance
(HD), the Jensen-Renyi Divergence (JRD), the Mutual In-
formation (MI) and the Sum Absolute of Differences plus
Gradient Inner Products (SADG). The experimental results
were evaluated both qualitative and quantitative for the reg-
istration and the change detection tasks.
Moreover, in order to evaluate quantitatively the devel-
oped algorithm, the standard quality metrics of Complete-
ness, Correctness and Quality were calculated on the de-
tected object level. The True Positives (TP), False Nega-
tives (FN) and False Positives (FP) were calculated in all
cases.
Completeness =
T P
T P + F N
(7)
Correctness =
T P
T P + F P
(8)
Quality =
T P
T P + F P + F N
(9)
where TP is the number of correctly detected changes, FN
is the number of changes that haven’t been detected by the
algorithm and FP is the number of false alarms.
Regarding the evaluation of the registration, a number
of ground control points (GCPs) were manually collected
in both unregistered and registered data. In particular, the
GCPs contained several points on building roof tops which
usually have the largest displacements. The displacement

Citations
More filters
Journal ArticleDOI
07 Aug 2015
TL;DR: A taxonomical view of the field is provided and the current methodologies for multimodal classification of remote sensing images are reviewed, which highlight the most recent advances, which exploit synergies with machine learning and signal processing.
Abstract: Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images. We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image classification; 4) multisensor image fusion exploiting physically-based feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.

319 citations

Journal ArticleDOI
TL;DR: This paper reviews the recent developments and applications of 3D CD using remote sensing and close-range data, in support of both academia and industry researchers who seek for solutions in detecting and analyzing 3D dynamics of various objects of interest.
Abstract: Due to the unprecedented technology development of sensors, platforms and algorithms for 3D data acquisition and generation, 3D spaceborne, airborne and close-range data, in the form of image based, Light Detection and Ranging (LiDAR) based point clouds, Digital Elevation Models (DEM) and 3D city models, become more accessible than ever before Change detection (CD) or time-series data analysis in 3D has gained great attention due to its capability of providing volumetric dynamics to facilitate more applications and provide more accurate results The state-of-the-art CD reviews aim to provide a comprehensive synthesis and to simplify the taxonomy of the traditional remote sensing CD techniques, which mainly sit within the boundary of 2D image/spectrum analysis, largely ignoring the particularities of 3D aspects of the data The inclusion of 3D data for change detection (termed 3D CD), not only provides a source with different modality for analysis, but also transcends the border of traditional top-view 2D pixel/object-based analysis to highly detailed, oblique view or voxel-based geometric analysis This paper reviews the recent developments and applications of 3D CD using remote sensing and close-range data, in support of both academia and industry researchers who seek for solutions in detecting and analyzing 3D dynamics of various objects of interest We first describe the general considerations of 3D CD problems in different processing stages and identify CD types based on the information used, being the geometric comparison and geometric-spectral analysis We then summarize relevant works and practices in urban, environment, ecology and civil applications, etc Given the broad spectrum of applications and different types of 3D data, we discuss important issues in 3D CD methods Finally, we present concluding remarks in algorithmic aspects of 3D CD

200 citations


Cites background from "Simultaneous registration and chang..."

  • ...A few of them investigated the possibility of using very high resolution (VHR) images for 2D CD in a finer level (Bouziani et al., 2010; Brunner et al., 2010; Huang et al., 2014; Košecka, 2012; Vakalopoulou et al., 2015)....

    [...]

Journal ArticleDOI
TL;DR: This paper presents the first large scale very high resolution semantic change detection dataset, which enables the usage of deep supervised learning methods for semantic changes detection with very highresolution images, and presents a network architecture that performs change detection and land cover mapping simultaneously.

142 citations


Cites methods from "Simultaneous registration and chang..."

  • ...Unsupervised methods have been used for change detection in many different ways (Hussain et al., 2013; Vakalopoulou et al., 2015; Liu et al., 2019)....

    [...]

Proceedings ArticleDOI
28 Jul 2019
TL;DR: A novel deep learning framework for urban change detection which combines state-of-the-art fully convolutional networks (similar to U-Net) for feature representation and powerful recurrent networks (such as LSTMs) for temporal modeling is presented.
Abstract: The advent of multitemporal high resolution data, like the Copernicus Sentinel-2, has enhanced significantly the potential of monitoring the earth’s surface and environmental dynamics. In this paper, we present a novel deep learning framework for urban change detection which combines state-of-the-art fully convolutional networks (similar to U-Net) for feature representation and powerful recurrent networks (such as LSTMs) for temporal modeling. We report our results on the recently publicly available bi-temporal Onera Satellite Change Detection (OSCD) Sentinel-2 dataset, enhancing the temporal information with additional images of the same region on different dates. Moreover, we evaluate the performance of the recurrent networks as well as the use of the additional dates on the unseen test-set using an ensemble cross-validation strategy. All the developed models during the validation phase have scored an overall accuracy of more than 95%, while the use of LSTMs and further temporal information, boost the F1 rate of the change class by an additional 1.5%.

103 citations


Cites background from "Simultaneous registration and chang..."

  • ...Change detection is a non-trivial problem as the accuracy of a method is highly influenced by registration errors [2] and illumination changes that do not really correspond to semantic changes....

    [...]

Journal ArticleDOI
TL;DR: The winning team developed a deep learning method to jointly address spatial scene labeling and temporal activity modeling using the available image and video data, and the second place team proposed a random field model to simultaneously perform coregistration of multitemporal data, semantic segmentation, and change detection.
Abstract: In this paper, the scientific outcomes of the 2016 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society are discussed. The 2016 Contest was an open topic competition based on a multitemporal and multimodal dataset, which included a temporal pair of very high resolution panchromatic and multispectral Deimos-2 images and a video captured by the Iris camera on-board the International Space Station. The problems addressed and the techniques proposed by the participants to the Contest spanned across a rather broad range of topics, and mixed ideas and methodologies from the remote sensing, video processing, and computer vision. In particular, the winning team developed a deep learning method to jointly address spatial scene labeling and temporal activity modeling using the available image and video data. The second place team proposed a random field model to simultaneously perform coregistration of multitemporal data, semantic segmentation, and change detection. The methodological key ideas of both these approaches and the main results of the corresponding experimental validation are discussed in this paper.

66 citations


Cites background or methods from "Simultaneous registration and chang..."

  • ...Following the notations of [6] and [55], the first graph, Greg, involved nodes where the labels corresponded to deformation vectors from the registration process, i....

    [...]

  • ...1) the registration (Vpq,reg(l reg p , l reg q )) and change detection (Vpq,ch(l p , l ch q )) pairwise terms followed the same formulation as in [6] and [55] and penalized neighboring nodes...

    [...]

  • ...In particular, the formulation of [6] and [55], was extended...

    [...]

References
More filters
Journal ArticleDOI
TL;DR: Results demonstrate that the proposed approach to building change detection in multitemporal VHR synthetic aperture radar images allows an accurate identification of new and demolished buildings while presents a low false-alarm rate and a high reliability.
Abstract: The increasing availability of very high resolution (VHR) images regularly acquired over urban areas opens new attractive opportunities for monitoring human settlements at the level of individual buildings. This paper presents a novel approach to building change detection in multitemporal VHR synthetic aperture radar (SAR) images. The proposed approach is based on two concepts: 1) the extraction of information on changes associated with increase and decrease of backscattering at the optimal building scale and 2) the exploitation of the expected backscattering properties of buildings to detect either new or fully demolished buildings. Each detected change is associated with a grade of reliability. The approach is validated on the following: 1) COSMO-SkyMed multitemporal spotlight images acquired in 2009 on the city of L'Aquila (Italy) before and after the earthquake that hit the region and 2) TerraSAR-X multitemporal spotlight images acquired on the urban area of the city of Trento (Italy). Results demonstrate that the proposed approach allows an accurate identification of new and demolished buildings while presents a low false-alarm rate and a high reliability.

133 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20]....

    [...]

Journal ArticleDOI
TL;DR: An unsupervised kernel-based approach to change detection is introduced and an evidence of the correctness, stability, and superiority of the proposed solution is provided through the analysis of two challenging change-detection problems.
Abstract: In this letter, an unsupervised kernel-based approach to change detection is introduced. Nonlinear clustering is utilized to partition in two a selected subset of pixels representing both changed and unchanged areas. Once the optimal clustering is obtained, the learned representatives of each group are exploited to assign all the pixels composing the multitemporal scenes to the two classes of interest. Two approaches based on different assumptions of the difference image are proposed. The first accounts for the difference image in the original space, while the second defines a mapping describing the difference image directly in feature spaces. To optimize the parameters of the kernels, a novel unsupervised cost function is proposed. An evidence of the correctness, stability, and superiority of the proposed solution is provided through the analysis of two challenging change-detection problems.

100 citations


"Simultaneous registration and chang..." refers methods in this paper

  • ...Among the various recently proposed methods, those based on Markov Random Fields [8, 24, 1], kernels [2, 26] and neural networks [22, 23] have gained important attention....

    [...]

Journal ArticleDOI
TL;DR: The experiments performed on panchromatic QuickBird images related to an urban area show the effectiveness of the proposed technique in detecting changes on the basis of the spatial morphology by preserving geometrical detail.
Abstract: A new approach to change detection in very high resolution remote sensing images based on morphological attribute profiles (APs) is presented. A multiresolution contextual transformation performed by APs allows the extraction of geometrical features related to the structures within the scene at different scales. The temporal changes are detected by comparing the geometrical features extracted from the image of each date. The experiments performed on panchromatic QuickBird images related to an urban area show the effectiveness of the proposed technique in detecting changes on the basis of the spatial morphology by preserving geometrical detail.

97 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20]....

    [...]

Journal ArticleDOI
TL;DR: An automatic context-sensitive technique robust to registration noise (RN) for change detection (CD) in multitemporal very high geometrical resolution (VHR) remote sensing images and experimental results obtained confirm the effectiveness of the proposed technique.
Abstract: This paper presents an automatic context-sensitive technique robust to registration noise (RN) for change detection (CD) in multitemporal very high geometrical resolution (VHR) remote sensing images. Exploiting the properties of RN in VHR images, the proposed technique analyzes the distribution of the spectral change vectors (SCVs) computed according to the change vector analysis (CVA) in a quantized polar domain. The method studies the SCVs falling into each quantization cell at different resolution levels (scales) to automatically identify the effects of RN in the polar domain. This information is jointly exploited with the spatial context information contained in the neighborhood of each pixel for generating the final CD map. The spatial context information is modeled through the definition of adaptive regions homogeneous both in spatial and temporal domain (parcels). Experimental results obtained on real VHR remote sensing multitemporal images confirm the effectiveness of the proposed technique.

89 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20]....

    [...]

Book
15 Dec 2009
TL;DR: This significantly expanded edition presents numerous image analysis examples and algorithms, all illustrated in the array-oriented language IDL, allowing readers to plug the illustrations and applications covered in the text directly into the ENVI system in a completely transparent fashion.
Abstract: Demonstrating the breadth and depth of growth in the field since the publication of the popular first edition, Image Analysis, Classification and Change Detection in Remote Sensing, with Algorithms for ENVI/IDL, Second Edition has been updated and expanded to keep pace with the latest versions of the ENVI software environment. Effectively interweaving theory, algorithms, and computer codes, the text supplies an accessible introduction to the techniques used in the processing of remote sensing imagery. This significantly expanded edition presents numerous image analysis examples and algorithms, all illustrated in the array-oriented language IDLallowing readers to plug the illustrations and applications covered in the text directly into the ENVI systemin a completely transparent fashion. Revised chapters on image arrays, linear algebra, and statistics convey the required foundation, while updated chapters detail kernel methods for principal component analysis, kernel-based clustering, and classification with support vector machines. New additions to the second edition include: An introduction to mutual information and entropy Algorithms and code for image segmentation In-depth treatment of ensemble classification (adaptive boosting ) Improved IDL code for all ENVI extensions, with routines that can take advantage of the parallel computational power of modern graphics processors Code that runs on all versions of the ENVI/IDL software environment from ENVI 4.1 up to the present Many new end-of-chapter exercises and programming projects With its numerous programming examples in IDL and many applications supporting ENVI, such as data fusion, statistical change detection, clustering and supervised classification with neural networksall available as downloadable source codethis self-contained text is the ideal resource for classroom use or self study.

81 citations

Frequently Asked Questions (2)
Q1. What have the authors contributed in "Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data" ?

To this end, in this paper the authors propose a modular, scalable, metric free single shot change detection/registration method. Promising results on large scale experiments demonstrate the extreme potentials of their method. 

The integration of prior knowledge regarding texture and geometric features is currently under development and a gpu implementation is among the future perspectives as well.