scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data

TL;DR: This paper proposes a modular, scalable, metric free single shot change detection/registration method that exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection.
Abstract: In order to exploit the currently continuous streams of massive, multi-temporal, high-resolution remote sensing datasets there is an emerging need to address efficiently the image registration and change detection challenges. To this end, in this paper we propose a modular, scalable, metric free single shot change detection/registration method. The approach exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection. The deformation space is discretized, while efficient linear programming and duality principles are used to optimize a joint solution space where local consistency is imposed on the deformation and the detection space. Promising results on large scale experiments demonstrate the extreme potentials of our method.

Summary (3 min read)

1. Introduction

  • The current generation of space-borne and airborne sensors are generating nearly continuous streams of massive multi-temporal high resolution remote sensing data.
  • Most remote sensing and GIS software still employ semi-automated registration procedures when it comes to very large, multispectral, very high resolution satellite data [17, 7].
  • Among the various recently proposed methods, those based on Markov Random Fields [8, 24, 1], kernels [2, 26] and neural networks [22, 23] have gained important attention.
  • Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20].

2.1. MRF formulation

  • The authors have designed and built an MRF model over two different graphs of the same dimensions.
  • The interaction between the two graphs is performed by the similarity cost which connect the registration with the change detection terms.
  • Each graph is superimposed on the image [9] and therefore every node of the graph acts and depends on a subset of pixels in the vicinity (depending on the chosen interpolation strategy).
  • In particular, the dimensions of the graph are related to the image dimensions forming a trade off between accuracy and computational complexity.
  • Ech and the authors couple the two different graphs to one.

2.2. The Registration Energy Term

  • The goal of image registration is to define a transformation map T which will project the source image to the target image.
  • The energy formulation for the registration comprises of a similarity cost (that seeks to satisfy the equation 2) and a smoothness penalty on the deformation domain.
  • The similarity cost depends on the presence of changes and will be subsequently defined.
  • The smoothness term penalises neighbouring nodes that have different displacement labels, depending on the distance of the labelled displacements.

2.3. The Change Detection Energy Term

  • The goal of the change detection term is to estimate the changed and unchanged image regions.
  • The authors employ two different labels in order to address the change detection problem lcp ∈ [0, 1].
  • The energy formulation for the change detection corresponds to a smoothness term which penalizes neighbouring nodes with different change labels.

2.4. Coupling the Energy Terms

  • The coupling between change detection and registration is achieved through the interconnection between the two graphs.
  • These two terms are integrated as in (equation 5) which simply uses a fixed cost in the presence of changes and the image matching cost in their absence.
  • With a slight abuse of notation the authors consider a node with an index p ∈ G (they recall that the two graphs are identical) corresponding to the same node throughout the two graphs (Greg, Gch).
  • In such a setting, optimizing an objective function seeking similarity correspondences is not meaningful and deformation vectors should be the outcome of the smoothness constraint on the displacement space.
  • Let us consider that this value is known and that it is independent from the image displacements, so the authors can distinguish the regions that have been changed.

2.5. Optimization

  • There are several techniques for the minimization of an MRF model which can be generally summarised into those based on the message passing and those on graph cut methods.
  • The first category is related to the linear programming relaxation [14].
  • The optimization of the implementation is performed by FastPD which is based on the dual theorem of linear programming [15, 16].

3. Implementation

  • Concerning the image, iteratively different levels of Gaussian image pyramids are used.
  • In all their experiments, 2 image and 3 grid levels were found adequate for the very high resolution satellite data.
  • Regarding the label space, a search for possible displacements along 8 directions (x, y and diagonal axes) is performed, while the change labels are always two and correspond to change or no change description.
  • Depending on the parameter label factor the values of registration labels change towards the optimal ones.
  • One of the problems in traditional change detection techniques, is that change in intensities does not directly mean semantic change.

4.1. Dataset

  • The developed framework was applied to several pairs of multispectal VHR images from different satellite sensors (i.e., Quickbird and WorldView-2).
  • The multi-temporal dataset covers approximately a 9 km2 region in the East Prefecture of Attica in Greece.
  • The dataset is quite challenging both due to its size and the pictured complexity derived from the different acquisition angles.
  • For the quantitative evaluation the ground truth was manually collected and annotated after an attentive and laborious photointerpretation done by an expert.

4.2. Experimental Results

  • Regarding the evaluation for the man-made change detection task, experimental results after the application of the developed method are shown in Figure 3 and Figure 4.
  • In particular, in Figure 3 the detected changes are shown with a red color while the ground truth polygons are shown with green.
  • The behaviour of the developed method can be further observed in Figure 5, where certain examples with True Positives, False Negatives and False Positives cases are presented.
  • The selected metric affects, also, the computational time significantly.

5. Conclusions

  • Developed and validated a novel framework which address concurrently the registration and change detection tasks in very high resolution multispectral multitemporal optical satellite data.the authors.
  • The developed method is modular, scalable and metric free.
  • The formulation exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection.
  • The framework was optimized for the detection of changes related to man-made objects in urban and peri-urban environments.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

HAL Id: hal-01264072
https://hal.archives-ouvertes.fr/hal-01264072
Submitted on 16 Feb 2016
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Simultaneous Registration and Change Detection in
Multitemporal, Very High Resolution Remote Sensing
Data
Maria Vakalopoulou, Konstantinos Karatzalos, Nikos Komodakis, Nikos
Paragios
To cite this version:
Maria Vakalopoulou, Konstantinos Karatzalos, Nikos Komodakis, Nikos Paragios. Simultaneous Reg-
istration and Change Detection in Multitemporal, Very High Resolution Remote Sensing Data. 2015
IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jun 2015,
Boston, United States. pp.61-69, �10.1109/CVPRW.2015.7301384�. �hal-01264072�

Simultaneous Registration and Change Detection in Multitemporal, Very High
Resolution Remote Sensing Data
Maria Vakalopoulou
1,2
, Konstantinos Karatzalos
1
, Nikos Komodakis
2
, Nikos Paragios
3
1
Remote Sensing Laboratory, Natio-
2
Ecole des Ponts ParisTech,
3
Center for Visual Computing
nal Technical University of Athens University Paris-Est Ecole Centrale de Paris
mariavak@mail.ntua.gr nikos.komodakis@enpc.fr nikos.paragios@ecp.fr
karank@central.ntua.gr
Abstract
In order to exploit the currently continuous streams of
massive, multi-temporal, high-resolution remote sensing
datasets there is an emerging need to address efficiently the
image registration and change detection challenges. To this
end, in this paper we propose a modular, scalable, metric
free single shot change detection/registration method. The
approach exploits a decomposed interconnected graphical
model formulation where registration similarity constraints
are relaxed in the presence of change detection. The defor-
mation space is discretized, while efficient linear program-
ming and duality principles are used to optimize a joint so-
lution space where local consistency is imposed on the de-
formation and the detection space. Promising results on
large scale experiments demonstrate the extreme potentials
of our method.
1. Introduction
The current generation of space-borne and airborne sen-
sors are generating nearly continuous streams of massive
multi-temporal high resolution remote sensing data. How-
ever, in order to efficiently exploit these datasets their ac-
curate co-registration is the first indispensable processing
step. Despite the fact that image registration is among the
most studied problems in computer vision, most remote
sensing and GIS software still employ semi-automated reg-
istration procedures when it comes to very large, multispec-
tral, very high resolution satellite data [17, 7]. This is, how-
ever, far from a cost-effective solution especially if we con-
sider huge multi-temporal datasets that require accurate co-
registration [13, 12].
In addition, the primary goal of the analysis of multi-
temporal datasets is the detection of changes between differ-
ent land cover types [3, 11]. In particular, change detection
Unregistered multi-temporal satellite data
July 2006 July 2011
Registration & Change Detection Framework
E = E
reg
+ E
chng
Registration & Change Detection Framework
E
reg,ch
= V
reg,ch
+ V
pg,ch
+ V
pg,reg
Registered Data Detected Changes
Figure 1. The developed framework addresses simultaneous the
registration and change detection tasks.
of man-made objects is still an emerging challenge due to
the significant importance for various engineering and en-
vironmental applications [18, 5, 10, 25, 3]. Apart from Na-
tional and local government applications like the update of
cadastral and other GIS databases, companies like Google
and Microsoft are seeking to include extensively up-to-date
2D and 3D urban models in their products (e.g., Microsoft
Virtual Earth and Google Earth).
1

Change detection, however, from multi-temporal earth
observation data is not a trivial task and still remains a chal-
lenge. Among the various recently proposed methods, those
based on Markov Random Fields [8, 24, 1], kernels [2, 26]
and neural networks [22, 23] have gained important atten-
tion. Focusing on man-made object change detection [4, 20]
in urban and peri-urban regions, several approaches have
been proposed based on very high resolution optical and
radar data [19, 23, 6, 20]. However, these change detec-
tion techniques assume and require accurately co-registered
data in order to perform pixel-by-pixel or region-by-region
multi-temporal data fusion, correlation or any change anal-
ysis.
In this paper, we propose a simultaneous registration and
change detection approach motivated by the fact that on one
hand, the registration of very high resolution data seems to
be optimally addressed through deformation grids and pow-
erful discrete optimization [12], while on the other hand, the
desired changes are located in the regions for which corre-
spondences between the unregistered multi-temporal data
can not be established (Figure 1).
To this end, we have designed, developed and evaluated
a modular, scalable, metric-free single shot change detec-
tion/registration method. The approach exploits a decom-
posed interconnected graphical model formulation where
registration similarity constraints are relaxed in the pres-
ence of change detection. We employ a discretized, grid-
based deformation space. State-of-the-art linear program-
ming and duality principles have been employed to opti-
mize the joint solution space where local consistency is im-
posed on the deformation and the detection space. The un-
supervised framework has been designed to handle and pro-
cess large very high resolution multispectral remote sens-
ing data, while optimized for man-made object change de-
tection in urban and peri-urban regions. The developed
method has been validated through large scale experiments
on several multi-temporal very high resolution optical satel-
lite datasets.
The main contributions of the developed method are (i)
the novel, single and modular joint registration and change
detection framework, (ii) the metric-free formulation which
allows numerous and change-specific implementations, (iii)
the low computational complexity which allows near real-
time performance with gpu implementation. It should be
mentioned that the detected changes can not be directly em-
ployed for the update of, e.g., a geospatial database, since
the developed unsupervised framework does not include
any prior information about the geometry of the man-made
objects.
G
reg
G
ch
V
reg,ch
qp
Figure 2. Each graph contains a smoothness term which imposes
the necessary homogeneity within the graph. The interaction be-
tween the two graphs is performed by the similarity cost which
connect the registration with the change detection terms.
2. Methodology
2.1. MRF formulation
We have designed and built an MRF model over two dif-
ferent graphs of the same dimensions. The first deformable
graph corresponds to the registration term (G
reg
) and the
second one to the change detection term (G
ch
). Each graph
contains a smoothness term which impose the necessary ho-
mogeneity within the graph. The interaction between the
two graphs is performed by the similarity cost which con-
nect the registration with the change detection terms.
Each graph is superimposed on the image [9] and there-
fore every node of the graph acts and depends on a subset of
pixels in the vicinity (depending on the chosen interpolation
strategy). With such a manner every pixel can participate
through a certain weight to the graph related to its distance
from the nodes. The computational complexity is therefore
lower as graph’s dimensions are smaller than the unregis-
tered raw images. In particular, the dimensions of the graph
are related to the image dimensions forming a trade off be-
tween accuracy and computational complexity. In such a
setting the deformation of a pixel is defined through an in-
terpolation of the displacement of the proximal graph nodes
as follows:
T (x) = x +
X
pG
η(||x p||)d
p
(1)
where d
p
is a displacement value, η(.) is the projection
function, p is a control point and x is a pixel in the image.
After the optimization, the optimal labels will be pro-
jected to the image pixels using the same projection func-
tion η(.). Once the similarity criterion has been defined, the
next step consists of imposing certain continuity on the de-
formation space which is discussed in the next subsection.
That way, we formulate an energy function E
reg,ch
=
E
reg
+ E
ch
and we couple the two different graphs to one.

The label for each node p belonging to the graph G, will
be l
p
= [l
c
, l
reg
] where l
c
are the labels for the change de-
tection, l
c
{0, 1} and l
reg
are the labels for the registra-
tion l
reg
where = [d
1
, . . . , d
n
] corresponding to all
possible displacements. Concluding the label space can be
summarized as L = {0, 1} × .
2.2. The Registration Energy Term
Let us denote a pair of images where A is the source
image and V is the target image defined on a domain . The
goal of image registration is to define a transformation map
T which will project the source image to the target image.
V (x) = A T (x) (2)
Let us consider a discrete set of labels L
reg
= [1, . . . , n],
and a set of discrete displacements = [d
1
, . . . , d
n
]. We
seek to assign a label l
reg
p
to each grid node p, where each
label corresponds to a discrete displacement d
l
reg
p
.
The energy formulation for the registration comprises of
a similarity cost (that seeks to satisfy the equation 2) and a
smoothness penalty on the deformation domain. The sim-
ilarity cost depends on the presence of changes and will
be subsequently defined. The smoothness term penalises
neighbouring nodes that have different displacement labels,
depending on the distance of the labelled displacements.
V
pq ,reg
(l
reg
p
, l
reg
q
) = ||d
l
reg
p
d
l
reg
q
|| (3)
where p and q are neighbouring nodes.
2.3. The Change Detection Energy Term
The goal of the change detection term is to estimate the
changed and unchanged image regions. We employ two dif-
ferent labels in order to address the change detection prob-
lem l
c
p
[0, 1]. The energy formulation for the change de-
tection corresponds to a smoothness term which penalizes
neighbouring nodes with different change labels.
V
pq ,ch
(l
c
p
, l
c
q
) = ||l
c
p
l
c
q
|| (4)
2.4. Coupling the Energy Terms
The coupling between change detection and registration
is achieved through the interconnection between the two
graphs. Assuming a pair of corresponding nodes of the two
graphs, one would expect that in the absence of change the
similarity cost should be satisfied and in that case the poten-
tial will be:
V
reg,ch
(l
reg
p
, l
c
p
) = (1 l
c
p
) ·
Z
ˆη(||x p||)
ρ(V (x), A(x + d
l
reg
p
))dx + l
c
p
· C
(5)
where we simply take all pixels in the vicinity of the graph
node and project them back to the grid node with a weight
that is proportional to the distance. In the presence of
change, we use a fixed cost C. These two terms are in-
tegrated as in (equation 5) which simply uses a fixed cost
in the presence of changes and the image matching cost in
their absence.
With a slight abuse of notation we consider a node with
an index p G (we recall that the two graphs are identical)
corresponding to the same node throughout the two graphs
(G
reg
, G
ch
).We can now integrate all terms to a single en-
ergy which detect changes, establishes correspondences and
impose smoothness in the change detection and the defor-
mation map as follows:
E
reg,ch
(l
c
, l
reg
) = w
1
·
X
pG
V
reg,ch
(l
reg
p
, l
c
p
)+
w
2
·
X
pG
reg
X
q N (p)
V
pq ,reg
(l
reg
p
, l
reg
q
)+
w
3
·
X
pG
ch
X
q N (p)
V
pq ,ch
(l
c
p
, l
c
q
)
(6)
where V
reg,ch
(l
reg
p
, l
c
p
) represents the coupling term for
each node at each label, V
pq ,reg
(l
reg
p
, l
reg
q
) the pairwise or
binary term for the registration and V
pq ,ch
(l
c
p
, l
c
q
) the pair-
wise for the change detection.
In particular, a similarity function ρ(.) is used in order
to compare the two images, while a constant value C is
used in order to define the changes. In such a setting, opti-
mizing an objective function seeking similarity correspon-
dences is not meaningful and deformation vectors should be
the outcome of the smoothness constraint on the displace-
ment space. However, the areas of change are unknown and
is one of the objective of the optimization process. Without
loss of generality we can assume that the matching cost cor-
responding to change can correspond to a value that can be
determined from the distribution of these costs on the en-
tire domain (it is metric dependent). Let us consider that
this value is known and that it is independent from the im-
age displacements, so we can distinguish the regions that
have been changed. The advantage of the methodology is
that by solving the two problems simultaneously first we
have less false changes caused by the misregistration of the
images and then the changed regions do not affect the en-
tire displacement map, as we do not calculate the displace-
ment there and their final displacement is caused by the un-
changed neighbour regions.
Finally, the pairwise costs for both terms have been de-
scribed in equation 3 and 4.
2.5. Optimization
There are several techniques for the minimization of an
MRF model which can be generally summarised into those

based on the message passing and those on graph cut meth-
ods. The first category is related to the linear programming
relaxation [14]. The optimization of the implementation is
performed by FastPD which is based on the dual theorem of
linear programming [15, 16].
3. Implementation
The minimization of the MRF energy is performed by a
multi-scale framework. Concerning the image, iteratively
different levels of Gaussian image pyramids are used. Con-
cerning the grid, in a similar way we consider different lev-
els of it, beginning with a sparser grid. Having very large re-
mote sensing imagery the multi-scale approach diminishes
the computational complexity without losing in terms of ac-
curacy. The different levels of the images and the grid with
the consistency of nodes in the grid are defined by the user.
In all our experiments, 2 image and 3 grid levels were found
adequate for the very high resolution satellite data.
Regarding the label space, a search for possible displace-
ments along 8 directions (x, y and diagonal axes) is per-
formed, while the change labels are always two and corre-
spond to change or no change description. The number of
the registration labels is the same at each level. Depending
on the parameter label factor the values of registration la-
bels change towards the optimal ones. The source image is
deformed according to the optimal labels and it is updated
for the next level. The value 0.8 was employed for updat-
ing the registration labels. Last but not least, the maximum
displacement was set smaller than 0.4 times the distance of
two consecutive nodes in order to preserve the right dis-
placement of every node.
In addition, several methods block matching methods
can be employed. Semantic changes in multitemporal im-
agery affect the local intensities and also change the struc-
ture of the region. One of the problems in traditional change
detection techniques, is that change in intensities does not
directly mean semantic change. This was crucial since the
focus, here, was on urban and peri-urban regions and man-
made object changes. The optimal displacement using the
SADG function are calculated using the weighted sum of
the difference between the pair and the gradient inner prod-
uct. On the other hand any other similarity measure as mu-
tual information, normalized cross correlation, ratio corre-
lation can be used. In Section 4 we have tested different
similarity functions. For the SADG metric and focused on
man-made object changes, the fixed cost C was set to 100.
In particular, higher C values result to less changes. The
parameter is not so sensitive since values between 90 to 120
lead to comparable results.
Last but not least, the number of iterations per level was
set to 10, the regularization parameter for the registration
task to 35 and for the change detection one to 3.5. The
function used for the projection from pixels to nodes and
reverse was the Cubic B-splines.
4. Experimental Results and Evaluation
4.1. Dataset
The developed framework was applied to several pairs
of multispectal VHR images from different satellite sen-
sors (i.e., Quickbird and WorldView-2). The multi-temporal
dataset covers approximately a 9 km
2
region in the East
Prefecture of Attica in Greece. All datasets were acquired
between the years of 2006 and 2011. The dataset is quite
challenging both due to its size and the pictured complex-
ity derived from the different acquisition angles. For the
quantitative evaluation the ground truth was manually col-
lected and annotated after an attentive and laborious photo-
interpretation done by an expert.
4.2. Experimental Results
Extensive experiments were performed over several im-
ages pairs and based on several similarity metrics namely
the Sum of Absolute Differences (SAD), the Sum of
Square Differences (SSD), the Normalized Cross Correla-
tion (NCC), the Normalized Mutual Information (NMI), the
Correlation Ratio (CR), the Sum of Gradient Inner Products
(GRAD), the Normalized Correlation Coefficient plus Sum
of Gradient Inner Products (CCGIP), the Hellinger Distance
(HD), the Jensen-Renyi Divergence (JRD), the Mutual In-
formation (MI) and the Sum Absolute of Differences plus
Gradient Inner Products (SADG). The experimental results
were evaluated both qualitative and quantitative for the reg-
istration and the change detection tasks.
Moreover, in order to evaluate quantitatively the devel-
oped algorithm, the standard quality metrics of Complete-
ness, Correctness and Quality were calculated on the de-
tected object level. The True Positives (TP), False Nega-
tives (FN) and False Positives (FP) were calculated in all
cases.
Completeness =
T P
T P + F N
(7)
Correctness =
T P
T P + F P
(8)
Quality =
T P
T P + F P + F N
(9)
where TP is the number of correctly detected changes, FN
is the number of changes that haven’t been detected by the
algorithm and FP is the number of false alarms.
Regarding the evaluation of the registration, a number
of ground control points (GCPs) were manually collected
in both unregistered and registered data. In particular, the
GCPs contained several points on building roof tops which
usually have the largest displacements. The displacement

Citations
More filters
Journal ArticleDOI
03 Feb 2018-Sensors
TL;DR: A new approach for change detection in 3D point clouds that combines classification and CD in one step using machine learning is suggested.
Abstract: This paper suggests a new approach for change detection (CD) in 3D point clouds. It combines classification and CD in one step using machine learning. The point cloud data of both epochs are merged for computing features of four types: features describing the point distribution, a feature relating to relative terrain elevation, features specific for the multi-target capability of laser scanning, and features combining the point clouds of both epochs to identify the change. All these features are merged in the points and then training samples are acquired to create the model for supervised classification, which is then applied to the whole study area. The final results reach an overall accuracy of over 90% for both epochs of eight classes: lost tree, new tree, lost building, new building, changed ground, unchanged building, unchanged tree, and unchanged ground.

63 citations

Journal ArticleDOI
TL;DR: In this paper , very high-resolution (VHR) remote sensing images can capture details of a ground object and offer an opportunity to detect land cover changes in detail, which is an important application of Earth observation data because it provides insights into environmental health, global warming, and city management.
Abstract: Land cover change detection (LCCD) with remote sensing images is an important application of Earth observation data because it provides insights into environmental health, global warming, and city management. In particular, very-high-resolution (VHR) remote sensing images can capture details of a ground object and offer an opportunity to detect land cover changes in detail.

49 citations

Book ChapterDOI
14 Dec 2015
TL;DR: This paper tackles the problem of man-made object detection from hyperspectral data through a deep learning classification framework using a Convolutional Neural Network to encode pixels’ spectral and spatial information and employ a Multi-Layer Perceptron to conduct the classification task.
Abstract: Hyperspectral sensing, due to its intrinsic ability to capture the spectral responses of depicted materials, provides unique capabilities towards object detection and identification. In this paper, we tackle the problem of man-made object detection from hyperspectral data through a deep learning classification framework. By the effective exploitation of a Convolutional Neural Network we encode pixels’ spectral and spatial information and employ a Multi-Layer Perceptron to conduct the classification task. Experimental results and the performed quantitative validation on widely used hyperspectral datasets demonstrating the great potentials of the developed approach towards accurate and automated man-made object detection.

45 citations


Cites background from "Simultaneous registration and chang..."

  • ...Most research and development efforts do consider a classification approach at the core of their man-made detection framework [3], while the detection is based mainly on satellite multispectral datasets [8, 9]....

    [...]

Journal ArticleDOI
TL;DR: A UNet-like architecture (LUNet) which models the temporal relationship of spatial feature representations using integrated fully convolutional LSTM blocks on top of every encoding level without the need to downsample or flatten them, forming an end-to-end trainable framework.
Abstract: In this article, we present a deep multitask learning framework able to couple semantic segmentation and change detection using fully convolutional long short-term memory (LSTM) networks. In particular, we present a UNet-like architecture (L-UNet) that models the temporal relationship of spatial feature representations using integrated fully convolutional LSTM blocks on top of every encoding level. In this way, the network is able to capture the temporal relationship of spatial feature vectors in all encoding levels without the need to downsample or flatten them, forming an end-to-end trainable framework. Moreover, we further enrich the L-UNet architecture with an additional decoding branch that performs semantic segmentation on the available semantic categories that are presented in the different input dates, forming a multitask framework. Different loss quantities are also defined and combined together in a circular way to boost the overall performance. The developed methodology has been evaluated on three different data sets, i.e., the challenging bitemporal high-resolution Office National d’Etudes et de Recherches Aerospatiales (ONERA) Satellite Change Detection (OSCD) Sentinel-2 data set, the very high-resolution (VHR) multitemporal data set of the East Prefecture of Attica, Greece, and finally, the multitemporal VHR SpaceNet7 data set. Promising quantitative and qualitative results demonstrated that the synergy among the tasks can boost up the achieved performances. In particular, the proposed multitask framework contributed to a significant decrease in false-positive detections, with the F1 rate outperforming other state-of-the-art methods by at least 2.1% and 4.9% in the Attica VHR and SpaceNet7 data set cases, respectively. Our models and code can be found at https://github.com/mpapadomanolaki/multi-task-L-UNet .

45 citations

Journal ArticleDOI
TL;DR: Tests have shown that the procedure provides results comparable to the traditional GCP-based strategy and both the manual and automatic selection of the anchor images can provide reliable results.
Abstract: During the past years, UAVs (Unmanned Aerial Vehicles) became very popular as low-cost image acquisition platforms since they allow for high resolution and repetitive flights in a flexible way. One application is to monitor dynamic scenes. However, the fully automatic co-registration of the acquired multi-temporal data still remains an open issue. Most UAVs are not able to provide accurate direct image georeferencing and the co-registration process is mostly performed with the manual introduction of ground control points (GCPs), which is time consuming, costly and sometimes not possible at all. A new technique to automate the co-registration of multi-temporal high resolution image blocks without the use of GCPs is investigated in this paper. The image orientation is initially performed on a reference epoch and the registration of the following datasets is achieved including some anchor images from the reference data. The interior and exterior orientation parameters of the anchor images are then fixed in order to constrain the Bundle Block Adjustment of the slave epoch to be aligned with the reference one. The study involved the use of two different datasets acquired over a construction site and a post-earthquake damaged area. Different tests have been performed to assess the registration procedure using both a manual and an automatic approach for the selection of anchor images. The tests have shown that the procedure provides results comparable to the traditional GCP-based strategy and both the manual and automatic selection of the anchor images can provide reliable results.

43 citations


Cites methods from "Simultaneous registration and chang..."

  • ...The registration was also investigated providing some solutions to simultaneously register and detect changes in remote sensing data with graphical models [33]....

    [...]

References
More filters
Journal ArticleDOI
Masroor Hussain1, Dongmei Chen1, Angela Cheng1, Hui Wei, David Stanley 
TL;DR: This paper begins with a discussion of the traditionally pixel-based and (mostly) statistics-oriented change detection techniques which focus mainly on the spectral values and mostly ignore the spatial context, followed by a review of object-basedchange detection techniques.
Abstract: The appetite for up-to-date information about earth’s surface is ever increasing, as such information provides a base for a large number of applications, including local, regional and global resources monitoring, land-cover and land-use change monitoring, and environmental studies. The data from remote sensing satellites provide opportunities to acquire information about land at varying resolutions and has been widely used for change detection studies. A large number of change detection methodologies and techniques, utilizing remotely sensed data, have been developed, and newer techniques are still emerging. This paper begins with a discussion of the traditionally pixel-based and (mostly) statistics-oriented change detection techniques which focus mainly on the spectral values and mostly ignore the spatial context. This is succeeded by a review of object-based change detection techniques. Finally there is a brief discussion of spatial data mining techniques in image processing and change detection from remote sensing data. The merits and issues of different techniques are compared. The importance of the exponential increase in the image data volume and multiple sensors and associated challenges on the development of change detection techniques are highlighted. With the wide use of very-high-resolution (VHR) remotely sensed images, object-based methods and data mining techniques may have more potential in change detection.

1,159 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...of man-made objects is still an emerging challenge due to the significant importance for various engineering and environmental applications [18, 5, 10, 25, 3]....

    [...]

Journal ArticleDOI
TL;DR: New extensions to the previously published multivariate alteration detection (MAD) method for change detection in bi-temporal, multi- and hypervariate data such as remote sensing imagery and three regularization schemes are described.
Abstract: This paper describes new extensions to the previously published multivariate alteration detection (MAD) method for change detection in bi-temporal, multi- and hypervariate data such as remote sensing imagery. Much like boosting methods often applied in data mining work, the iteratively reweighted (IR) MAD method in a series of iterations places increasing focus on "difficult" observations, here observations whose change status over time is uncertain. The MAD method is based on the established technique of canonical correlation analysis: for the multivariate data acquired at two points in time and covering the same geographical region, we calculate the canonical variates and subtract them from each other. These orthogonal differences contain maximum information on joint change in all variables (spectral bands). The change detected in this fashion is invariant to separate linear (affine) transformations in the originally measured variables at the two points in time, such as 1) changes in gain and offset in the measuring device used to acquire the data, 2) data normalization or calibration schemes that are linear (affine) in the gray values of the original variables, or 3) orthogonal or other affine transformations, such as principal component (PC) or maximum autocorrelation factor (MAF) transformations. The IR-MAD method first calculates ordinary canonical and original MAD variates. In the following iterations we apply different weights to the observations, large weights being assigned to observations that show little change, i.e., for which the sum of squared, standardized MAD variates is small, and small weights being assigned to observations for which the sum is large. Like the original MAD method, the iterative extension is invariant to linear (affine) transformations of the original variables. To stabilize solutions to the (IR-)MAD problem, some form of regularization may be needed. This is especially useful for work on hyperspectral data. This paper describes ordinary two-set canonical correlation analysis, the MAD transformation, the iterative extension, and three regularization schemes. A simple case with real Landsat Thematic Mapper (TM) data at one point in time and (partly) constructed data at the other point in time that demonstrates the superiority of the iterative scheme over the original MAD method is shown. Also, examples with SPOT High Resolution Visible data from an agricultural region in Kenya, and hyperspectral airborne HyMap data from a small rural area in southeastern Germany are given. The latter case demonstrates the need for regularization

595 citations


Additional excerpts

  • ...Results have been also compared with the unsupervised IRMAD [21] change detection algorithm....

    [...]

  • ...Method Complet % Corr % Quality % IRMAD 67.2 36.8 30.1 SADG 92.2 80.1 74.4 SAD 95.2 64.9 60.01 SSD 94.1 67.3 61.4 NCC 77.7 40.5 34.8 NMI 55.3 62.8 50.1 CR 60.5 30.3 25.2 GRAD 35.1 49.3 23.1 CCGIP 77.8 40.4 34.9 JRD 39.6 50.1 30.4 HD 83.6 60.1 57.8 MI 41.9 52.8 30.1 Table 2....

    [...]

BookDOI
23 Oct 2009
TL;DR: This paper presents a meta-modelling architecture for semi-supervised image classification of hyperspectral remote sensing data using a SVM and a proposed circular validation strategy for land-cover maps updating.
Abstract: About the editors. List of authors. Preface. Acknowledgments. List of symbols. List of abbreviations. I Introduction. 1 Machine learning techniques in remote sensing data analysis (Bjorn Waske, Mathieu Fauvel, Jon Atli Benediktsson and Jocelyn Chanussot). 1.1 Introduction. 1.2 Supervised classification: algorithms and applications. 1.3 Conclusion. Acknowledgments. References. 2 An introduction to kernel learning algorithms (Peter V. Gehler and Bernhard Scholkopf). 2.1 Introduction. 2.2 Kernels. 2.3 The representer theorem. 2.4 Learning with kernels. 2.5 Conclusion. References. II Supervised image classification. 3 The Support Vector Machine (SVM) algorithm for supervised classification of hyperspectral remote sensing data (J. Anthony Gualtieri). 3.1 Introduction. 3.2 Aspects of hyperspectral data and its acquisition. 3.3 Hyperspectral remote sensing and supervised classification. 3.4 Mathematical foundations of supervised classification. 3.5 From structural risk minimization to a support vector machine algorithm. 3.6 Benchmark hyperspectral data sets. 3.7 Results. 3.8 Using spatial coherence. 3.9 Why do SVMs perform better than other methods? 3.10 Conclusions. References. 4 On training and evaluation of SVM for remote sensing applications (Giles M. Foody). 4.1 Introduction. 4.2 Classification for thematic mapping. 4.3 Overview of classification by a SVM. 4.4 Training stage. 4.5 Testing stage. 4.6 Conclusion. Acknowledgments. References. 5 Kernel Fisher's Discriminant with heterogeneous kernels (M. Murat Dundar and Glenn Fung). 5.1 Introduction. 5.2 Linear Fisher's Discriminant. 5.3 Kernel Fisher Discriminant. 5.4 Kernel Fisher's Discriminant with heterogeneous kernels. 5.5 Automatic kernel selection KFD algorithm. 5.6 Numerical results. 5.7 Conclusion. References. 6 Multi-temporal image classification with kernels (Jordi Munoz-Mari, Luis Gomez-Choa, Manel Martinez-Ramon, Jose Luis Rojo-Alvarez, Javier Calpe-Maravilla and Gustavo Camps-Valls). 6.1 Introduction. 6.2 Multi-temporal classification and change detection with kernels. 6.3 Contextual and multi-source data fusion with kernels. 6.4 Multi-temporal/-source urban monitoring. 6.5 Conclusions. Acknowledgments. References. 7 Target detection with kernels (Nasser M. Nasrabadi). 7.1 Introduction. 7.2 Kernel learning theory. 7.3 Linear subspace-based anomaly detectors and their kernel versions. 7.4 Results. 7.5 Conclusion. References. 8 One-class SVMs for hyperspectral anomaly detection (Amit Banerjee, Philippe Burlina and Chris Diehl). 8.1 Introduction. 8.2 Deriving the SVDD. 8.3 SVDD function optimization. 8.4 SVDD algorithms for hyperspectral anomaly detection. 8.5 Experimental results. 8.6 Conclusions. References. III Semi-supervised image classification. 9 A domain adaptation SVM and a circular validation strategy for land-cover maps updating (Mattia Marconcini and Lorenzo Bruzzone). 9.1 Introduction. 9.2 Literature survey. 9.3 Proposed domain adaptation SVM. 9.4 Proposed circular validation strategy. 9.5 Experimental results. 9.6 Discussions and conclusion. References. 10 Mean kernels for semi-supervised remote sensing image classification (Luis Gomez-Chova, Javier Calpe-Maravilla, Lorenzo Bruzzone and Gustavo Camps-Valls). 10.1 Introduction. 10.2 Semi-supervised classification with mean kernels. 10.3 Experimental results. 10.4 Conclusions. Acknowledgments. References. IV Function approximation and regression. 11 Kernel methods for unmixing hyperspectral imagery (Joshua Broadwater, Amit Banerjee and Philippe Burlina). 11.1 Introduction. 11.2 Mixing models. 11.3 Proposed kernel unmixing algorithm. 11.4 Experimental results of the kernel unmixing algorithm. 11.5 Development of physics-based kernels for unmixing. 11.6 Physics-based kernel results. 11.7 Summary. References. 12 Kernel-based quantitative remote sensing inversion (Yanfei Wang, Changchun Yang and Xiaowen Li). 12.1 Introduction. 12.2 Typical kernel-based remote sensing inverse problems. 12.3 Well-posedness and ill-posedness. 12.4 Regularization. 12.5 Optimization techniques. 12.6 Kernel-based BRDF model inversion. 12.7 Aerosol particle size distribution function retrieval. 12.8 Conclusion. Acknowledgments. References. 13 Land and sea surface temperature estimation by support vector regression (Gabriele Moser and Sebastiano B. Serpico). 13.1 Introduction. 13.2 Previous work. 13.3 Methodology. 13.4 Experimental results. 13.5 Conclusions. Acknowledgments. References. V Kernel-based feature extraction. 14 Kernel multivariate analysis in remote sensing feature extraction (Jeronimo Arenas-Garcia and Kaare Brandt Petersen). 14.1 Introduction. 14.2 Multivariate analysis methods. 14.3 Kernel multivariate analysis. 14.4 Sparse Kernel OPLS. 14.5 Experiments: pixel-based hyperspectral image classification. 14.6 Conclusions. Acknowledgments. References. 15 KPCA algorithm for hyperspectral target/anomaly detection (Yanfeng Gu). 15.1 Introduction. 15.2 Motivation. 15.3 Kernel-based feature extraction in hyperspectral images. 15.4 Kernel-based target detection in hyperspectral images. 15.5 Kernel-based anomaly detection in hyperspectral images. 15.6 Conclusions. Acknowledgments References. 16 Remote sensing data Classification with kernel nonparametric feature extractions (Bor-Chen Kuo, Jinn-Min Yang and Cheng-Hsuan Li). 16.1 Introduction. 16.2 Related feature extractions. 16.3 Kernel-based NWFE and FLFE. 16.4 Eigenvalue resolution with regularization. 16.5 Experiments. 16.6 Comments and conclusions. References. Index.

393 citations

Journal ArticleDOI
TL;DR: It is shown that by appropriately choosing what subproblems to use, one can design novel and very powerful MRF optimization algorithms, which are able to derive algorithms that generalize and extend state-of-the-art message-passing methods, and take full advantage of the special structure that may exist in particular MRFs.
Abstract: This paper introduces a new rigorous theoretical framework to address discrete MRF-based optimization in computer vision. Such a framework exploits the powerful technique of Dual Decomposition. It is based on a projected subgradient scheme that attempts to solve an MRF optimization problem by first decomposing it into a set of appropriately chosen subproblems, and then combining their solutions in a principled way. In order to determine the limits of this method, we analyze the conditions that these subproblems have to satisfy and demonstrate the extreme generality and flexibility of such an approach. We thus show that by appropriately choosing what subproblems to use, one can design novel and very powerful MRF optimization algorithms. For instance, in this manner we are able to derive algorithms that: 1) generalize and extend state-of-the-art message-passing methods, 2) optimize very tight LP-relaxations to MRF optimization, and 3) take full advantage of the special structure that may exist in particular MRFs, allowing the use of efficient inference techniques such as, e.g., graph-cut-based methods. Theoretical analysis on the bounds related with the different algorithms derived from our framework and experimental results/comparisons using synthetic and real data for a variety of tasks in computer vision demonstrate the extreme potentials of our approach.

353 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...The first category is related to the linear programming relaxation [14]....

    [...]

Book
20 Mar 2019
TL;DR: In this article, images, arrays, and vectors are used for image registration and classification, and transformations are used to represent the transformation of images, images, and arrays of vectors.
Abstract: Images, Arrays, and Vectors. Image Statistics. Transformations. Radiometric Enhancement. Topographic Modeling. Image Registration. Image Sharpening. Change Detection. Unsupervised Classification. Supervised Classification. Hyperspectral Analysis.

285 citations


"Simultaneous registration and chang..." refers background in this paper

  • ...In addition, the primary goal of the analysis of multitemporal datasets is the detection of changes between different land cover types [3, 11]....

    [...]

  • ...of man-made objects is still an emerging challenge due to the significant importance for various engineering and environmental applications [18, 5, 10, 25, 3]....

    [...]

Frequently Asked Questions (2)
Q1. What have the authors contributed in "Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data" ?

To this end, in this paper the authors propose a modular, scalable, metric free single shot change detection/registration method. Promising results on large scale experiments demonstrate the extreme potentials of their method. 

The integration of prior knowledge regarding texture and geometric features is currently under development and a gpu implementation is among the future perspectives as well.