Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data
Summary (3 min read)
1. Introduction
- The current generation of space-borne and airborne sensors are generating nearly continuous streams of massive multi-temporal high resolution remote sensing data.
- Most remote sensing and GIS software still employ semi-automated registration procedures when it comes to very large, multispectral, very high resolution satellite data [17, 7].
- Among the various recently proposed methods, those based on Markov Random Fields [8, 24, 1], kernels [2, 26] and neural networks [22, 23] have gained important attention.
- Focusing on man-made object change detection [4, 20] in urban and peri-urban regions, several approaches have been proposed based on very high resolution optical and radar data [19, 23, 6, 20].
2.1. MRF formulation
- The authors have designed and built an MRF model over two different graphs of the same dimensions.
- The interaction between the two graphs is performed by the similarity cost which connect the registration with the change detection terms.
- Each graph is superimposed on the image [9] and therefore every node of the graph acts and depends on a subset of pixels in the vicinity (depending on the chosen interpolation strategy).
- In particular, the dimensions of the graph are related to the image dimensions forming a trade off between accuracy and computational complexity.
- Ech and the authors couple the two different graphs to one.
2.2. The Registration Energy Term
- The goal of image registration is to define a transformation map T which will project the source image to the target image.
- The energy formulation for the registration comprises of a similarity cost (that seeks to satisfy the equation 2) and a smoothness penalty on the deformation domain.
- The similarity cost depends on the presence of changes and will be subsequently defined.
- The smoothness term penalises neighbouring nodes that have different displacement labels, depending on the distance of the labelled displacements.
2.3. The Change Detection Energy Term
- The goal of the change detection term is to estimate the changed and unchanged image regions.
- The authors employ two different labels in order to address the change detection problem lcp ∈ [0, 1].
- The energy formulation for the change detection corresponds to a smoothness term which penalizes neighbouring nodes with different change labels.
2.4. Coupling the Energy Terms
- The coupling between change detection and registration is achieved through the interconnection between the two graphs.
- These two terms are integrated as in (equation 5) which simply uses a fixed cost in the presence of changes and the image matching cost in their absence.
- With a slight abuse of notation the authors consider a node with an index p ∈ G (they recall that the two graphs are identical) corresponding to the same node throughout the two graphs (Greg, Gch).
- In such a setting, optimizing an objective function seeking similarity correspondences is not meaningful and deformation vectors should be the outcome of the smoothness constraint on the displacement space.
- Let us consider that this value is known and that it is independent from the image displacements, so the authors can distinguish the regions that have been changed.
2.5. Optimization
- There are several techniques for the minimization of an MRF model which can be generally summarised into those based on the message passing and those on graph cut methods.
- The first category is related to the linear programming relaxation [14].
- The optimization of the implementation is performed by FastPD which is based on the dual theorem of linear programming [15, 16].
3. Implementation
- Concerning the image, iteratively different levels of Gaussian image pyramids are used.
- In all their experiments, 2 image and 3 grid levels were found adequate for the very high resolution satellite data.
- Regarding the label space, a search for possible displacements along 8 directions (x, y and diagonal axes) is performed, while the change labels are always two and correspond to change or no change description.
- Depending on the parameter label factor the values of registration labels change towards the optimal ones.
- One of the problems in traditional change detection techniques, is that change in intensities does not directly mean semantic change.
4.1. Dataset
- The developed framework was applied to several pairs of multispectal VHR images from different satellite sensors (i.e., Quickbird and WorldView-2).
- The multi-temporal dataset covers approximately a 9 km2 region in the East Prefecture of Attica in Greece.
- The dataset is quite challenging both due to its size and the pictured complexity derived from the different acquisition angles.
- For the quantitative evaluation the ground truth was manually collected and annotated after an attentive and laborious photointerpretation done by an expert.
4.2. Experimental Results
- Regarding the evaluation for the man-made change detection task, experimental results after the application of the developed method are shown in Figure 3 and Figure 4.
- In particular, in Figure 3 the detected changes are shown with a red color while the ground truth polygons are shown with green.
- The behaviour of the developed method can be further observed in Figure 5, where certain examples with True Positives, False Negatives and False Positives cases are presented.
- The selected metric affects, also, the computational time significantly.
5. Conclusions
- Developed and validated a novel framework which address concurrently the registration and change detection tasks in very high resolution multispectral multitemporal optical satellite data.the authors.
- The developed method is modular, scalable and metric free.
- The formulation exploits a decomposed interconnected graphical model formulation where registration similarity constraints are relaxed in the presence of change detection.
- The framework was optimized for the detection of changes related to man-made objects in urban and peri-urban environments.
Did you find this useful? Give us your feedback
Citations
38 citations
Cites methods from "Simultaneous registration and chang..."
...In this work, we propose a novel mid-level representation that assists in performing domain adaptation by extracting a domain-invariant feature for every image region (a super-pixel, Figure 1b) in each image in terms of the spatial distribution of its spectral neighbors (SDSN)....
[...]
27 citations
Cites background from "Simultaneous registration and chang..."
...Moreover, we extend the recently proposed change detection framework [30] by providing information about the type of detected from-to change trajectories....
[...]
19 citations
Cites background or methods from "Simultaneous registration and chang..."
...In particular, we extended the formulation of [6, 7] by adding another graph which is related to the segmentation problem....
[...]
...Following the notations of [6, 7], here, we add another graph Gseg associated with the image segmentation problem....
[...]
...Implementation details: Regarding the employed parameters, in a similar way as in [6] we used 3 image and 4 grid levels....
[...]
...Both the registration and change detection terms are following the same formulations as in [6, 7], while the goal of segmentation is to assign the correct segmentation label to each node of the target image....
[...]
18 citations
Cites methods from "Simultaneous registration and chang..."
...So, the best classical approaches are methods based on anomaly detection framework on time series of multispectral low-resolution satellite images and so called spectral indices [34], methods based on Markov Random Fields and global optimization on graphs [35, 36, 37], approaches using object-based segmentation with post-classification of changes [38, 39, 40] and methods based on Multivariate Alteration Detection [41, 30]....
[...]
17 citations
Cites methods from "Simultaneous registration and chang..."
...egory being then subdivided in standard techniques, CNNs and Fully-Convolutional Neural Networks. Unsupervised methods have been used for change detection in many dierent ways (Hussain et al., 2013; Vakalopoulou et al., 2015; Liu et al., 2019). In the context of change detection, annotated datasets are extremely scarce and often kept private. Thus, unsupervised methods are extremely useful, since, unlike supervised metho...
[...]
References
1,159 citations
"Simultaneous registration and chang..." refers background in this paper
...of man-made objects is still an emerging challenge due to the significant importance for various engineering and environmental applications [18, 5, 10, 25, 3]....
[...]
595 citations
Additional excerpts
...Results have been also compared with the unsupervised IRMAD [21] change detection algorithm....
[...]
...Method Complet % Corr % Quality % IRMAD 67.2 36.8 30.1 SADG 92.2 80.1 74.4 SAD 95.2 64.9 60.01 SSD 94.1 67.3 61.4 NCC 77.7 40.5 34.8 NMI 55.3 62.8 50.1 CR 60.5 30.3 25.2 GRAD 35.1 49.3 23.1 CCGIP 77.8 40.4 34.9 JRD 39.6 50.1 30.4 HD 83.6 60.1 57.8 MI 41.9 52.8 30.1 Table 2....
[...]
393 citations
353 citations
"Simultaneous registration and chang..." refers background in this paper
...The first category is related to the linear programming relaxation [14]....
[...]
285 citations
"Simultaneous registration and chang..." refers background in this paper
...In addition, the primary goal of the analysis of multitemporal datasets is the detection of changes between different land cover types [3, 11]....
[...]
...of man-made objects is still an emerging challenge due to the significant importance for various engineering and environmental applications [18, 5, 10, 25, 3]....
[...]
Related Papers (5)
Frequently Asked Questions (2)
Q2. What are the future works in "Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data" ?
The integration of prior knowledge regarding texture and geometric features is currently under development and a gpu implementation is among the future perspectives as well.