Robust wide-baseline stereo from maximally stable extremal regions

TLDR: The high utility of MSERs, multiple measurement regions and the robust metric is demonstrated in wide-baseline experiments on image pairs from both indoor and outdoor scenes.

About: This article is published in Image and Vision Computing.The article was published on 2004-09-01 and is currently open access. It has received 3422 citations till now. The article focuses on the topics: Maximally stable extremal regions & Epipolar geometry.

Figures

Table 1: Definitions used in Section 2

Figure 1: BOOKSHELF: Estimated epipolar geometry on indoor scene with significant scale change. In the cutouts the change in the resolution of detected DRs is clearly visible.

Table 2: Number of DRs detected in images. The number of tentative correspondences is given in the TC column.

Figure 2: VALBONNE: Estimated epipolar geometry and points associated to the matched regions are shown in the first row. Cutouts in the second row show matched bricks.

Figure 5: CYLINDRICAL BOX: Epipolar geometry (top) and matched regions (bottom left). Fully affine distortion, a non-planar object, textured surface and a strong specular reflection are present in the scene. SHOUT (bottom right), a scene with a change of illumination spectral power distribution.

Figure 3: WASH: Epipolar geometry and dense matched regions with fully affine distortion.

Frequently asked questions

Q1. What is the definition of a MSER?

The MSERs are sets of image elements, closed under the affine transformation of image coordinates and invariant to affine transformation of intensity. 

Q2. What have the authors contributed in "Robust wide baseline stereo from maximally stable extremal regions" ?

The wide-baseline stereo problem, i. e. the problem of establishing correspondences between a pair of images taken from different viewpoints is studied. A new set of image elements that are put into correspondence, the so called extremal regions, is introduced. Extremal regions possess highly desirable properties: the set is closed under 1. continuous ( and thus projective ) transformation of image coordinates and 2. monotonic transformation of image intensities. An efficient ( near linear complexity ) and practically fast detection algorithm ( near frame rate ) is presented for an affinely-invariant stable subset of extremal regions, the maximally stable extremal regions ( MSER ). 

Q3. What are the future works mentioned in the paper "Robust wide baseline stereo from maximally stable extremal regions" ?

In future work, the authors intend to proceed towards fully automatic projective reconstruction of the 3D scene, which requires computing projective reconstruction and dense matching. 

Q4. What is the final step of all wide-baseline algorithms?

Finding epipolar geometry consistent with the largest number of tentative (local) correspondences is the final step of all wide-baseline algorithms. 

Q5. What are the main novelties of the paper?

The three main novelties are: the introduction of MSERs, robust matching of local features and the use of multiple scaled measurement regions. 

Q6. What is the common method of establishing tentative correspondences?

distinguished regions or their scaled version serve as measurement regions and tentative correspondences are established by comparing invariants using Mahalanobis distance [10, 16, 11]. 

Q7. How did the MSER detector perform on the epipolar scene?

In future work, the authors intend to proceed towards fully automatic projective reconstruction of the 3D scene, which requires computing projective reconstruction and dense matching. 

Q8. What are the main design decisions at this stage?

Important design decisions at this stage include: 1. the choice of measurement regions, i.e. the parts of the image on which invariants are computed, 2. the method of selecting tentative correspondences given the invariant description and 3. 

Q9. What is the definition of a merge of two components?

A merge of two components is viewed as termination of existence of the smaller component and an insertion of all pixels of the smaller component into the larger one. 

Q10. What is the procedure for determining the EG?

an affine transformation between pairs of potentially corresponding DRs, i.e. the DRs consistent with the rough EG, is computed. 

Q11. What is the advantage of a robust MR matching algorithm?

Since matching is accomplished in a robust manner, the authors benefit from the increase of distinctiveness of large regions without being severely affected by clutter or non-planarity of the DR’s pre-image. 

Q12. What is the important paper by Schmid and Mohr?

Since the influential paper by Schmid and Mohr [11] many image matching and wide-baseline stereo algorithms have been proposed, most commonly usingHarris interest points as distinguished regions. 

Q13. What is the definition of a good measurement?

A measurement taken from an almost planar patch of the scene with stable invariant description will be referred to as a ’good measurement’. 

Q14. What is the description of the proposed similarity measure?

The robustness of the proposed similarity measure allows us to use invariants from a collection of measurement regions, even some that are much larger than the associated distinguished region. 

Q15. Why did the authors have to consider invariants from multiple measurement regions?

Due to the robustness, the authors were able to consider invariants from multiple measurement regions, even some that were significantly larger (and hence probably discriminative) than the associated MSER. 

Q16. What is the probability of the success of the procedure?

Probabilistic analysis of the likelihood of the success of the procedure is not simple, since the distribution of invariants and their noise is image-dependent. 

Q17. What is the way to define a MSER?

Finally the authors remark that MSERs can be defined on any image (even high-dimensional) whose pixel values are from a totally ordered set. 

Q18. What is the way to find a reliable correspondence between two images?

In the wide-baseline set-up, local image deformations cannot be realistically approximated by translation or translation with rotation and a full affine model is required.