scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Automated architectural reconstruction using reference planes under convex optimization

01 Jun 2016-International Journal of Control Automation and Systems (Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers)-Vol. 14, Iss: 3, pp 814-826
TL;DR: A method for the automated reconstruction of architectures from two views of a monocular camera using reference planes to estimate image homography instead of using the conventional camera pose estimation method and the texture of faces is mapped from 2D images to a 3D plane.
Abstract: In this paper, a method for the automated reconstruction of architectures from two views of a monocular camera is proposed. While this research topic has been studied over the last few decades, we contend that a satisfactory approach has not yet been devised. Here, a new method to solve the same problem with several points of novelty is proposed. First, reference planes are automatically detected using color, straight lines, and edge/vanishing points. This approach is quite robust and fast even when different views and complicated conditions are presented. Second, the camera pose and 3D points are accurately estimated by a two-view geometry constraint in the convex optimization approach. It has been demonstrated that camera rotations are appropriately estimated, while translations induce a significant error in short baseline images. To overcome this problem, we rely only on reference planes to estimate image homography instead of using the conventional camera pose estimation method. Thus, the problem associated with short baseline images is adequately addressed. The 3D points and translation are then simultaneously triangulated. Furthermore, both the homography and 3D point triangulation are computed via the convex optimization approach. The error of back-projection and measured points is minimized in L ∞-norm so as to overcome the local minima problem of the canonical L 2-norm method. Consequently, extremely accurate homography and point clouds can be achieved with this scheme. In addition, a robust plane fitting method is introduced to describe a scene. The corners are considered as properties of the plane in order to limit the boundary. Thus, it is necessary to find the exact corresponding corner positions by searching along the epipolar line in the second view. Finally, the texture of faces is mapped from 2D images to a 3D plane. The simulation results demonstrate the effectiveness of the proposed method for scenic images in an outdoor environment.
Citations
More filters
Journal ArticleDOI
TL;DR: A vision-based non-contact gap andflush measurement system that can deal with complex surface in noisy industrial environment and achieve higher specifications compared with current gap and flush measurement sensors is developed.
Abstract: The accurate fitting of various parts inspected by measuring the width of the gap between two adjacent panels and the alignment of the two surfaces, known as flushness, is an important task in assembling vehicles. The optimal solution requires high accuracy and fast measurement. Toward this end, we develop a vision-based non-contact gap and flush measurement. The vision system consists of a high-resolution camera and a multi-line laser generator. The proposed gap and flush measurement sensor projects laser lines onto the panels that are observed by the high-resolution camera. The measurement is initiated when the operator brings the device closer to the surface until it is within operating range. During the process, the line features are digitized by using proposed approach, the desired calculations are made, the non-conforming images are discarded, and the remaining images are used to perform the gap and flush measurement. The measurement system can deal with complex surface in noisy industrial environment and achieve higher specifications compared with current gap and flush measurement sensors. The usefulness of the proposed system has been demonstrated using real tests with accurate know-size patterns and a real inline vehicle assembly system in Korea.

22 citations

Journal ArticleDOI
TL;DR: A multi-directional scanning strategy is proposed where the AUV determines the direction of the next scan by analyzing the 3-D data of the object until the scanning is complete, which enables adaptive scanning based on the shape of the target object while reducing the amount of scanning time.
Abstract: In this study, we propose an autonomous underwater vehicle (AUV)-based multi-directional scanning method of underwater objects using forward scan sonar (FSS). Recently, a method was developed that can generate a 3-D point cloud of an underwater object using FSS. However, the data comprised sparse and noisy characteristics that made it difficult for 3-D recognition. Another limitation was the absence of back and side surface information of an object. These limitations degraded the results of the 3-D reconstruction. We propose a multi-directional scanning strategy to improve the 3-D point cloud and reconstruction results where the AUV determines the direction of the next scan by analyzing the 3-D data of the object until the scanning is complete. This enables adaptive scanning based on the shape of the target object while reducing the amount of scanning time. Based on the scanning strategy, a polygonal approximation method for real-time 3-D reconstruction is developed to process scanned data groups of the 3-D point cloud. This process can efficiently handle multiple 3-D point cloud data for real-time operation and reduce its uncertainty. To verify the performance of our proposed method, simulations were performed with various objects and conditions. In addition, experiments were conducted in an indoor water tank, and the results were compared with the simulation results. Field experiments were conducted to verify the proposed method for more diverse environments and objects.

17 citations

Book ChapterDOI
23 Mar 2020
TL;DR: Different from most existing image matching methods, the proposed method extracts image features using deep learning approach directly estimates locations of features between pairwise constraint of images by maximizing an image- patch similarity metric between images.
Abstract: Image stitching is an important task in image processing and computer vision. Image stitching is the process of combining multiple photographic images with overlapping fields of view to produce a segmented panorama, resolution image. It is widely used in object reconstruction, panoramic creating. In this paper, we present an approach based on deep learning for image stitching, which are applied to generate high resolution panoramic image supporting for virtual tour interaction. Different from most existing image matching methods, the proposed method extracts image features using deep learning approach. Our approach directly estimates locations of features between pairwise constraint of images by maximizing an image- patch similarity metric between images. A large dataset high resolution images and videos from natural tourism scenes were collected for training and evaluation. Experimental results illustrated that the deep feature approach outperforms.

17 citations

Journal ArticleDOI
TL;DR: A method for estimating the vision-based 3-D motion of a vehicle with several parts that requires only two corresponding points of consecutive images to estimate the vehicle motion and applies the bundle adjustment-based quasiconvex optimization.
Abstract: Recently, there have been several studies on vision-based motion estimation under a supposition that planar motion follows a nonholonomic constraint This allows reducing computational time However, the vehicle motion in an outdoor environment does not accept this assumption This paper presents a method for estimating the vision-based 3-D motion of a vehicle with several parts as follows First, the Ackermann steering model is applied to reduce constraint parameters of the 3-D motion In difference to the previous contribution, the proposed approach requires only two corresponding points of consecutive images to estimate the vehicle motion Second, motion parameters are extracted based on a closed-form solution on geometric constraints Third, the estimation approach applies the bundle adjustment-based quasiconvex optimization This task aims to take into account advantage of omnidirectional vision-based features for reducing errors The omnidirectional vision supports for landmarks tracking in long travel and large rotation, which is appropriate for a bundle adjustment technique Evaluated results show that the proposed method is applicable in the practical condition of outdoor environments

15 citations

Journal ArticleDOI
TL;DR: In this method, the outliers are rejected based on the differing characteristics of algebraic errors between outliers and inliers, and the homography is estimated by minimising the residual vector through integrating the outlier rejection into the estimation pipeline.
Abstract: To solve the homography estimation problem containing outliers and noise, a fast, robust, and accurate method is proposed. In this method, the outliers are rejected based on the differing characteristics of algebraic errors between outliers and inliers, and the homography is estimated by minimising the residual vector. The advantage of this method is in integrating the outlier rejection into the estimation pipeline. The computational complexity of the proposed method is not increased, and the random sample consensus algorithm is not needed to extract the inliers, as was previously necessary. Since the outlier rejection process is based on an algebraic criterion without computing the re-projection error at each step, the speed of the proposed method is improved. Several simulations based on synthetic and real images illustrate the performance of the proposed method in terms of subjective visual quality, objective quality measurement, and computational time. The experimental results demonstrate that the proposed method achieves accurate, efficient and robust homography estimation under different image transformation degrees and different outlier ratios.

9 citations

References
More filters
Journal ArticleDOI
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Abstract: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

46,906 citations

Book
01 Mar 2004
TL;DR: In this article, the focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them, and a comprehensive introduction to the subject is given. But the focus of this book is not on the optimization problem itself, but on the problem of finding the appropriate technique to solve it.
Abstract: Convex optimization problems arise frequently in many different fields. A comprehensive introduction to the subject, this book shows in detail how such problems can be solved numerically with great efficiency. The focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. The text contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance, and economics.

33,341 citations

Journal ArticleDOI
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing

23,396 citations

Book
01 Jan 2000
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Abstract: From the Publisher: A basic problem in computer vision is to understand the structure of a real world scene given several images of it. Recent major developments in the theory and practice of scene reconstruction are described in detail in a unified framework. The book covers the geometric principles and how to represent objects algebraically so they can be computed and applied. The authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly.

15,558 citations

Proceedings ArticleDOI
01 Jan 1988
TL;DR: The problem the authors are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work.
Abstract: The problem we are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work. For example, we desire to obtain an understanding of natural scenes, containing roads, buildings, trees, bushes, etc., as typified by the two frames from a sequence illustrated in Figure 1. The solution to this problem that we are pursuing is to use a computer vision system based upon motion analysis of a monocular image sequence from a mobile camera. By extraction and tracking of image features, representations of the 3D analogues of these features can be constructed.

13,993 citations