Showing papers on "Camera auto-calibration published in 2017"

SfM-Net: Learning of Structure and Motion from Video

Abstract: We propose SfM-Net, a geometry-aware neural network for motion estimation in videos that decomposes frame-to-frame pixel motion in terms of scene and object depth, camera motion and 3D object rotations and translations. Given a sequence of frames, SfM-Net predicts depth, segmentation, camera and rigid object motions, converts those into a dense frame-to-frame motion field (optical flow), differentiably warps frames in time to match pixels and back-propagates. The model can be trained with various degrees of supervision: 1) self-supervised by the re-projection photometric error (completely unsupervised), 2) supervised by ego-motion (camera motion), or 3) supervised by depth (e.g., as provided by RGBD sensors). SfM-Net extracts meaningful depth estimates and successfully estimates frame-to-frame camera rotations and translations. It often successfully segments the moving objects in the scene, even though such supervision is never provided.

A New Low-Light Image Enhancement Algorithm Using Camera Response Model

Abstract: Low-light images are not conducive to human observation and computer vision algorithms due to their low visibility. To solve this problem, many image enhancement techniques have been proposed. However, existing techniques inevitably introduce color and lightness distortion when increasing visibility. To lower the distortion, we propose a novel enhancement method using the response characteristics of cameras. First, we investigate the relationship between two images with different exposures to obtain an accurate camera response model. Then we borrow the illumination estimation techniques to estimate the exposure ratio map. Finally, we use our camera response model to adjust each pixel to its desired exposure according to the estimated exposure ratio map. Experiments show that our method can obtain enhancement results with less color and lightness distortion compared to several state-of-the-art methods.

The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring

Abstract: We propose a concept for a lens attachment that turns a standard DSLR camera and lens into a light field camera. The attachment consists of eight low-resolution, low-quality side cameras arranged around the central high-quality SLR lens. Unlike most existing light field camera architectures, this design provides a high-quality 2D image mode, while simultaneously enabling a new high-quality light field mode with a large camera baseline but little added weight, cost, or bulk compared with the base DSLR camera. From an algorithmic point of view, the high-quality light field mode is made possible by a new light field super-resolution method that first improves the spatial resolution and image quality of the side cameras and then interpolates additional views as needed. At the heart of this process is a super-resolution method that we call iterative Patch- And Depth-based Synthesis (iPADS), which combines patch-based and depth-based synthesis in a novel fashion. Experimental results obtained for both real captured data and synthetic data confirm that our method achieves substantial improvements in super-resolution for side-view images as well as the high-quality and view-coherent rendering of dense and high-resolution light fields.

A Unifying Model for Camera Calibration

Abstract: This paper proposes a unified theory for calibrating a wide variety of camera models such as pinhole, fisheye, cata-dioptric, and multi-camera networks. We model any camera as a set of image pixels and their associated camera rays in space. Every pixel measures the light traveling along a (half-) ray in 3-space, associated with that pixel. By this definition, calibration simply refers to the computation of the mapping between pixels and the associated 3D rays. Such a mapping can be computed using images of calibration grids, which are objects with known 3D geometry, taken from unknown positions. This general camera model allows to represent non-central cameras; we also consider two special subclasses, namely central and axial cameras. In a central camera, all rays intersect in a single point, whereas the rays are completely arbitrary in a non-central one. Axial cameras are an intermediate case: the camera rays intersect a single line. In this work, we show the theory for calibrating central, axial and non-central models using calibration grids, which can be either three-dimensional or planar.

High-precision method of binocular camera calibration with a distortion model.

Abstract: A high-precision camera calibration method for binocular stereo vision system based on a multi-view template and alternative bundle adjustment is presented in this paper. The proposed method could be achieved by taking several photos on a specially designed calibration template that has diverse encoded points in different orientations. In this paper, the method utilized the existing algorithm used for monocular camera calibration to obtain the initialization, which involves a camera model, including radial lens distortion and tangential distortion. We created a reference coordinate system based on the left camera coordinate to optimize the intrinsic parameters of left camera through alternative bundle adjustment to obtain optimal values. Then, optimal intrinsic parameters of the right camera can be obtained through alternative bundle adjustment when we create a reference coordinate system based on the right camera coordinate. We also used all intrinsic parameters that were acquired to optimize extrinsic parameters. Thus, the optimal lens distortion parameters and intrinsic and extrinsic parameters were obtained. Synthetic and real data were used to test the method. The simulation results demonstrate that the maximum mean absolute relative calibration errors are about 3.5e-6 and 1.2e-6 for the focal length and the principal point, respectively, under zero-mean Gaussian noise with 0.05 pixels standard deviation. The real result shows that the reprojection error of our model is about 0.045 pixels with the relative standard deviation of 1.0e-6 over the intrinsic parameters. The proposed method is convenient, cost-efficient, highly precise, and simple to carry out.

Real-time panoramic tracking for event cameras

Abstract: Event cameras are a paradigm shift in camera technology. Instead of full frames, the sensor captures a sparse set of events caused by intensity changes. Since only the changes are transferred, those cameras are able to capture quick movements of objects in the scene or of the camera itself. In this work we propose a novel method to perform camera tracking of event cameras in a panoramic setting with three degrees of freedom. We propose a direct camera tracking formulation, similar to state-of-the-art in visual odometry. We show that the minimal information needed for simultaneous tracking and mapping is the spatial position of events, without using the appearance of the imaged scene point. We verify the robustness to fast camera movements and dynamic objects in the scene on a recently proposed dataset [18] and self-recorded sequences.

3D digital image correlation using single color camera pseudo-stereo system

Abstract: Three dimensional digital image correlation (3D-DIC) has been widely used by industry to measure the 3D contour and whole-field displacement/strain. In this paper, a novel single color camera 3D-DIC setup, using a reflection-based pseudo-stereo system, is proposed. Compared to the conventional single camera pseudo-stereo system, which splits the CCD sensor into two halves to capture the stereo views, the proposed system achieves both views using the whole CCD chip and without reducing the spatial resolution. In addition, similarly to the conventional 3D-DIC system, the center of the two views stands in the center of the CCD chip, which minimizes the image distortion relative to the conventional pseudo-stereo system. The two overlapped views in the CCD are separated by the color domain, and the standard 3D-DIC algorithm can be utilized directly to perform the evaluation. The system’s principle and experimental setup are described in detail, and multiple tests are performed to validate the system.

Simultaneous calibration: a joint optimization approach for multiple Kinect and external cameras

Abstract: Camera calibration is a crucial problem in many applications, such as 3D reconstruction, structure from motion, object tracking and face alignment. Numerous methods have been proposed to solve the above problem with good performance in the last few decades. However, few methods are targeted at joint calibration of multi-sensors (more than four devices), which normally is a practical issue in the real-time systems. In this paper, we propose a novel method and a corresponding workflow framework to simultaneously calibrate relative poses of a Kinect and three external cameras. By optimizing the final cost function and adding corresponding weights to the external cameras in different locations, an effective joint calibration of multiple devices is constructed. Furthermore, the method is tested in a practical platform, and experiment results show that the proposed joint calibration method can achieve a satisfactory performance in a project real-time system and its accuracy is higher than the manufacturer’s calibration.

Parallel Structure from Motion from Local Increment to Global Averaging

Abstract: In this paper, we tackle the accurate and consistent Structure from Motion (SfM) problem, in particular camera registration, far exceeding the memory of a single computer in parallel. Different from the previous methods which drastically simplify the parameters of SfM and sacrifice the accuracy of the final reconstruction, we try to preserve the connectivities among cameras by proposing a camera clustering algorithm to divide a large SfM problem into smaller sub-problems in terms of camera clusters with overlapping. We then exploit a hybrid formulation that applies the relative poses from local incremental SfM into a global motion averaging framework and produce accurate and consistent global camera poses. Our scalable formulation in terms of camera clusters is highly applicable to the whole SfM pipeline including track generation, local SfM, 3D point triangulation and bundle adjustment. We are even able to reconstruct the camera poses of a city-scale data-set containing more than one million high-resolution images with superior accuracy and robustness evaluated on benchmark, Internet, and sequential data-sets.

Multiscale gigapixel video: A cross resolution image matching and warping approach

Abstract: We present a multi-scale camera array to capture and synthesize gigapixel videos in an efficient way. Our acquisition setup contains a reference camera with a short-focus lens to get a large field-of-view video and a number of unstructured long-focus cameras to capture local-view details. Based on this new design, we propose an iterative feature matching and image warping method to independently warp each local-view video to the reference video. The key feature of the proposed algorithm is its robustness to and high accuracy for the huge resolution gap (more than 8x resolution gap between the reference and the local-view videos), camera parallaxes, complex scene appearances and color inconsistency among cameras. Experimental results show that the proposed multi-scale camera array and cross resolution video warping scheme is capable of generating seamless gigapixel video without the need of camera calibration and large overlapping area constraints between the local-view cameras.

Systems and methods for dynamic object tracking using a single camera mounted on a moving object

Abstract: A self-contained, low-cost, low-weight guidance system for vehicles is provided. The guidance system can include an optical camera, a case, a processor, a connection between the processor and an on-board control system, and computer algorithms running on the processor. The guidance system can be integrated with a vehicle control system through “plug and play” functionality or a more open Software Development Kit. The computer algorithms re-create 3D structures as the vehicle travels and continuously updates a 3D model of the environment. The guidance system continuously identifies and tracks terrain, static objects, and dynamic objects through real-time camera images. The guidance system can receive inputs from the camera and the onboard control system. The guidance system can be used to assist vehicle navigation and to avoid possible collisions. The guidance system can communicate with the control system and provide navigational direction to the control system.

Unsupervised Vanishing Point Detection and Camera Calibration from a Single Manhattan Image with Radial Distortion

Abstract: The article concerns the automatic calibration of a camera with radial distortion from a single image. It is known that, under the mild assumption of square pixels and zero skew, lines in the scene project into circles in the image, and three lines suffice to calibrate the camera up to an ambiguity between focal length and radial distortion. The calibration results highly depend on accurate circle estimation, which is hard to accomplish because lines tend to project into short circular arcs. To overcome this problem, we show that, given a short circular arc edge, it is possible to robustly determine a line that goes through the center of the corresponding circle. These lines, henceforth called Lines of Circle Centres (LCCs), are used in a new method that detects sets of parallel lines and estimates the calibration parameters, including the center and amount of distortion, focal length, and camera orientation with respect to the Manhattan frame. Extensive experiments in both semi-synthetic and real images show that our algorithm outperforms state-of-the-art approaches in unsupervised calibration from a single image, while providing more information.

Bionic Orientation and Visual Enhancement With a Novel Polarization Camera

Abstract: The remarkable polarization vision of animals provides a significant inspiration for robotic navigation and visual enhancement, as polarization pattern provides additional information besides spectral signatures. A novel bio-inspired polarization camera is proposed in this paper, which can realize real-time image-based polarization measurement. The composition of the system is described and the optimal estimation of the polarization state is derived based on the least square algorithm. This paper concentrates particularly on the camera orientation algorithms and visual enhancement methods with it. To estimate the camera’s heading angle with the skylight polarization pattern, the sun vector is established as an optimization problem of finding the minimum eigenvector. The solar meridian is also estimated from the degree of polarization pattern by detecting reflectional symmetry axes. The result shows that the measured polarization patterns are very close to the theory. The maximum orientation error of the proposed method based on angle of polarization is about 0.5°. The average error is 0.012° with standard deviation of 0.28°. Thus, the novel polarization camera could be used as sun compass. When observing scenes in distance, the polarization camera is used to decouple the airlight from the object radiance, which results in much better contrast. More importantly, the polarization information is helpful for scene identification and object detection. The result also shows that the polarization camera can reasonably cope with the semi-reflection problem.

Automatic Camera Calibration Using Active Displays of a Virtual Pattern.

Abstract: Camera calibration plays a critical role in 3D computer vision tasks. The most commonly used calibration method utilizes a planar checkerboard and can be done nearly fully automatically. However, it requires the user to move either the camera or the checkerboard during the capture step. This manual operation is time consuming and makes the calibration results unstable. In order to solve the above problems caused by manual operation, this paper presents a full-automatic camera calibration method using a virtual pattern instead of a physical one. The virtual pattern is actively transformed and displayed on a screen so that the control points of the pattern can be uniformly observed in the camera view. The proposed method estimates the camera parameters from point correspondences between 2D image points and the virtual pattern. The camera and the screen are fixed during the whole process; therefore, the proposed method does not require any manual operations. Performance of the proposed method is evaluated through experiments on both synthetic and real data. Experimental results show that the proposed method can achieve stable results and its accuracy is comparable to the standard method by Zhang.

Fast Multi-frame Stereo Scene Flow with Motion Segmentation

Abstract: We propose a new multi-frame method for efficiently computing scene flow (dense depth and optical flow) and camera ego-motion for a dynamic scene observed from a moving stereo camera rig. Our technique also segments out moving objects from the rigid scene. In our method, we first estimate the disparity map and the 6-DOF camera motion using stereo matching and visual odometry. We then identify regions inconsistent with the estimated camera motion and compute per-pixel optical flow only at these regions. This flow proposal is fused with the camera motion-based flow proposal using fusion moves to obtain the final optical flow and motion segmentation. This unified framework benefits all four tasks - stereo, optical flow, visual odometry and motion segmentation leading to overall higher accuracy and efficiency. Our method is currently ranked third on the KITTI 2015 scene flow benchmark. Furthermore, our CPU implementation runs in 2-3 seconds per frame which is 1-3 orders of magnitude faster than the top six methods. We also report a thorough evaluation on challenging Sintel sequences with fast camera and object motion, where our method consistently outperforms OSF [Menze and Geiger, 2015], which is currently ranked second on the KITTI benchmark.

A Comprehensive and Versatile Camera Model for Cameras with Tilt Lenses

Abstract: We propose camera models for cameras that are equipped with lenses that can be tilted in an arbitrary direction (often called Scheimpflug optics). The proposed models are comprehensive: they can handle all tilt lens types that are in common use for machine vision and consumer cameras and correctly describe the imaging geometry of lenses for which the ray angles in object and image space differ, which is true for many lenses. Furthermore, they are versatile since they can also be used to describe the rectification geometry of a stereo image pair in which one camera is perspective and the other camera is telecentric. We also examine the degeneracies of the models and propose methods to handle the degeneracies. Furthermore, we examine the relation of the proposed camera models to different classes of projective camera matrices and show that all classes of projective cameras can be interpreted as cameras with tilt lenses in a natural manner. In addition, we propose an algorithm that can calibrate an arbitrary combination of perspective and telecentric cameras (no matter whether they are tilted or untilted). The calibration algorithm uses a planar calibration object with circular control points. It is well known that circular control points may lead to biased calibration results. We propose two efficient algorithms to remove the bias and thus obtain accurate calibration results. Finally, we perform an extensive evaluation of the proposed camera models and calibration algorithms that establishes the validity and accuracy of the proposed models.

An Iterative Distortion Compensation Algorithm for Camera Calibration Based on Phase Target

Abstract: Camera distortion is a critical factor affecting the accuracy of camera calibration A conventional calibration approach cannot satisfy the requirement of a measurement system demanding high calibration accuracy due to the inaccurate distortion compensation This paper presents a novel camera calibration method with an iterative distortion compensation algorithm The initial parameters of the camera are calibrated by full-field camera pixels and the corresponding points on a phase target An iterative algorithm is proposed to compensate for the distortion A 2D fitting and interpolation method is also developed to enhance the accuracy of the phase target Compared to the conventional calibration method, the proposed method does not rely on a distortion mathematical model, and is stable and effective in terms of complex distortion conditions Both the simulation work and experimental results show that the proposed calibration method is more than 100% more accurate than the conventional calibration method

Unsupervised Adaptive Re-identification in Open World Dynamic Camera Networks

Abstract: Person re-identification is an open and challenging problem in computer vision. Existing approaches have concentrated on either designing the best feature representation or learning optimal matching metrics in a static setting where the number of cameras are fixed in a network. Most approaches have neglected the dynamic and open world nature of the re-identification problem, where a new camera may be temporarily inserted into an existing system to get additional information. To address such a novel and very practical problem, we propose an unsupervised adaptation scheme for re-identification models in a dynamic camera network. First, we formulate a domain perceptive re-identification method based on geodesic flow kernel that can effectively find the best source camera (already installed) to adapt with a newly introduced target camera, without requiring a very expensive training phase. Second, we introduce a transitive inference algorithm for re-identification that can exploit the information from best source camera to improve the accuracy across other camera pairs in a network of multiple cameras. Extensive experiments on four benchmark datasets demonstrate that the proposed approach significantly outperforms the state-of-the-art unsupervised learning based alternatives whilst being extremely efficient to compute.

Robust Geometric Self-Calibration of Generic Multi-Projector Camera Systems

Abstract: Calibration of multi-projector-camera systems (MPCS) is a cumbersome and time-consuming process. It is of great importance to have robust, fast and accurate calibration procedures at hand for a wide variety of practical applications. We propose a fully automated self-calibration method for arbitrarily complex MPCS. It enables reliable and accurate intrinsic and extrinsic calibration without any human parameter tuning. We evaluated the proposed methods using more than ten multi-projection datasets ranging from a toy castle set up consisting of three cameras and one projector up to a half dome display system with more than 30 devices. Comparisons to reference calibrations, which were generated using the standard checkerboard calibration approach [44], show the reliability of our proposed pipeline, while a ground truth evaluation also shows that the resulting reconstructed point cloud accurately matches the shape of the reference geometry. Besides being fully automatic without the necessity of parameter fine tuning, the proposed method also significantly reduces the installation time of MPCS compared to checkerboard-based methods and makes it more suitable for real-world applications.

Operator-based homogeneous coordinates: application in camera document scanning

Abstract: An operator-based approach for the study of homogeneous coordinates and projective geometry is proposed. First, some basic geometrical concepts and properties of the operators are investigated in the one- and two-dimensional cases. Then, the pinhole camera model is derived, and a simple method for homography estimation and camera calibration is explained. The usefulness of the analyzed theoretical framework is exemplified by addressing the perspective correction problem for a camera document scanning application. Several experimental results are provided for illustrative purposes. The proposed approach is expected to provide practical insights for inexperienced students on camera calibration, computer vision, and optical metrology among others.

Appearance and motion based deep learning architecture for moving object detection in moving camera

Abstract: Background subtraction from the given image is a widely used method for moving object detection. However, this method is vulnerable to dynamic background in a moving camera video. In this paper, we propose a novel moving object detection approach using deep learning to achieve a robust performance even in a dynamic background. The proposed approach considers appearance features as well as motion features. To this end, we design a deep learning architecture composed of two networks: an appearance network and a motion network. The two networks are combined to detect moving object robustly to the background motion by utilizing the appearance of the target object in addition to the motion difference. In the experiment, it is shown that the proposed method achieves 50 fps speed in GPU and outperforms state-of-the-art methods for various moving camera videos.

A prompt methodology to georeference complex hypogea environments

Abstract: . Actually complex underground structures and facilities occupy a wide space in our cities, most of them are often unsurveyed; cable duct, drainage system are not exception. Furthermore, several inspection operations are performed in critical air condition, that do not allow or make more difficult a conventional survey. In this scenario a prompt methodology to survey and georeferencing such facilities is often indispensable. A visual based approach was proposed in this paper; such methodology provides a 3D model of the environment and the path followed by the camera using the conventional photogrammetric/Structure from motion software tools. The key-role is played by the lens camera; indeed, a fisheye system was employed to obtain a very wide field of view (FOV) and therefore high overlapping among the frames. The camera geometry is in according to a forward motion along the axis camera. Consequently, to avoid instability of bundle adjustment algorithm a preliminary calibration of camera was carried out. A specific case study was reported and the accuracy achieved.

Review of techniques for 2D camera calibration suitable for industrial vision systems

Abstract: In camera calibration the goal is to determine a set of camera parameters that describe the mapping between 3-D reference coordinates and 2-D image ones. Correction for image distortion in cameras is an important topic in order to obtain accurate information from the vision systems. A review of calibration techniques for the vision systems is presented among the different methods for camera calibration found in literature. The advantages and limitations of these techniques are also discussed. This work analyzes the techniques that can be useful in modern industrial contest with the aim of knowing the effective distortion of the lens and the possibility to determinate the measurement uncertainty of the vision system.