This study presents a review of the state-of-the-art and a novel classification of current vision-based localisation techniques in unknown environments. Indeed, because of progresses made in computer vision, it is now possible to consider vision-based systems as promising navigation means that can complement traditional navigation sensors like global navigation satellite systems (GNSSs) and inertial navigation systems. This study aims to review techniques employing a camera as a localisation sensor, provide a classification of techniques and introduce schemes that exploit the use of video information within a multi-sensor system. In fact, a general model is needed to better compare existing techniques in order to decide which approach is appropriate and which are the innovation axes. In addition, existing classifications only consider techniques based on vision as a standalone tool and do not consider video as a sensor among others. The focus is addressed to scenarios where no a priori knowledge of the environment is provided. In fact, these scenarios are the most challenging since the system has to cope with objects as they appear in the scene without any prior information about their expected position.

/pdf/review-and-classification-of-vision-based-localisation-21ctu1tf06.pdf

Review and classification of vision-based localisation techniques in unknown environments

Obtaining the land vehicle’s position, azimuth and attitudes autonomously and accurately is significant for vehicle-based weapon’s combat effectiveness. High performance inertial navigation system (INS) can provide high precision attitudes and azimuth reference information, however, it is usually very expensive. In addition, due to gyroscope drifts and accelerometer biases, INS navigation errors would diverge with time. To solve the problem, this paper proposed an integrated positioning and orientation method based on the FOG single-axis rotational INS (FRINS) and the odometer (OD). The proposed method adopted the rotation modulation technique to suppress the inertial sensor errors, particularly $z$ -axis FOG drift, by which the navigation accuracy, especially the azimuth accuracy, can be improved. At the same time, the proposed method integrated the FRINS with OD to suppress INS error’s divergence with time. Combining the above rotation modulation technique and the integration of FRINS and OD, the proposed method could improve the integrated system positioning and orientation accuracy as well as reduce the integrated system cost. This paper presented the FRINS/OD integrated navigation system configuration, established the FRINS/OD integrated system errors model, and then verified the performance of proposed method by simulations. The results show that the proposed method could achieve positioning accuracy of 0.03% (of total traveled distance) and orientation of ±20, respectively, using rotational INS with FOGs of 0.1°/h and accelerometers of 500 ug.

Study on Integration of FOG Single-Axis Rotational INS and Odometer for Land Vehicle

Global Navigation Satellite Systems (GNSS) integrated with multiple sensors have been widely applied in many Intelligent Transport Systems (ITS). Intelligent vehicles, increasingly a key component of the future transport system, require high-performance positioning, navigation and timing (PNT) technologies. This cannot be achieved by current 2-dimnsional systems. This is because accurate comprehensive state information (time, position, derivatives and attitude) is required to estimate amongst others, the vehicle’s power usage as well as to enable precise environmental mapping based on accurate state of the relevant sensors and path planning including in complex multi-level intersections. Because of the multipath effects and signal interruption in urban environments, comprehensive vehicle state estimation is not always available at the required level of performance. To address this issue, we propose an effective method to integrate the Inertial Measurement Unit (IMU), Global Positioning System (GPS) and monocular Visual Odometry (VO) for urban vehicle navigation. A robust Extended Kalman Filter (EKF) based two-step integration algorithm is developed with a non-holonomic constraint (NHC). In particular, the NHC is not only applied on the offline VO error modelling process, but also on the online sensor fusion process to improve the 3D vehicle state estimation. The proposed IMU/GPS/VO integration scheme is tested with various sensor levels in different urban environments. The results show that the proposed IMU/GPS/VO fusion algorithm could deliver a 3D RMSE of 3.285m, which outperforms the other conventional candidate fusion schemes in the noisy GNSS urban areas. The further test in the urban with outages has demonstrated that the proposed algorithm delivers an overall 3D RMSE of 1.290m, 0.073m/s and 0.486 degrees, in terms of positioning, velocity and attitude, respectively. It is also demonstrated that the proposed low-cost VO integration with IMU/GPS could achieve similar performance with the high-cost odometer based integration in deep urban areas but with advantages of higher flexibility and lower cost.

Robust IMU/GPS/VO Integration for Vehicle Navigation in GNSS Degraded Urban Areas

Vision navigation technique, especially the vision-based simultaneous localization and mapping (V-SLAM), plays a critical role in robotic navigation. As a relative positioning technique, V-SLAM often suffers from drift and scale uncertainty problems which incur bias increasing over time. To overcome these drawbacks and to improve the robustness and accuracy of localization, an effective way is to fuse global navigation satellite system (GNSS) with V-SLAM. In this paper, we propose a novel GNSS and SLAM fusion algorithm, which provides ego-motion estimation through tightly coupling GNSS pseudo-range measurements and camera feature points. It first decomposes the pose state into basic motion vectors, based on which an asynchronous tracking is performed. Then, a 10-DoF joint-optimization formulation on manifold is proposed to achieve tight fusion of the raw measurement from camera and GNSS. Finally, this formulation is solved to calculate the ego-motion state. The proposed algorithm is verified on an autonomous ground vehicle in two typical environments. The results demonstrated that, the new algorithm can amend the bias in vision SLAM and constrain the GNSS solution, which achieves a better localization result than the traditional methods.

Tightly Coupled Integration of GNSS and Vision SLAM Using 10-DoF Optimization on Manifold

Online calibration and compensation of total odometer error in an integrated system

Combining GPS/INS/odometer data has been considered one of the most attractive methodologies for ground vehicle navigation. In the case of long GPS signal blockages inherent to complex urban environments, however, the accuracy of this approach is largely deteriorated. To overcome this limitation, this study proposes a novel ground vehicle navigation system that combines INS, odometer and omnidirectional vision sensor. Compared to traditional cameras, omnidirectional vision sensors can acquire much more information from the environment thanks to their wide field of view. The proposed system automatically extracts and tracks vanishing points in omnidirectional images to estimate the vehicle rotation. This scheme provides robust navigation information: specifically by combining the advantages of vision, odometer and INS, we estimate the attitude without error accumulation and at a fast running rate. The accurate rotational information is fed back into a Kalman filter to improve the quality of the INS bridging in harsh urban conditions. Extensive experiments have demonstrated that the proposed approach significantly reduces the accumulation of position, velocity and attitude errors during simulated GPS outages. Specifically, the position accuracy is improved by over 30% during simulated GPS outages.

S.-B. Kim

Papers

Ground vehicle navigation in harsh urban conditions by integrating inertial navigation system, global positioning system, odometer and vision data