The visual virtual tourist guide: a markerless camera-based LBS system

TLDR: This work has developed a system which localizes solely based on naturally occurring landmarks, using a panoramic camera system which provides omnidirectional images of the environment and a Bayesian framework makes it possible to track the system's position in quasi real time.

Abstract: This work was motivated by the goal of building a wearable virtual tourist system that could guide people around in large complex environments. It must yield a precise localization, even in situations in which Global Positioning Systems (GPS) cannot provide navigational information, including include indoor locations and narrow streets where there is no line of sight to the GPS satellites. Because installing active badges or beacon systems involves substantial effort and expense, we have developed a system which localizes solely based on naturally occurring landmarks. As sensory input, we only use a panoramic camera system which provides omnidirectional images of the environment. During the training stage, the system is led around in the environment while recording images at regular time intervals. Offline, these images are automatically archived in a world model. Unlike traditional approaches we don’t build an Euclidean metrical map. The used world model is a graph reflecting the topological structure of the environment: e.g. for indoor environments rooms are nodes and corridors are edges of the graph. Image comparison is done using both global color measures and matching of specially developed local features. These measures are designed to be robust, respectively invariant, to image distortions caused by viewpoint changes, illumination changes and occlusions. This leads to a system that can recognize a certain place even if its location is not exactly the same as the location from where the reference image was taken, even if the illumination is substantially different, and even if there are large occluded parts. Using this world model, localization can be done by comparing a new query image, taken at the current position of the mobile system, with the images in the model. A Bayesian framework makes it possible to track the system’s position in quasi real time. When the present location is known, context­sensitive touristical information can be given to the tourist, or a path to a target location can be carried out using the topological map.