An Improved Method for Image Retrieval using Speech Annotation

TLDR: This paper presents a system for the image indexing and retrieval using speech annotations based on a pre-defined structured syntax, and a query expansion technique is explored to enhance the query terms and to improve retrieval effectiveness.

Abstract: In this paper, we present a system for the image indexing and retrieval using speech annotations based on a pre-defined structured syntax. In addition to the introduction of N-best lists for index generation, a query expansion technique is explored to enhance the query terms and to improve retrieval effectiveness. By adding the most probable substitutions for the query terms, more relevant images are distinguished from the data collection. This approach is particularly helpful to deal with those less frequently used words, including out-of-vocabulary (OOV) words, which are very common for names of people and places. Experiments on a collection of 1,200 photos show that the retrieval effectiveness is increased considerably for segment of individual domain on People, Location and Event. With this method, the average value of precision versus recall over a combination of segments has improved significantly, from 50% to 72.4%.