Users of image databases often prefer to retrieve relevant images by categories. Unfortunately, images are usually indexed by low-level features like color, texture and shape, which often fail to capture high-level concepts well. To address this issue, relevance feedback has been extensively used to associate low-level image features with highlevel concepts. Among all existing relevance feedback approaches, query movement and feature re-weighting have been proven to be suitable for large-scaled image databases with high dimensional image features. We present a feature re-weighting approach using relevant images as well as irrelevant ones in the relevance feedback. As far as feature re-weighting approaches are concerned, one of their common drawbacks is that the feature re-weighting process is prone to be trapped by suboptimal states. To overcome this problem, we introduce a disturbing factor, which is based on the Fisher criterion, to push the feature weights out of sub-optimum. Experimental results on a large-scaled image database with 31,438 COREL images demonstrate the effectiveness of the presented method.

A feature re-weighting approach for relevance feedback in image retrieval

Understanding the subjective meaning of a visual query, by converting it into numerical parameters that can be extracted and compared by a computer, is the paramount challenge in the field of intelligent image retrieval, also referred to as the ?semantic gap? problem. In this paper, an innovative approach is proposed that combines a relevance feedback (RF) approach with an evolutionary stochastic algorithm, called particle swarm optimizer (PSO), as a way to grasp user's semantics through optimized iterative learning. The retrieval uses human interaction to achieve a twofold goal: 1) to guide the swarm particles in the exploration of the solution space towards the cluster of relevant images; 2) to dynamically modify the feature space by appropriately weighting the descriptive features according to the users' perception of relevance. Extensive simulations showed that the proposed technique outperforms traditional deterministic RF approaches of the same class, thanks to its stochastic nature, which allows a better exploration of complex, nonlinear, and highly-dimensional solution spaces.

A Stochastic Approach to Image Retrieval Using Relevance Feedback and Particle Swarm Optimization

Due to the advancement of camera hardware and machine learning techniques, video object tracking for surveillance has received noticeable attention from the computer vision research community Object tracking and trajectory modeling have important applications in surveillance video analysis For example, trajectory clustering, summarization or synopsis generation, and detection of anomalous or abnormal events in videos are mainly being exploited by the research community However, barring one research work (which is almost a decade old), there is no recent review that emphasizes the use of video object trajectories, particularly in the perspective of visual surveillance This paper presents a survey of trajectory-based surveillance applications with a focus on clustering, anomaly detection, summarization, and synopsis generation The methods reviewed in this paper broadly summarize the abovementioned applications The main purpose of this survey is to summarize the state-of-the-art video object trajectory analysis techniques used in the indoor and outdoor surveillance

Trajectory-Based Surveillance Analysis: A Survey

An integrated approach to region based image retrieval using firefly algorithm and support vector machine

The bag-of-words (BoW) model has been known as an effective method for large-scale image search and indexing. Recent work shows that the performance of the model can be further improved by using the embedding method. While different variants of the BoW model and embedding method have been developed, less effort has been made to discover their underlying working mechanism. In this paper, we systematically investigate the image search performance variation with respect to a few factors of the BoW model, and study how to employ the embedding method to further improve the image search performance. Subsequently, we summarize several observations based on the experiments on descriptor matching. To validate these observations in a real image search, we propose an effective and efficient image search scheme, in which the BoW model and embedding method are jointly optimized in terms of effectiveness and efficiency by following these observations. Our comprehensive experiments demonstrate that it is beneficial to employ these observations to develop an image search algorithm, and the proposed image search scheme outperforms state-of-the-art methods in both effectiveness and efficiency.

Joint Optimization Toward Effective and Efficient Image Search

The focus of this chapter is to present a survey on the most recent advances in representation and analysis of video object trajectories, with application to indexing and retrieval systems. We will review the main methodologies for the description of motion trajectories, as well as the indexing techniques and similarity metrics used in the retrieval process. Strengths and weaknesses of different solutions will be discussed through a comparative analysis, taking into account performance and implementation issues. In order to provide a deeper insight on the exploitation of these technologies in real world products, a selection of exampleswill be introduced and examined. The set of possible applications is very wide and includes (but it is not limited to) generic browsing of video databases, as well as more specific and context-dependent scenarios such as indexing and retrieval in visual surveillance, traffic monitoring, sport events analysis, video-on-demand, and video broadcasting.

Object Trajectory Analysis in Video Indexing and Retrieval Applications

An innovative approach based on an evolutionary stochastic algorithm, namely the Particle Swarm Optimizer (PSO), is proposed in this paper as a solution to the problem of intelligent retrieval of images in large databases. The problem is recast to an optimization one, where a suitable cost function is minimized through a customized PSO. Accordingly, the relevance-feedback is used in order to exploit the information of the user with the aim of both guiding the particles inside the search space and dynamically assigning different weights to the features.

Content-based image retrieval by a semi-supervised Particle Swarm Optimization

The automatic extraction of video structure from content is of key importance to enable a variety of multimedia services that span from search and retrieval to content manipulation. An unsupervised independent unimodal clustering method for anchorpersons detection and differentiation in newscasts is presented in this paper. The algorithm exploits audio, frame and face information to identify major cast in the content. These three components are first processed independently during the cluster analysis and then jointly in a compositional mining phase. A differentiation of the role played by the people in the video has been implemented exploiting the temporal characteristics of the detected anchorpersons. Experiments show significant precision/recall results thus opening further research directions in video analysis, particularly when the content is highly structured as in TV newscasts.

Unsupervised anchorpersons differentiation in news video

In the age of content snacking and mobisodes (mobile episodes) the paradigm of media consumption is radically changing. Media consumption is moving from monolithic, prepackaged, well-edited, and elaborate content presentation to a continuous feed of brief segments as singleton episodes and few-minutes videos, that are often supported by or initiated via tweets and status updates. In these updates, attention spans are small and the content packaging is less relevant with respect to the dynamic, 'streaming' aspect of information. This trend has a profound influence on the segmentation requirements that are needed to make this stream of possible information. In this paper, we present a novel method to automatically extract structured content, events, where events include major cast interviews, dialogs, background segments, etc. from news video in an unsupervised fashion. Two key ideas differentiate this unsupervised method from the others: the type of information that we use to find events and the method utilized to combine this information for coherent multimedia events. The proposed system exploits audio, visual appearance, detected faces, and mid-level semantic concepts from every video shot, but instead of combining everything together, the framework clusters them independently and by applying coherence rules assembles the multimedia events. Additionally, we discuss the effect of segmentation errors in practical retrieval and content consumption tasks.

Mattia Broilo

Papers

A Stochastic Approach to Image Retrieval Using Relevance Feedback and Particle Swarm Optimization

Object Trajectory Analysis in Video Indexing and Retrieval Applications

Content-based image retrieval by a semi-supervised Particle Swarm Optimization

Unsupervised anchorpersons differentiation in news video

Unsupervised event segmentation of news content with multimodal cues