Formulations of the Image Decomposition Problem [18] as a Multicut Problem (MP) w.r.t. a superpixel graph have received considerable attention. In contrast, instances of the MP w.r.t. a pixel grid graph have received little attention, firstly, because the MP is NP-hard and instances w.r.t. a pixel grid graph are hard to solve in practice, and, secondly, due to the lack of long-range terms in the objective function of the MP. We propose a generalization of the MP with long-range terms (LMP). We design and implement two efficient algorithms (primal feasible heuristics) for the MP and LMP which allow us to study instances of both problems w.r.t. the pixel grid graphs of the images in the BSDS-500 benchmark. The decompositions we obtain do not differ significantly from the state of the art, suggesting that the LMP is a competitive formulation of the Image Decomposition Problem. To demonstrate the generality of the LMP, we apply it also to the Mesh Decomposition Problem posed by the Princeton benchmark [16], obtaining state-of-the-art decompositions.

/pdf/efficient-decomposition-of-image-and-mesh-graphs-by-lifted-14690owu4h.pdf

Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts

The last decade has seen innovations that make video recording, manipulation, storage, and sharing easier than ever before, thus impacting many areas of life. New video retrieval scenarios emerged as well, which challenge the state-of-the-art video retrieval approaches. Despite recent advances in content analysis, video retrieval can still benefit from involving the human user in the loop. We present our experience with a class of interactive video retrieval scenarios and our methodology to stimulate the evolution of new interactive video retrieval approaches. More specifically, the video browser showdown evaluation campaign is thoroughly analyzed, focusing on the years 2015–2017. Evaluation scenarios, objectives, and metrics are presented, complemented by the results of the annual evaluations. The results reveal promising interactive video retrieval techniques adopted by the most successful tools and confirm assumptions about the different complexity of various types of interactive retrieval scenarios. A comparison of the interactive retrieval tools with automatic approaches (including fully automatic and manual query formulation) participating in the TRECVID 2016 ad hoc video search task is discussed. Finally, based on the results of data analysis, a substantial revision of the evaluation methodology for the following years of the video browser showdown is provided.

On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015–2017

Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which could be arbitrarily worse than the optimal solution. Inspired by the relationship between the negative log-likelihood function and the Kullback-Leibler (KL) divergence, we propose an alternative formulation for estimating the GMM parameters using the sliced Wasserstein distance, which gives rise to a new algorithm. Specifically, we propose minimizing the sliced-Wasserstein distance between the mixture model and the data distribution with respect to the GMM parameters. In contrast to the KL-divergence, the energy landscape for the sliced-Wasserstein distance is more well-behaved and therefore more suitable for a stochastic gradient descent scheme to obtain the optimal GMM parameters. We show that our formulation results in parameter estimates that are more robust to random initializations and demonstrate that it can estimate high-dimensional data distributions more faithfully than the EM algorithm.

/pdf/sliced-wasserstein-distance-for-learning-gaussian-mixture-493j5uy1m7.pdf

Sliced Wasserstein Distance for Learning Gaussian Mixture Models

The similarity or distance measure between Gaussian mixture models (GMMs) plays a crucial role in content-based image matching. Though the Earth Mover's Distance (EMD) has shown its advantages in matching histogram features, its potentials in matching GMMs remain unclear and are not fully explored. To address this problem, we propose a novel EMD methodology for GMM matching. We first present a sparse representation based EMD called SR-EMD by exploiting the sparse property of the underlying problem. SR-EMD is more efficient and robust than the conventional EMD. Second, we present two novel ground distances between component Gaussians based on the information geometry. The perspective from the Riemannian geometry distinguishes the proposed ground distances from the classical entropy-or divergence-based ones. Furthermore, motivated by the success of distance metric learning of vector data, we make the first attempt to learn the EMD distance metrics between GMMs by using a simple yet effective supervised pair-wise based method. It can adapt the distance metrics between GMMs to specific classification tasks. The proposed method is evaluated on both simulated data and benchmark real databases and achieves very promising performance.

/pdf/a-novel-earth-mover-s-distance-methodology-for-image-3wmdg67ce7.pdf

A Novel Earth Mover's Distance Methodology for Image Matching with Gaussian Mixture Models

Cell detection and segmentation in microscopy images is important for quantitative high-throughput experiments We present a learning-based method that is applicable to different modalities and cell types, in particular to cells that appear almost transparent in the images We first train a classifier to detect (partial) cell boundaries The resulting predictions are used to obtain superpixels and a weighted region adjacency graph Here, edge weights can be either positive (attractive) or negative (repulsive) The graph partitioning problem is then solved using correlation clustering segmentation One variant we newly propose here uses a length constraint that achieves state-of-art performance and improvements in some datasets This constraint is approximated using non-planar correlation clustering We demonstrate very good performance in various bright field and phase contrast microscopy experiments

/pdf/cell-detection-and-segmentation-using-correlation-clustering-2c0jj3pvot.pdf

Cell Detection and Segmentation Using Correlation Clustering

Modeling image similarity for browsing and searching in voluminous image databases is a challenging task of nearly all content-based image retrieval systems. One promising way of defining image similarity consists in applying distance-based similarity measures on compact image representations. Beyond feature histograms and feature signatures, more general feature representations are mixture models of which the Gaussian mixture model is the most prominent one. This feature representation can be compared by employing approximations of the Kullback-Leibler Divergence. Although several of those approximations have been successfully applied to model image similarity, their applicability to mixture models based on high-dimensional feature descriptors is questionable. In this paper, we thus introduce the Signature Quadratic Form Distance to measure the distance between two Gaussian mixture models of high-dimensional feature descriptors. We show the analytical computation of the proposed Gaussian Quadratic Form Distance and evaluate its retrieval performance by making use of different benchmark image databases.

Modeling image similarity by Gaussian mixture models and the Signature Quadratic Form Distance

We propose a simple yet effective approach to content-based image retrieval: the signature matching distance. While recent approaches to content-based image retrieval utilize the bag-of-visual-words model, where image descriptors are matched through a common visual vocabulary, signature-based approaches use a distance between signatures, i.e. between image-specific bags of locally aggregated descriptors, in order to quantify image dissimilarity. In this paper, we focus on the signature-based approach to content-based image retrieval and propose a novel distance function, the signature matching distance. This distance matches coincident visual properties of images based on their signatures. In particular, by investigating different descriptor matching strategies and their suitability to match signatures, we show that our approach is able to outperform other signature-based approaches to content-based image retrieval. Moreover, in combination with a simple color and texture-based image descriptor, our approach is able to compete with the majority of bag-of-visual-words approaches.

Signature matching distance for content-based image retrieval

Retrieving similar images from large image databases is a challenging task for today's content-based retrieval systems. Aiming at high retrieval performance, these systems frequently capture the user's notion of similarity through expressive image models and adaptive similarity measures. On the query side, image models can significantly differ in quality compared to those stored on the database side. Thus, similarity measures have to be robust against these individual quality changes in order to maintain high retrieval performance. In this paper, we investigate the robustness of the family of signature-based similarity measures in the context of content-based image retrieval. To this end, we introduce the generic concept of average precision stability, which measures the stability of a similarity measure with respect to changes in quality between the query and database side. In addition to the mathematical definition of average precision stability, we include a performance evaluation of the major signature-based similarity measures focusing on their stability with respect to querying image databases by examples of varying quality. Our performance evaluation on recent benchmark image databases reveals that the highest retrieval performance does not necessarily coincide with the highest stability.

On stability of signature-based similarity measures for content-based image retrieval

We address the problem of segmenting an image into a previously unknown number of segments from the perspective of graph partitioning. Specifically, we consider minimum multicuts of superpixel affinity graphs in which all affinities between non-adjacent superpixels are negative. We propose a relaxation by Lagrangian decomposition and a constrained set of re-parameterizations for which we can optimize exactly and efficiently. Our contribution is to show how the planarity of the adjacency graph can be exploited if the affinity graph is non-planar. We demonstrate the effectiveness of this approach in user-assisted image segmentation and show that the solution of the relaxed problem is fast and the relaxation is tight in practice.

/pdf/segmentingzplanarzsuperpixelzadjacencyzgraphs-w-r-t-znon-3c153541ly.pdf

SegmentingźPlanarźSuperpixelźAdjacencyźGraphs w.r.t.źNon-planar Superpixel Affinity Graphs

Video segmentation requires separating foreground from background, but the general problem extends to more complicated scene segmentations of different objects and their multiple parts. We develop a new approach to interactive multi-label video segmentation where many objects are segmented simultaneously with consistent spatio-temporal boundaries, based on intuitive multi-colored brush scribbles. From these scribbles, we derive constraints to define a combinatorial problem known as the multicut---a problem notoriously difficult and slow to solve. We describe a solution using efficient heuristics to make multi-label video segmentation interactive. As our solution generalizes typical binary segmentation tasks, while also improving efficiency in multi-label tasks, our work shows the promise of multicuts for interactive video segmentation.

https://hal.archives-ouvertes.fr/hal-01378800/document

Steffen Kirchhoff

Papers

Modeling image similarity by Gaussian mixture models and the Signature Quadratic Form Distance

Signature matching distance for content-based image retrieval

On stability of signature-based similarity measures for content-based image retrieval

SegmentingźPlanarźSuperpixelźAdjacencyźGraphs w.r.t.źNon-planar Superpixel Affinity Graphs

Interactive multicut video segmentation