There is a large variety of trackers, which have been proposed in the literature during the last two decades with some mixed success. Object tracking in realistic scenarios is a difficult problem, therefore, it remains a most active area of research in computer vision. A good tracker should perform well in a large number of videos involving illumination changes, occlusion, clutter, camera motion, low contrast, specularities, and at least six more aspects. However, the performance of proposed trackers have been evaluated typically on less than ten videos, or on the special purpose datasets. In this paper, we aim to evaluate trackers systematically and experimentally on 315 video fragments covering above aspects. We selected a set of nineteen trackers to include a wide variety of algorithms often cited in literature, supplemented with trackers appearing in 2010 and 2011 for which the code was publicly available. We demonstrate that trackers can be evaluated objectively by survival curves, Kaplan Meier statistics, and Grubs testing. We find that in the evaluation practice the F-score is as effective as the object tracking accuracy (OTA) score. The analysis under a large variety of circumstances provides objective insight into the strengths and weaknesses of trackers.

/pdf/visual-tracking-an-experimental-survey-50w2t40998.pdf

Visual Tracking: An Experimental Survey

In this paper we present a tracker, which is radically different from state-of-the-art trackers: we apply no model updating, no occlusion detection, no combination of trackers, no geometric matching, and still deliver state-of-theart tracking performance, as demonstrated on the popular online tracking benchmark (OTB) and six very challenging YouTube videos The presented tracker simply matches the initial patch of the target in the first frame with candidates in a new frame and returns the most similar patch by a learned matching function The strength of the matching function comes from being extensively trained generically, ie, without any data of the target, using a Siamese deep neural network, which we design for tracking Once learned, the matching function is used as is, without any adapting, to track previously unseen targets It turns out that the learned matching function is so powerful that a simple tracker built upon it, coined Siamese INstance search Tracker, SINT, which only uses the original observation of the target from the first frame, suffices to reach state-of-theart performance Further, we show the proposed tracker even allows for target re-identification after the target was absent for a complete video shot

/pdf/siamese-instance-search-for-tracking-3dxbd1x0a6.pdf

Siamese Instance Search for Tracking

This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.

/pdf/a-novel-performance-evaluation-methodology-for-single-target-3lbt3wcee0.pdf

A Novel Performance Evaluation Methodology for Single-Target Trackers

The Visual Object Tracking challenge VOT2017 is the fifth annual tracker benchmarking activity organized by the VOT initiative. Results of 51 trackers are presented; many are state-of-the-art published at major computer vision conferences or journals in recent years. The evaluation included the standard VOT and other popular methodologies and a new "real-time" experiment simulating a situation where a tracker processes images as if provided by a continuously running sensor. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The VOT2017 goes beyond its predecessors by (i) improving the VOT public dataset and introducing a separate VOT2017 sequestered dataset, (ii) introducing a realtime tracking experiment and (iii) releasing a redesigned toolkit that supports complex experiments. The dataset, the evaluation kit and the results are publicly available at the challenge website1.

/pdf/the-visual-object-tracking-vot2017-challenge-results-x1xld7mnel.pdf

The Visual Object Tracking VOT2017 Challenge Results

The Visual Object Tracking challenge 2014, VOT2014, aims at comparing short-term single-object visual trackers that do not apply pre-learned models of object appearance. Results of 38 trackers are presented. The number of tested trackers makes VOT 2014 the largest benchmark on short-term tracking to date. For each participating tracker, a short description is provided in the appendix. Features of the VOT2014 challenge that go beyond its VOT2013 predecessor are introduced: (i) a new VOT2014 dataset with full annotation of targets by rotated bounding boxes and per-frame attribute, (ii) extensions of the VOT2013 evaluation methodology, (iii) a new unit for tracking speed assessment less dependent on the hardware and (iv) the VOT2014 evaluation toolkit that significantly speeds up execution of experiments. The dataset, the evaluation kit as well as the results are publicly available at the challenge website (http://votchallenge.net).

/pdf/the-visual-object-tracking-vot2014-challenge-results-122dmijtq7.pdf

The Visual Object Tracking VOT2014 challenge results

In this paper, we present an algorithm for fast calculation of the normalized cross correlation and its application to the problem of template matching. Given a template t, whose position is to be determined in an image f, the basic idea of the algorithm is to represent the template, for which the normalized cross correlation is calculated, as a sum of rectangular basis functions. Then the correlation is calculated for each basis function instead of the whole template. The result of the correlation of the template t and the image f is obtained as the weighted sum of the correlation functions of the basis functions. Depending on the approximation, the algorithm can by far outperform Fourier-transform based implementations of the normalized cross correlation algorithm and it is especially suited to problems, where many different templates are to be found in the same image f.

Template Matching using Fast Normalized Cross Correlation

This paper is concerned with recursively estimating the internal
state of a nonlinear dynamic system by processing noisy measurements and the known system input. In the case of continuous states, an exact analytic representation of the probability
density characterizing the estimate is generally too complex for
recursive estimation or even impossible to obtain. Hence, it is replaced by a convenient type of approximate density characterized by a finite set of parameters. Of course, parameters are desired that systematically minimize a given measure of deviation between
the (often unknown) exact density and its approximation, which in general leads to a complicated optimization problem. Here, a new framework for state estimation based on progressive processing is proposed. Rather than trying to solve the original problem, it is exactly converted into a corresponding system of explicit ordinary first-order differential equations. Solving this system over a finite "time" interval yields the desired optimal density
parameters.

Progressive Bayes: a new framework for nonlinear state estimation

In this paper, the problem of recursive robot localization based on relative bearing measurements is considered, where unknown but bounded measurement uncertainties are assumed. A common approach is to approximate the resulting set of feasible states by simple-shaped bounding sets such as, e.g., axis-aligned boxes, and calculate the optimal parameters of this approximation based on the measurements and prior knowledge. In the novel approach presented here, a nonlinear transformation of the measurement equation into a higher dimensional space is performed. This yields a tight, possibly complex-shaped, bounding set in a closed-form representation whose parameters can be determined analytically for the measurement step. It is shown that the new bound is superior to commonly used outer bounds.

Localization of a mobile robot using relative bearing measurements

This paper derives a fundamental result for processing two correlated random vectors with unknown cross-correlation, where constraints on the maximum absolute correlation coefficient are given. A tight upper bound for the joint covariance matrix is derived on the basis of the individual covariances and the correlation constraint. For symmetric constraints, the bounding covariance matrix naturally possesses zero cross covariances, which further increases their usefulness in applications. Performance is demonstrated by recursively propagating a state through a linear dynamical system suffering from stochastic noise correlated with the system state.

A tight bound for the joint covariance of two random vectors with unknown but constrained cross-correlation

This paper considers state estimation for dynamic systems in the case of non-white, mutually correlated noise processes. Here, the problem is complicated by the fact, that only the individual covariances are known; cross covariances between random variables obtained by taking individual noise processes at different time steps and between different noise processes are completely unknown. New estimator equations for solving this problem are derived in feedback form for both the prediction step and the filtering step based on existing ideas known as covariance intersection. Solutions are given for the most general case of updating an N-dimensional state vector estimate based on M-dimensional observations. Furthermore, computationally efficient solutions for obtaining minimum covariance estimates are derived to avoid numerical optimization otherwise required.

Kai Briechle

Papers

Template Matching using Fast Normalized Cross Correlation

Progressive Bayes: a new framework for nonlinear state estimation

Localization of a mobile robot using relative bearing measurements

A tight bound for the joint covariance of two random vectors with unknown but constrained cross-correlation

New results for stochastic prediction and filtering with unknown correlations