RGB-T object tracking: Benchmark and baseline

RGB-Thermal (RGB-T) object tracking receives more and more attention due to the strongly complementary benefits of thermal information to visible data. However, RGB-T research is limited by lacking a comprehensive evaluation platform. In this paper, we propose a large-scale video benchmark dataset for RGB-T this http URL has three major advantages over existing ones: 1) Its size is sufficiently large for large-scale performance evaluation (total frame number: 234K, maximum frame per sequence: 8K). 2) The alignment between RGB-T sequence pairs is highly accurate, which does not need pre- or post-processing. 3) The occlusion levels are annotated for occlusion-sensitive performance analysis of different tracking algorithms.Moreover, we propose a novel graph-based approach to learn a robust object representation for RGB-T tracking. In particular, the tracked object is represented with a graph with image patches as nodes. This graph including graph structure, node weights and edge weights is dynamically learned in a unified ADMM (alternating direction method of multipliers)-based optimization framework, in which the modality weights are also incorporated for adaptive fusion of multiple source data.Extensive experiments on the large-scale dataset are executed to demonstrate the effectiveness of the proposed tracker against other state-of-the-art tracking methods. We also provide new insights and potential research directions to the field of RGB-T object tracking.

RGB-T Object Tracking:Benchmark and Baseline

Multiple cloud storage mechanism based on blockchain in smart homes

Existing deep Thermal InfraRed (TIR) trackers only use semantic features to represent the TIR object, which lack the sufficient discriminative capacity for handling distractors. This becomes worse when the feature extraction network is only trained on RGB images. To address this issue, we propose a multi-level similarity model under a Siamese framework for robust TIR object tracking. Specifically, we compute different pattern similarities using the proposed multi-level similarity network. One of them focuses on the global semantic similarity and the other computes the local structural similarity of the TIR object. These two similarities complement each other and hence enhance the discriminative capacity of the network for handling distractors. In addition, we design a simple while effective relative entropy based ensemble subnetwork to integrate the semantic and structural similarities. This subnetwork can adaptive learn the weights of the semantic and structural similarities at the training stage. To further enhance the discriminative capacity of the tracker, we propose a large-scale TIR video sequence dataset for training the proposed model. To the best of our knowledge, this is the first and the largest TIR object tracking training dataset to date. The proposed TIR dataset not only benefits the training for TIR object tracking but also can be applied to numerous TIR visual tasks. Extensive experimental results on three benchmarks demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

Learning Deep Multi-Level Similarity for Thermal Infrared Object Tracking

Application of the best evacuation model of deep learning in the design of public structures

Visual tracking using spatio-temporally nonlocally regularized correlation filter

The invention discloses an unsupervised video segmentation method integrated with temporal-spatial multi-feature representation. The features of a target are extracted and identified according to themotion information and saliency and color features of the target, and the target is segmented stably and accurately through a Gaussian mixture model. The method includes the following steps: super pixel segmentation; optical flow matching; optimizing the matching result; building a graph model, and calculating the super pixel-level segmentation result; using the segmentation result to train the parameters of a Gaussian mixture model; calculating the pixel-level segmentation result; and getting a final segmentation result according to the super pixel-level segmentation result and the pixel-level segmentation result. Through super pixel segmentation on each frame of image, the computational complexity is greatly reduced. The robustness of segmentation is improved by optimizing the optical flow matching information according to non-local temporal-spatial information. The introduction of the Gaussian mixture model makes up for the large edge matching error in the process of super pixel segmentation. The saliency feature further improves the accuracy and credibility of the segmentation result.

Unsupervised video segmentation method integrated with temporal-spatial multi-feature representation

Video object segmentation is challenging due to the factors like rapidly fast motion, cluttered backgrounds, arbitrary object appearance variation and shape deformation. Most existing methods only explore appearance information between two consecutive frames, which do not make full use of the usefully long-term nonlocal information that is helpful to make the learned appearance stable, and hence they tend to fail when the targets suffer from large viewpoint changes and significant non-rigid deformations. In this paper, we propose a simple yet effective approach to mine the long-term sptatio-temporally nonlocal appearance information for unsupervised video segmentation. The motivation of our algorithm comes from the spatio-temporal nonlocality of the region appearance reoccurrence in a video. Specifically, we first generate a set of superpixels to represent the foreground and background, and then update the appearance of each superpixel with its long-term sptatio-temporally nonlocal counterparts generated by the approximate nearest neighbor search method with the efficient KD-tree algorithm. Then, with the updated appearances, we formulate a spatio-temporal graphical model comprised of the superpixel label consistency potentials. Finally, we generate the segmentation by optimizing the graphical model via iteratively updating the appearance model and estimating the labels. Extensive evaluations on the SegTrack and Youtube-Objects datasets demonstrate the effectiveness of the proposed method, which performs favorably against some state-of-art methods.

Unsupervised Video Segmentation via Spatio-Temporally Nonlocal Appearance Learning

The invention discloses an unsupervised video segmentation method based on non-local space-time characteristic learning. The method comprises the following steps of acquiring a video sequence which needs to be segmented; using a superpixel to segment and process the video sequence; using a light stream to carry out previous and next frame information matching; according to adjacent frame information of the video sequence, acquiring a range of a motion target and taking as model initialization input; using global information to optimize a matching result; and establishing a graph model and using a graph segmentation algorithm to solve a segmentation result, and through video segmentation, acquiring output of the motion target. Through carrying out superpixel segmentation on each frame of image in an input video, an operation complexity can be greatly reduced; and non-local space-time information is used to optimize matching information acquired through the light stream so that segmentation robustness can be increased and a noise influence is reduced. Any manual intervention is not needed, and an accurate segmentation result can be acquired completely based on video image information.

Li Xuejun

Papers

Visual tracking using spatio-temporally nonlocally regularized correlation filter

Unsupervised video segmentation method integrated with temporal-spatial multi-feature representation

Unsupervised Video Segmentation via Spatio-Temporally Nonlocal Appearance Learning

Unsupervised video segmentation method based on non-local space-time characteristic learning