In the past few years, the field of computer vision has gone through a revolution fueled mainly by the advent of large datasets and the adoption of deep convolutional neural networks for end-to-end learning. The person re-identification subfield is no exception to this. Unfortunately, a prevailing belief in the community seems to be that the triplet loss is inferior to using surrogate losses (classification, verification) followed by a separate metric learning step. We show that, for models trained from scratch as well as pretrained ones, using a variant of the triplet loss to perform end-to-end deep metric learning outperforms most other published methods by a large margin.

In Defense of the Triplet Loss for Person Re-Identification

In this paper, we introduce Random Erasing, a new data augmentation method for training the convolutional neural network (CNN). In training, Random Erasing randomly selects a rectangle region in an image and erases its pixels with random values. In this process, training images with various levels of occlusion are generated, which reduces the risk of over-fitting and makes the model robust to occlusion. Random Erasing is parameter learning free, easy to implement, and can be integrated with most of the CNN-based recognition models. Albeit simple, Random Erasing is complementary to commonly used data augmentation techniques such as random cropping and flipping, and yields consistent improvement over strong baselines in image classification, object detection and person re-identification. Code is available at: https://github.com/zhunzhong07/Random-Erasing.

/pdf/random-erasing-data-augmentation-3v1mnwjsv7.pdf

Random Erasing Data Augmentation

Employing part-level features offers fine-grained information for pedestrian image description. A prerequisite of part discovery is that each part should be well located. Instead of using external resources like pose estimator, we consider content consistency within each part for precise part location. Specifically, we target at learning discriminative part-informed features for person retrieval and make two contributions. (i) A network named Part-based Convolutional Baseline (PCB). Given an image input, it outputs a convolutional descriptor consisting of several part-level features. With a uniform partition strategy, PCB achieves competitive results with the state-of-the-art methods, proving itself as a strong convolutional baseline for person retrieval. (ii) A refined part pooling (RPP) method. Uniform partition inevitably incurs outliers in each part, which are in fact more similar to other parts. RPP re-assigns these outliers to the parts they are closest to, resulting in refined parts with enhanced within-part consistency. Experiment confirms that RPP allows PCB to gain another round of performance boost. For instance, on the Market-1501 dataset, we achieve (77.4+4.2)% mAP and (92.3+1.5)% rank-1 accuracy, surpassing the state of the art by a large margin. Code is available at: https://github.com/syfafterzy/PCB_RPP

/pdf/beyond-part-models-person-retrieval-with-refined-part-46sgh30bbv.pdf

Beyond Part Models: Person Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)

Model distillation is an effective and widely used technique to transfer knowledge from a teacher to a student network The typical application is to transfer from a powerful large network or ensemble to a small network, in order to meet the low-memory or fast execution requirements In this paper, we present a deep mutual learning (DML) strategy Different from the one-way transfer between a static pre-defined teacher and a student in model distillation, with DML, an ensemble of students learn collaboratively and teach each other throughout the training process Our experiments show that a variety of network architectures benefit from mutual learning and achieve compelling results on both category and instance recognition tasks Surprisingly, it is revealed that no prior powerful teacher network is necessary - mutual learning of a collection of simple student networks works, and moreover outperforms distillation from a more powerful yet static teacher

Deep Mutual Learning

The combination of global and partial features has been an essential solution to improve discriminative performances in person re-identification (Re-ID) tasks. Previous part-based methods mainly focus on locating regions with specific pre-defined semantics to learn local representations, which increases learning difficulty but not efficient or robust to scenarios with large variances. In this paper, we propose an end-to-end feature learning strategy integrating discriminative information with various granularities. We carefully design the Multiple Granularity Network (MGN), a multi-branch deep network architecture consisting of one branch for global feature representations and two branches for local feature representations. Instead of learning on semantic regions, we uniformly partition the images into several stripes, and vary the number of parts in different local branches to obtain local feature representations with multiple granularities. Comprehensive experiments implemented on the mainstream evaluation datasets including Market-1501, DukeMTMC-reid and CUHK03 indicate that our method robustly achieves state-of-the-art performances and outperforms any existing approaches by a large margin. For example, on Market-1501 dataset in single query mode, we obtain a top result of Rank-1/mAP=96.6%/94.2% with this method after re-ranking.

Learning Discriminative Features with Multiple Granularities for Person Re-Identification

When considering person re-identification (re-ID) as a retrieval process, re-ranking is a critical step to improve its accuracy. Yet in the re-ID community, limited effort has been devoted to re-ranking, especially those fully automatic, unsupervised solutions. In this paper, we propose a k-reciprocal encoding method to re-rank the re-ID results. Our hypothesis is that if a gallery image is similar to the probe in the k-reciprocal nearest neighbors, it is more likely to be a true match. Specifically, given an image, a k-reciprocal feature is calculated by encoding its k-reciprocal nearest neighbors into a single vector, which is used for re-ranking under the Jaccard distance. The final distance is computed as the combination of the original distance and the Jaccard distance. Our re-ranking method does not require any human interaction or any labeled data, so it is applicable to large-scale datasets. Experiments on the large-scale Market-1501, CUHK03, MARS, and PRW datasets confirm the effectiveness of our method.

/pdf/re-ranking-person-re-identification-with-k-reciprocal-3erzmilgv1.pdf

Re-ranking Person Re-identification with k-Reciprocal Encoding

Re-ranking Person Re-identification with k-reciprocal Encoding

In multi-label learning, different labels may have their own inherent characteristics for distinguishing each other, in the meanwhile, exploiting the correlations among labels is another practical yet challenging task to improve the performance. In this work, we present a new method for the joint learning of label-specific features and label correlations. The key is the design of an optimization framework to learn the weight assignment scheme of features, and the correlations among labels are taken into account by constructing additional features at the same time. Through iteratively optimizing the two sets of unknown variables, which are referred to feature weights and label correlations-based features, label-specific features of each label are available to achieve multi-label classification. Comprehensive experiments on various multi-label data sets including two collected traditional Chinese medicine data sets reveal the advantages of our proposed algorithm.

Multi-label learning with label-specific features by resolving label correlations

Sentiment analysis on social networks has attracted ever increasing attention in recent years. To this end, most existing methods mainly focus on analyzing the textual information. This has faced huge difficulty as nowadays users are more likely to express their feelings in a hybrid manner not only with texts, but also with images. It is therefore essential to take images into account. In this paper, we propose a novel Cross-media Bag-of-words Model (CBM) for Microblog sentiment analysis. In this model, we represent the text and image of a Weibo tweet as a unified Bag-of-words representation. Based on this model, we use Logistic Regression to classify the Microblog sentiment. It performs well in the sentiment classification task since it doesn't require the conditional dependence assumption. We also use SVM and Naive Bayes to make a comparison. Experiments on 5,000 Microblog messages demonstrate that our CBM model performs better than text-based methods. The sentiment classification accuracy on Microblog messages of our model is 80%, improved by 4% than the text-based methods.

Microblog Sentiment Analysis Based on Cross-media Bag-of-words Model

Predicting sentiments of multimodal microblogs composed of text, image, and emoticon have attracted ever-increasing research focus recently The key challenge lies in the difficulty of collecting a sufficient amount of training labels to train a discriminative model for multimodal prediction One potential solution is to exploit the labels collected from social media users, which is, however, restricted by the negative effect of label noise Besides, we have quantitatively found that sentiments in different modalities may be independent, which disables the usage of previous multimodal sentiment analysis schemes in our problem In this paper, we introduce a weakly supervised multimodal deep learning (WS-MDL) scheme toward robust and scalable sentiment prediction WS-MDL learns convolutional neural networks iteratively and selectively from “weak” emoticon labels, which are cheaply available and noise containing In particular, to filter out the label noise and to capture the modality dependency, a probabilistic graphical model is introduced to simultaneously learn discriminative multimodal descriptors and infer the confidence of label noise Extensive evaluations are conducted in a million-scale, real-world microblog sentiment dataset crawled from Sina Weibo We have validated the merits of the proposed scheme by quantitatively showing its superior performance over several state-of-the-art and alternative approaches

Donglin Cao

Papers

Re-ranking Person Re-identification with k-Reciprocal Encoding

Re-ranking Person Re-identification with k-reciprocal Encoding

Multi-label learning with label-specific features by resolving label correlations

Microblog Sentiment Analysis Based on Cross-media Bag-of-words Model

Predicting Microblog Sentiments via Weakly Supervised Multimodal Deep Learning