Person re-identification (re-ID) has become increasingly popular in the community due to its application and research significance. It aims at spotting a person of interest in other cameras. In the early days, hand-crafted algorithms and small-scale evaluation were predominantly reported. Recent years have witnessed the emergence of large-scale datasets and deep learning systems which make use of large data volumes. Considering different tasks, we classify most current re-ID methods into two classes, i.e., image-based and video-based; in both tasks, hand-crafted and deep learning systems will be reviewed. Moreover, two new re-ID tasks which are much closer to real-world applications are described and discussed, i.e., end-to-end re-ID and fast re-ID in very large galleries. This paper: 1) introduces the history of person re-ID and its relationship with image classification and instance retrieval; 2) surveys a broad selection of the hand-crafted systems and the large-scale methods in both image- and video-based re-ID; 3) describes critical future directions in end-to-end re-ID and fast retrieval in large galleries; and 4) finally briefs some important yet under-developed issues.

/pdf/person-re-identification-past-present-and-future-44l6xye7rx.pdf

Person Re-identification: Past, Present and Future

While deep convolutional neural networks (CNNs) have shown a great success in single-label image classification, it is important to note that real world images generally contain multiple labels, which could correspond to different objects, scenes, actions and attributes in an image. Traditional approaches to multi-label image classification learn independent classifiers for each category and employ ranking or thresholding on the classification results. These techniques, although working well, fail to explicitly exploit the label dependencies in an image. In this paper, we utilize recurrent neural networks (RNNs) to address this problem. Combined with CNNs, the proposed CNN-RNN framework learns a joint image-label embedding to characterize the semantic label dependency as well as the image-label relevance, and it can be trained end-to-end from scratch to integrate both information in a unified framework. Experimental results on public benchmark datasets demonstrate that the proposed architecture achieves better performance than the state-of-the-art multi-label classification models.

/pdf/cnn-rnn-a-unified-framework-for-multi-label-image-4ckssxbf6c.pdf

CNN-RNN: A Unified Framework for Multi-label Image Classification

Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure- and property-preserving network embedding methods, the network embedding methods with side information, and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions.

A Survey on Network Embedding

Hashing-based approximate nearest neighbor (ANN) search in huge databases has become popular due to its computational and memory efficiency. The popular hashing methods, e.g., Locality Sensitive Hashing and Spectral Hashing, construct hash functions based on random or principal projections. The resulting hashes are either not very accurate or are inefficient. Moreover, these methods are designed for a given metric similarity. On the contrary, semantic similarity is usually given in terms of pairwise labels of samples. There exist supervised hashing methods that can handle such semantic similarity, but they are prone to overfitting when labeled data are small or noisy. In this work, we propose a semi-supervised hashing (SSH) framework that minimizes empirical error over the labeled set and an information theoretic regularizer over both labeled and unlabeled sets. Based on this framework, we present three different semi-supervised hashing methods, including orthogonal hashing, nonorthogonal hashing, and sequential hashing. Particularly, the sequential hashing method generates robust codes in which each hash function is designed to correct the errors made by the previous ones. We further show that the sequential learning paradigm can be extended to unsupervised domains where no labeled pairs are available. Extensive experiments on four large datasets (up to 80 million samples) demonstrate the superior performance of the proposed SSH methods over state-of-the-art supervised and unsupervised hashing techniques.

/pdf/semi-supervised-hashing-for-large-scale-search-1g67bshmw7.pdf

Semi-Supervised Hashing for Large-Scale Search

In this paper, we present a new hashing method to learn compact binary codes for highly efficient image retrieval on large-scale datasets. While the complex image appearance variations still pose a great challenge to reliable retrieval, in light of the recent progress of Convolutional Neural Networks (CNNs) in learning robust image representation on various vision tasks, this paper proposes a novel Deep Supervised Hashing (DSH) method to learn compact similarity-preserving binary code for the huge body of image data. Specifically, we devise a CNN architecture that takes pairs of images (similar/dissimilar) as training inputs and encourages the output of each image to approximate discrete values (e.g. +1/-1). To this end, a loss function is elaborately designed to maximize the discriminability of the output space by encoding the supervised information from the input image pairs, and simultaneously imposing regularization on the real-valued outputs to approximate the desired discrete values. For image retrieval, new-coming query images can be easily encoded by propagating through the network and then quantizing the network outputs to binary codes representation. Extensive experiments on two large scale datasets CIFAR-10 and NUS-WIDE show the promising performance of our method compared with the state-of-the-arts.

/pdf/deep-supervised-hashing-for-fast-image-retrieval-rhxxzyenwu.pdf

Deep Supervised Hashing for Fast Image Retrieval

This paper introduces a web image dataset created by NUS's Lab for Media Search. The dataset includes: (1) 269,648 images and the associated tags from Flickr, with a total of 5,018 unique tags; (2) six types of low-level features extracted from these images, including 64-D color histogram, 144-D color correlogram, 73-D edge direction histogram, 128-D wavelet texture, 225-D block-wise color moments extracted over 5x5 fixed grid partitions, and 500-D bag of words based on SIFT descriptions; and (3) ground-truth for 81 concepts that can be used for evaluation. Based on this dataset, we highlight characteristics of Web image collections and identify four research issues on web image annotation and retrieval. We also provide the baseline results for web image annotation by learning from the tags using the traditional k-NN algorithm. The benchmark results indicate that it is possible to learn effective models from sufficiently large image dataset to facilitate general image retrieval.

NUS-WIDE: a real-world web image database from National University of Singapore

This paper presents a novel system to explore places of interests based on the large amount of photos that are placed on Google Maps. The system, named ViewFocus, estimates the view directions of photos via robust object matching and camera reconstruction techniques, and geo-registers the directions on the map. Thus users are able to select the places they are interested in, and the system automatically returns a set of precise photos of the target places for users to focus their exploration, by filtering out photos that are pointing to other directions.

ViewFocus: explore places of interests on Google maps using photos with view direction filtering

In recent years, we have witnessed the blooming of Web 2.0 content such as Wikipedia, Flickr and YouTube, etc. How might we benefit from such rich media resources available on the internet? This paper presents a novel concept called Mediapedia, a dynamic multimedia encyclopedia that takes advantage of, and in fact is built from the text and image resources on the Web. The Mediapedia distinguishes itself from the traditional encyclopedia in four main ways. (1) It tries to present users with multimedia contents (e.g., text, image, video) which we believed are more intuitive and informative to users. (2) It is fully automated because it downloads the media contents as well as the corresponding textual descriptions from the Web and assembles them for presentation. (3) It is dynamic as it will use the latest multimedia content to compose the answer. This is not true for the traditional encyclopedia. (4) The design of Mediapedia is flexible and extensible such that we can easily incorporate new kinds of mediums such as video and languages into the framework. The effectiveness of Mediapedia is demonstrated and two potential applications are described in this paper.

Mediapedia: mining web knowledge to construct multimedia encyclopedia

This paper presents a lightweight tele-immersive video chat system named CuteChat. Based on our recently developed video object cutout technology, the CuteChat system is designed and optimized to provide a radically new video chat experience by merging each participant in the same shared space, allowing them to interact more naturally in an integrated manner. With the goal to make the system easily accessible by massive consumers, we address the challenges in the whole pipeline of video processing, coding, communication, composition, and playback. Extensive experiments have shown that the proposed CuteChat system runs reliably and comfortably in real time on one's laptop or desktop PC, and it needs only a commodity webcam for video acquisition and just public Internet for tele-immersive video conferencing. With such a really minimal deployment requirement, we present a variety of interesting applications and user experiences created by the CuteChat system.

CuteChat: a lightweight tele-immersive video chat system

This paper proposes a novel algorithm, named Pseudo-inverse Locality Preserving Projections (PLPP), for dimensionality reduction involving undersampled problems. This algorithm considers the matrix singularity caused by undersampled problems by substituting the Moore-Penrose pseudo-inverse for the inverse of the matrix. Under the pseudo-inverse form eigenequation, the optimal locality preserving projections can be found by using the simultaneous diagonalization of three matrices technique, which intuitively solves the generalized eigenvalue decomposition problem. Theoretical analysis shows the flexible time complexity and better locality preserving power of PLPP. We compare the proposed PLPP with PCA, PCA+LDA, PCA+LPP on ORL face database and 20-Newsgroups text data sets. Experimental results show the effectiveness of the proposed algorithm.

Zhiping Luo

Papers

NUS-WIDE: a real-world web image database from National University of Singapore

ViewFocus: explore places of interests on Google maps using photos with view direction filtering

Mediapedia: mining web knowledge to construct multimedia encyclopedia

CuteChat: a lightweight tele-immersive video chat system

Pseudo-inverse Locality Preserving Projections