Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Detection of visually salient image regions is useful for applications like object segmentation, adaptive compression, and object recognition. In this paper, we introduce a method for salient region detection that outputs full resolution saliency maps with well-defined boundaries of salient objects. These boundaries are preserved by retaining substantially more frequency content from the original image than other existing techniques. Our method exploits features of color and luminance, is simple to implement, and is computationally efficient. We compare our algorithm to five state-of-the-art salient region detection methods with a frequency domain analysis, ground truth, and a salient object segmentation application. Our method outperforms the five algorithms both on the ground-truth evaluation and on the segmentation task by achieving both higher precision and better recall.

/pdf/frequency-tuned-salient-region-detection-4po18ylvds.pdf

Frequency-tuned salient region detection

Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contents of the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrast based salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as a more challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existing salient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithm can be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) via simple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided image retrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important target object region information.

https://mmcheng.net/mftp/SalObj/PosterSaliency.pdf

Global contrast based salient region detection

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Video data are of two different intrinsic modes, in-frame and temporal. It is beneficial to incorporate static in-frame features to acquire dynamic features for video applications. However, some existing methods such as recurrent neural networks do not have a good performance, and some other such as 3D convolutional neural networks (CNNs) are both memory consuming and time consuming. This study proposes an effective framework that takes the advantage of deep learning on the static image feature extraction to tackle the video data. After extracting in-frame feature vectors using a pretrained deep network, the authors integrate them and form a multi-mode feature matrix, which preserves the multi-mode structure and high-level representation. They propose two models for follow-up classification. The authors first introduce a temporal CNN, which directly feeds the multi-mode feature matrix into a CNN. However, they show that characteristics of the multi-mode features differ significantly in distinct modes. The authors therefore further propose the multi-mode neural network (MMNN), in which different modes deploy different types of layers. They evaluate their algorithm with the task of human action recognition. The experimental results show that the MMNN achieves a much better performance than the existing long short-term memory-based methods and consumes far fewer resources than the existing 3D end-to-end models.

https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/iet-cvi.2019.0761

Multi-mode neural network for human action recognition

The aim of Content-based Image Retrieval (CBIR) is to find a set of images that best match the query based on visual features. Most existing CBIR systems find similar images in low level features, while Text-based Image Retrieval (TBIR) systems find images with relevant tags regardless of contents in the images. Generally, people are more interested in images with similarity both in contours and high-level concepts. Therefore, we propose a new strategy called Iterative Search to meet this requirement. It mines knowledge from the similar images of original queries, in order to compensate for the missing information in feature extraction process. To evaluate the performance of Iterative Search approach, we apply this method to four different CBIR systems (HOF Zhou et al. in ACM international conference on multimedia, 2012; Zhou and Zhang in Neural information processing—international conference, ICONIP 2011, Shanghai, 2011, HOG Dalal and Triggs in IEEE computer society conference on computer vision pattern recognition, 2005, GIST Oliva and Torralba in Int J Comput Vision 42:145–175, 2001 and CNN Krizhevsky et al. in Adv Neural Inf Process Syst 25:2012, 2012) in our experiments. The results show that Iterative Search improves the performance of original CBIR features by about $$20\%$$
 on both the Oxford Buildings dataset and the Object Sketches dataset. Meanwhile, it is not restricted to any particular visual features.

Content-Based Image Retrieval Using Iterative Search

We propose a framework for multi-age group gait video translation in which, for the first time, individuality-preserving aging patterns in walking style are learnt. More specifically, we build our framework on an existing multi-domain image translation model. Because the existing multi-domain image translation model was originally designed for a still image, we extend it to gait video by introducing a motion-augmented network architecture with three streams, where gait period, period-normalized phase-synchronized gait video, and its frame difference sequence are each input to one stream. We then train the network to ensure three aspects: aging effect (using an age group classification loss), individuality preservation (using a reconstruction loss), and gait realism (using an adversarial loss). Our framework quantitatively and qualitatively outperforms state-of-the-art age progression/regression methods on the largest gait database, OULP-Age, with respect to both age group classification and identity recognition.

/pdf/learn-to-walk-across-ages-motion-augmented-multi-age-group-425imahe13.pdf

Learn to Walk Across Ages: Motion Augmented Multi-Age Group Gait Video Translation

This paper deals with automatic human action recognition in videos. Rather than considering traditional hand-craft features such as HOG, HOF and MBH, we explore how to learn both static and motion features from CNNs trained on large-scale datasets such as ImagNet and UCF101. We propose a novel method named multi-resolution latent concept descriptor (mLCD) to encode two-stream CNNs. Entensive experiments are conducted to demonstrate the performance of the proposed model. By combining our mLCD features with the improved dense trajectory features, we can achieve comparable performance with state-of-the-art algorithms on both Hollywood2 and Olympic Sports datasets.

Liqing Zhang

Papers

Multi-mode neural network for human action recognition

Content-Based Image Retrieval Using Iterative Search

Learn to Walk Across Ages: Motion Augmented Multi-Age Group Gait Video Translation

Encoding Multi-resolution Two-Stream CNNs for Action Recognition

Assessment of mediators in the associations between urban green spaces and self-reported health