Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations, 20 000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.

https://www.cityscapes-dataset.com/wordpress/wp-content/papercite-data/pdf/cordts2016cityscapes.pdf

The Cityscapes Dataset for Semantic Urban Scene Understanding

Images containing faces are essential to intelligent vision-based human-computer interaction, and research efforts in face processing include face recognition, face tracking, pose estimation and expression recognition. However, many reported methods assume that the faces in an image or an image sequence have been identified and localized. To build fully automated systems that analyze the information contained in face images, robust and efficient face detection algorithms are required. Given a single image, the goal of face detection is to identify all image regions which contain a face, regardless of its 3D position, orientation and lighting conditions. Such a problem is challenging because faces are non-rigid and have a high degree of variability in size, shape, color and texture. Numerous techniques have been developed to detect faces in a single image, and the purpose of this paper is to categorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics and benchmarking. After analyzing these algorithms and identifying their limitations, we conclude with several promising directions for future research.

/pdf/detecting-faces-in-images-a-survey-4yhbbns6mr.pdf

Detecting faces in images: a survey

In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research. Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction. Adaptive Computation and Machine Learning series

/pdf/semi-supervised-learning-scg1exixez.pdf

Semi-Supervised Learning

The PASCAL Visual Object Classes Challenge

Digital watermarking is a key ingredient to copyright protection. It provides a solution to illegal copying of digital material and has many other useful applications such as broadcast monitoring and the recording of electronic transactions. Now, for the first time, there is a book that focuses exclusively on this exciting technology. Digital Watermarking covers the crucial research findings in the field: it explains the principles underlying digital watermarking technologies, describes the requirements that have given rise to them, and discusses the diverse ends to which these technologies are being applied. As a result, additional groundwork is laid for future developments in this field, helping the reader understand and anticipate new approaches and applications.

Digital Watermarking

In this paper, we present two watermarking approaches that are robust to geometric distortions. The first approach is based on image normalization, in which both watermark embedding and extraction are carried out with respect to an image normalized to meet a set of predefined moment criteria. We propose a new normalization procedure, which is invariant to affine transform attacks. The resulting watermarking scheme is suitable for public watermarking applications, where the original image is not available for watermark extraction. The second approach is based on a watermark resynchronization scheme aimed to alleviate the effects of random bending attacks. In this scheme, a deformable mesh is used to correct the distortion caused by the attack. The watermark is then extracted from the corrected image. In contrast to the first scheme, the latter is suitable for private watermarking applications, where the original image is necessary for watermark detection. In both schemes, we employ a direct-sequence code division multiple access approach to embed a multibit watermark in the discrete cosine transform domain of the image. Numerical experiments demonstrate that the proposed watermarking schemes are robust to a wide range of geometric attacks.

/pdf/digital-watermarking-robust-to-geometric-distortions-4wryq5g9h9.pdf

Digital watermarking robust to geometric distortions

In inductive transfer learning, fine-tuning pre-trained convolutional networks substantially outperforms training from scratch When using fine-tuning, the underlying assumption is that the pre-trained model extracts generic features, which are at least partially relevant for solving the target task, but would be difficult to extract from the limited amount of data available on the target task However, besides the initialization with the pre-trained model and the early stopping, there is no mechanism in fine-tuning for retaining the features learned on the source task In this paper, we investigate several regularization schemes that explicitly promote the similarity of the final solution with the initial model We show the benefit of having an explicit inductive bias towards the initial model, and we eventually recommend a simple $L^2$ penalty with the pre-trained model being a reference as the baseline of penalty for transfer learning tasks

Explicit Inductive Bias for Transfer Learning with Convolutional Networks

This article addresses the issue of expressive face modelling using an active appearance model for facial expression recognition and synthesis. We consider the six universal emotional categories namely joy, anger, fear, disgust, sadness and surprise. After a description of the active appearance model (computed with 3 or only one PCA), we address the active appearance model contribution to automatic facial expression recognition. Then we propose a new method for analysis and synthesis allowing, from a single photo, to cancel the facial expression on a given face and to artificially synthesize novel expressions on this same face. In this last framework, we propose two facial expression modelling approaches.

Facial expression recognition and synthesis based on an appearance model

The design of a recognition system requires careful attention to pattern representation and classifier design. Some statistical approaches choose those features, in a d-dimensional initial space, which allow sample vectors belonging to different categories to occupy compact and disjoint regions in a low-dimensional subspace. The effectiveness of the representation subspace is then determined by how well samples from different classes can be separated. In this paper, we propose a feature selection process that sorts the principal components, generated by principal component analysis, in the order of their importance to solve a specific recognition task. This method provides a low-dimensional representation subspace which has been optimized to improve the classification accuracy. We focus on the problem of facial expression recognition to demonstrate this technique. We also propose a decision tree-based classifier that provides a ‘‘coarse-to-fine’’ classification of new samples by successive projections onto more and more precise representation subspaces. Results confirm, first, that the choice of the representation strongly influences the classification results, second that a classifier has to be designed for a specific representation. r 2002 Elsevier Science B.V. All rights reserved.

A solution for facial expression representation and recognition

Presents a new scheme for fractal image compression based on adaptive Delaunay triangulation. Such a partition is computed on an initial set of points obtained with a split and merge algorithm in a grey level dependent way. The triangulation is thus fully flexible and returns a limited number of blocks allowing good compression ratios. Moreover, a second original approach is the integration of a classification step based on a modified version of the Lloyd algorithm (vector quantization) in order to reduce the encoding complexity. The vector quantization algorithm is implemented on pixel histograms directly generated from the triangulation. The aim is to reduce the number of comparisons between the two sets of blocks involved in fractal image compression by keeping only the best representative triangles in the domain blocks set. Quality coding results are achieved at rates between 0.25-0.5 b/pixel depending on the nature of the original image and on the number of triangles retained.

Franck Davoine

Papers

Digital watermarking robust to geometric distortions

Explicit Inductive Bias for Transfer Learning with Convolutional Networks

Facial expression recognition and synthesis based on an appearance model

A solution for facial expression representation and recognition

Fractal image compression based on Delaunay triangulation and vector quantization