“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

A morphable model for the synthesis of 3D faces

Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging. We explore the central prevailing themes of this emerging area and present a taxonomy that can be used to categorize different problems and reconstruction methods. Our taxonomy is organized along two central axes: (1) whether or not a forward model is known and to what extent it is used in training and testing, and (2) whether or not the learning is supervised or unsupervised, i.e., whether or not the training relies on access to matched ground truth image and measurement pairs. We also discuss the tradeoffs associated with these different reconstruction approaches, caveats and common failure modes, plus open problems and avenues for future work.

Deep Learning Techniques for Inverse Problems in Imaging

Generative adversarial networks (GANs) are a hot research topic recently. GANs have been widely studied since 2014, and a large number of algorithms have been proposed. However, there is few comprehensive study explaining the connections among different GANs variants, and how they have evolved. In this paper, we attempt to provide a review on various GANs methods from the perspectives of algorithms, theory, and applications. Firstly, the motivations, mathematical representations, and structure of most GANs algorithms are introduced in details. Furthermore, GANs have been combined with other machine learning algorithms for specific applications, such as semi-supervised learning, transfer learning, and reinforcement learning. This paper compares the commonalities and differences of these GANs methods. Secondly, theoretical issues related to GANs are investigated. Thirdly, typical applications of GANs in image processing and computer vision, natural language processing, music, speech and audio, medical field, and data science are illustrated. Finally, the future open research problems for GANs are pointed out.

A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications

This paper reviews the first challenge on image dehazing (restoration of rich details in hazy image) with focus on proposed solutions and results. The challenge had 2 tracks. Track 1 employed the indoor images (using I-HAZE dataset), while Track 2 outdoor images (using O-HAZE dataset). The hazy images have been captured in presence of real haze, generated by professional haze machines. I-HAZE dataset contains 35 scenes that correspond to indoor domestic environments, with objects with different colors and specularities. O-HAZE contains 45 different outdoor scenes depicting the same visual content recorded in haze-free and hazy conditions, under the same illumination parameters. The dehazing process was learnable through provided pairs of haze-free and hazy train images. Each track had ~ 120 registered participants and 21 teams competed in the final testing phase. They gauge the state-of-the-art in image dehazing.

/pdf/ntire-2018-challenge-on-image-dehazing-methods-and-results-2auhtl0o6b.pdf

NTIRE 2018 Challenge on Image Dehazing: Methods and Results

In this paper, we present an end-to-end network, called Cycle-Dehaze, for single image dehazing problem, which does not require pairs of hazy and corresponding ground truth images for training. That is, we train the network by feeding clean and hazy images in an unpaired manner. Moreover, the proposed approach does not rely on estimation of the atmospheric scattering model parameters. Our method enhances CycleGAN formulation by combining cycle-consistency and perceptual losses in order to improve the quality of textural information recovery and generate visually better haze-free images. Typically, deep learning models for dehazing take low resolution images as input and produce low resolution outputs. However, in the NTIRE 2018 challenge on single image dehazing, high resolution images were provided. Therefore, we apply bicubic downscaling. After obtaining low-resolution outputs from the network, we utilize the Laplacian pyramid to upscale the output images to the original resolution. We conduct experiments on NYU-Depth, I-HAZE, and O-HAZE datasets. Extensive experiments demonstrate that the proposed approach improves CycleGAN method both quantitatively and qualitatively.

/pdf/cycle-dehaze-enhanced-cyclegan-for-single-image-dehazing-37omclvlbm.pdf

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing.

In this paper, we have explored the effect of pose normalization for cross-pose facial expression recognition. We have first presented an expression preserving face frontalization method. After face frontalization step, for facial expression representation and classification, we have employed both a traditional approach, by using hand-crafted features, namely local binary patterns, in combination with support vector machine classification and a relatively more recent approach based on convolutional neural networks. To evaluate the impact of face frontalization on facial expression recognition performance, we have conducted cross-pose, subject-independent expression recognition experiments using the BU3DFE database. Experimental results show that pose normalization improves the performance for cross-pose facial expression recognition. Especially, when local binary patterns in combination with support vector machine classifier is used, since this facial expression representation and classification does not handle pose variations, the obtained performance increase is significant. Convolutional neural networks-based approach is found to be more successful handling pose variations, when it is fine-tuned on a dataset that contains face images with varying pose angles. Its performance is further enhanced by benefiting from face frontalization.

Face Frontalization for Cross-Pose Facial Expression Recognition

Research on offline signature verification has explored a large variety of methods on multiple signature datasets, which are collected under controlled conditions. However, these datasets may not fully reflect the characteristics of the signatures in some practical use cases. Real-world signatures extracted from the formal documents may contain different types of occlusions, for example, stamps, company seals, ruling lines, and signature boxes. Moreover, they may have very high intra-class variations, where even genuine signatures resemble forgeries. In this paper, we address a real-world writer independent offline signature verification problem, in which, a bank's customers' transaction request documents that contain their occluded signatures are compared with their clean reference signatures. Our proposed method consists of two main components, a stamp cleaning method based on CycleGAN and signature representation based on CNNs. We extensively evaluate different verification setups, fine-tuning strategies, and signature representation approaches to have a thorough analysis of the problem. Moreover, we conduct a human evaluation to show the challenging nature of the problem. We run experiments both on our custom dataset, as well as on the publicly available Tobacco-800 dataset. The experimental results validate the difficulty of offline signature verification on real-world documents. However, by employing the stamp cleaning process, we improve the signature verification performance significantly.

Offline Signature Verification on Real-World Documents

To be able to use supervised machine learning methods in natural language processing, there is a need of labeled data in large quantities. In some cases, especially when there are multiple tasks conducted on the same data, the annotation process may become exhausting and time consuming for both the annotatore and interpreters. Thus, an effective annotation tool becomes crucial in order to both increase the annotation quality and reduce the annotation time. In this paper, a semi-automatic annotation tool, which aims to decrease the manual work and user faults, is proposed. The interface of the tool is designed in a user-friendly manner in order to ease the process. The characteristics and input/output formats of the tool is explained in detail within the paper. The effects on the speed and accuracy of the users are analyzed as well as automatic labeling accuracy with conducted performance tests. It is noted that a deep-learning model trained with a small dataset can decrease the manual entity annotation workload up to 78,43%.

Deniz Engin

Papers

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing.

Face Frontalization for Cross-Pose Facial Expression Recognition

Offline Signature Verification on Real-World Documents

A Semi-Automatic Annotation Interface for Named Entity and Relation Annotation on Document Images