“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the `creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -- where it is designed to augment, rather than replace, human creativity.

/pdf/artificial-intelligence-in-the-creative-industries-a-review-33ttbvg0ir.pdf

Artificial Intelligence in the Creative Industries: A Review

This paper reviews the first NTIRE challenge on perceptual image enhancement with the focus on proposed solutions and results. The participating teams were solving a real-world photo enhancement problem, where the goal was to map low-quality photos from the iPhone 3GS device to the same photos captured with Canon 70D DSLR camera. The considered problem embraced a number of computer vision subtasks, such as image denoising, image resolution and sharpness enhancement, image color/contrast/exposure adjustment, etc. The target metric used in this challenge combined PSNR and SSIM scores with solutions' perceptual results measured in the user study. The proposed solutions significantly improved baseline results, defining the state-of-the-art for practical image enhancement.

/pdf/ntire-2019-challenge-on-image-enhancement-methods-and-1sb09bgi2g.pdf

NTIRE 2019 Challenge on Image Enhancement: Methods and Results

This paper reviews the second AIM learned ISP challenge and provides the description of the proposed solutions and results. The participating teams were solving a real-world RAW-to-RGB mapping problem, where to goal was to map the original low-quality RAW images captured by the Huawei P20 device to the same photos obtained with the Canon 5D DSLR camera. The considered task embraced a number of complex computer vision subtasks, such as image demosaicing, denoising, white balancing, color and contrast correction, demoireing, etc. The target metric used in this challenge combined fidelity scores (PSNR and SSIM) with solutions' perceptual results measured in a user study. The proposed solutions significantly improved the baseline results, defining the state-of-the-art for practical image signal processing pipeline modeling.

AIM 2020 Challenge on Learned Image Signal Processing Pipeline

This paper reviews the first AIM challenge on mapping camera RAW to RGB images with the focus on proposed solutions and results. The participating teams were solving a real-world photo enhancement problem, where the goal was to map the original low-quality RAW images from the Huawei P20 device to the same photos captured with the Canon 5D DSLR camera. The considered problem embraced a number of computer vision subtasks, such as image demosaicing, denoising, gamma correction, image resolution and sharpness enhancement, etc. The target metric used in this challenge combined fidelity scores (PSNR and SSIM) with solutions' perceptual results measured in a user study. The proposed solutions significantly improved baseline results, defining the state-of-the-art for RAW to RGB image restoration.

AIM 2019 Challenge on RAW to RGB Mapping: Methods and Results

This paper reviews the NTIRE challenge on image colorization (estimating color information from the corresponding gray image) with focus on proposed solutions and results. It is the first challenge of its kind. The challenge had 2 tracks. Track 1 takes a single gray image as input. In Track 2, in addition to the gray input image, some color seeds (randomly samples from the latent color image) are also provided for guiding the colorization process. The operators were learnable through provided pairs of gray and color training images. The tracks had 188 registered participants, and 8 teams competed in the final testing phase.

NTIRE 2019 Challenge on Image Colorization: Report

This paper presents a novel method for a single image rain streak removal problem which exploits the spatial as well as wavelet transformed coefficients of the rainy images. The proposed method adopts the Conditional Generative Adversarial Network [1] framework and consists of two following networks: Generator and Discriminator. The generator model receives the input from both spatial, frequency domain of the rainy image and yields five de-rained image candidates. A Deep Residual Network [2] has been used to merge these de-rained candidates and predict a single de-rained image. To ensure the visual quality of the de-rained image, Perceptual loss function [3] in addition to adversarial training has been incorporated. Extensive experiments on the synthetic and real-world rainy images dataset reveal an improvement over the existing state-of-the-art methods [4], [5] by $\sim 1.08\% ,2.57\% $ in Structural Similarity Index [6] and $\sim 7.39\% ,9.95\% $ in Peak signal-to-noise ratio respectively.

Priyankar Jain

Papers

NTIRE 2019 Challenge on Image Colorization: Report

Dual-Domain Single Image De-Raining Using Conditional Generative Adversarial Network