Motion blur is a common photography artifact in dynamic environments that typically comes jointly with the other types of degradation. This paper reviews the NTIRE 2021 Challenge on Image Deblurring. In this challenge report, we describe the challenge specifics and the evaluation results from the 2 competition tracks with the proposed solutions. While both the tracks aim to recover a high-quality clean image from a blurry image, different artifacts are jointly involved. In track 1, the blurry images are in a low resolution while track 2 images are compressed in JPEG format. In each competition, there were 338 and 238 registered participants and in the final testing phase, 18 and 17 teams competed. The winning methods demonstrate the state-of-the-art performance on the image deblurring task with the jointly combined artifacts.

/pdf/ntire-2021-challenge-on-image-deblurring-4xwplefo3i.pdf

NTIRE 2021 Challenge on Image Deblurring

In this paper, we present an attention-guided deformable convolutional network for hand-held multi-frame high dynamic range (HDR) imaging, namely ADNet. This problem comprises two intractable challenges of how to handle saturation and noise properly and how to tackle misalignments caused by object motion or camera jittering. To address the former, we adopt a spatial attention module to adaptively select the most appropriate regions of various expo-sure low dynamic range (LDR) images for fusion. For the latter one, we propose to align the gamma-corrected images in the feature-level with a Pyramid, Cascading and Deformable (PCD) alignment module. The proposed AD-Net shows state-of-the-art performance compared with previous methods, achieving a PSNR-l of 39.4471 and a PSNR-μ of 37.6359 in NTIRE 2021 Multi-Frame HDR Challenge.

ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging

Most consumer-grade digital cameras can only capture a limited range of luminance in real-world scenes due to sensor constraints. Besides, noise and quantization errors are often introduced in the imaging process. In order to obtain high dynamic range (HDR) images with excellent visual quality, the most common solution is to combine multiple images with different exposures. However, it is not always feasible to obtain multiple images of the same scene and most HDR reconstruction methods ignore the noise and quantization loss. In this work, we propose a novel learning-based approach using a spatially dynamic encoder-decoder network, HDRUNet, to learn an end-to-end mapping for single image HDR reconstruction with denoising and dequantization. The network consists of a UNet-style base network to make full use of the hierarchical multi-scale information, a condition network toperform pattern-specific modulation and a weighting network for selectively retaining information. Moreover, we propose a Tanh_L 1 loss function to balance the impact of over-exposed values and well-exposed values on the network learning. Our method achieves the state-of-the-art performance in quantitative comparisons and visual quality. The proposed HDRUNet model won the second place in the single frame track of NITRE2021 High Dynamic Range Challenge. The code is available at https://github.com/chxy95/HDRUNet.

/pdf/hdrunet-single-image-hdr-reconstruction-with-denoising-and-4v258i8vnw.pdf

HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization

This paper reviews the NTIRE 2021 challenge on learning the super-Resolution space. It focuses on the participating methods and final results. The challenge addresses the problem of learning a model capable of predicting the space of plausible super-resolution (SR) images, from a single low-resolution image. The model must thus be capable of sampling diverse outputs, rather than just generating a single SR image. The goal of the challenge is to spur research into developing learning formulations and models better suited for the highly ill-posed SR problem. And thereby advance the state-of-the-art in the broader SR field. In order to evaluate the quality of the predicted SR space, we propose a new evaluation metric and perform a comprehensive analysis of the participating methods. The challenge contains two tracks: 4× and 8 scale factor. In total, 11 teams competed in the final testing× phase.

/pdf/ntire-2021-learning-the-super-resolution-space-challenge-2rbcxvbz27.pdf

NTIRE 2021 Learning the Super-Resolution Space Challenge

This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021. As a new type of image processing technology, perceptual image processing algorithms based on Generative Adversarial Networks (GAN) have produced images with more realistic textures. These output images have completely different characteristics from traditional distortions, thus pose a new challenge for IQA methods to evaluate their visual quality. In comparison with previous IQA challenges, the training and testing datasets in this challenge include the outputs of perceptual image processing algorithms and the corresponding subjective scores. Thus they can be used to develop and evaluate IQA methods on GAN-based distortions. The challenge has 270 registered participants in total. In the final testing stage, 13 participating teams submitted their models and fact sheets. Almost all of them have achieved much better results than existing IQA methods, while the winning method can demonstrate state-of-the-art performance.

/pdf/ntire-2021-challenge-on-perceptual-image-quality-assessment-18h691tz8u.pdf

NTIRE 2021 Challenge on Perceptual Image Quality Assessment

This paper reviews the first challenge on high-dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021. This manuscript focuses on the newly introduced dataset, the proposed methods and their results. The challenge aims at estimating a HDR image from one or multiple respective low-dynamic range (LDR) observations, which might suffer from under-or over-exposed regions and different sources of noise. The challenge is composed by two tracks: In Track 1 only a single LDR image is provided as input, whereas in Track 2 three differently-exposed LDR images with inter-frame motion are available. In both tracks, the ultimate goal is to achieve the best objective HDR reconstruction in terms of PSNR with respect to a ground-truth image, evaluated both directly and with a canonical tonemapping operation.

/pdf/ntire-2021-challenge-on-high-dynamic-range-imaging-dataset-dmp4lwf5u6.pdf

NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results

Laplacian Pyramid Blending is a commonly used method for several seamless image blending tasks. While the method works well for images with comparable intensity levels, it is often unable to produce artifact free images for applications which handle images with large intensity variation such as exposure fusion. This paper proposes a spatially varying Laplacian Pyramid Blending to blend images with large intensity differences. The proposed method dynamically alters the blending levels during the final stage of Pyramid Reconstruction based on the amount of local intensity variation. The proposed algorithm out performs state-of-the-art methods for image blending both qualitatively as well as quantitatively on publicly available High Dynamic Range (HDR) imaging dataset. Qualitative improvements are demonstrated in terms of details, halos and dark halos. For quantitative comparison, the no-reference perceptual metric MEF-SSIM was used.

Spatially Variant Laplacian Pyramids for Multi-Frame Exposure Fusion

Hand-held photography in low-light conditions presents a number of challenges to capture high quality images. Capturing using a high ISO results in noisy images, while capturing using longer exposure results in blurry images. This necessitates post-processing techniques to restore the latent image. Most existing methods try to estimate the latent image either by denoising or by deblurring a single image. Both these approaches are ill-posed and often result in unsatisfactory results. A few methods try to alleviate this ill-posedness using a pair of noisy-blurry images as inputs. However, most of the methods using this approach are computationally very expensive. In this paper, we propose a fast method to estimate a latent image given a pair of noisy-blurry images. To accomplish this, we propose a deep-learning based approach that uses scale space representation of the images. To improve computational efficiency, we process higher scale spaces using shallower networks and the lowest scale using a deeper network. Also, unlike existing scale-space methods that use bi-cubic interpolation, we propose a content preserving scale space transformation for decimation and interpolation. The proposed method generates state-of-the-art results at reduced computational complexity compared to state-of-the-art method. Finally, we also show that computational efficiency can be improved by 90% compared to baseline with only a marginal drop in PSNR.

Sachin Deepak Lomte

Papers

NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results

Spatially Variant Laplacian Pyramids for Multi-Frame Exposure Fusion

Content Preserving Scale Space Network for Fast Image Restoration from Noisy-Blurry Pairs

Spatially Variant Laplacian Pyramids for Multi-frame Exposure Fusion.