Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Radu Timofte,Maurizio Denna,Abdel Younes,Andrew Lek,Mustafa Ayazoglu,Jie Liu,Zongcai Du,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan,Zexin Zhang,Yixin Chen,Yunbo Peng,Yue Lin,Xindong Zhang,Hui Zeng,Kun Zeng,Peirong Li,Zhihuang Liu,Shiqi Xue,Shengpeng Wang +22 more
- pp 2525-2534
TLDR
In this paper, the authors introduced the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a realtime performance on mobile or edge NPUs.Abstract:
Image super-resolution is one of the most popular computer vision problems with many important applications to mobile devices. While many solutions have been proposed for this task, they are usually not optimized even for common smartphone AI hardware, not to mention more constrained smart TV platforms that are often supporting INT8 inference only. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a real-time performance on mobile or edge NPUs. For this, the participants were provided with the DIV2K dataset and trained quantized models to do an efficient 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated NPU capable of accelerating quantized neural networks. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms while achieving high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.read more
Citations
More filters
Proceedings ArticleDOI
Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Andrés Romero,Heewon Kim,Radu Timofte,Chiu Man Ho,Zibo Meng,Kyoung Mu Lee,Yuxiang Chen,Yutong Wang,Zeyu Long,Chenhao Wang,Yifei Chen,Boshen Xu,Shuhang Gu,Lixin Duan,Wen Li,Wang Bofei,Zhang Diankai,Zheng Chengjian,Liu Shaoli,Gao Si,Zhang Xiaofeng,Lu Kaidi,Xu Tianyu,Zheng Hui,Xinbo Gao,Xiumei Wang,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan +30 more
TL;DR: In this paper, the first Mobile AI challenge was introduced, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs.
Proceedings ArticleDOI
Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices
TL;DR: XLSR as mentioned in this paper is a hardware limitation aware, extremely lightweight quantization robust real-time super resolution network (xLSR) based on root modules introduced in Image classification.
Proceedings ArticleDOI
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Cheng-Ming Chiang,Hsien-Kai Kuo,Anastasia Sycheva,Radu Timofte,Min-Hung Chen,Man-Yu Lee,Yu-Syuan Xu,Yu Tseng,Shusong Xu,Jin Guo,Chao-Hung Chen,Ming-Chun Hsyu,Wen-Chia Tsai,Chao-Wei Chen,Grigory Malivenko,Minsu Kwon,Myungje Lee,Jaeyoon Yoo,Changbeom Kang,Shinjo Wang,Zheng Shaolong,Hao Dejun,Xie Fen,Feng Zhuang,Yipeng Ma,Jingyang Peng,Tao Wang,Fenglong Song,Chih-Chung Hsu,Kwan-Lin Chen,Mei-Hsuang Wu,Vishal Chudasama,Kalpesh Prajapati,Heena Patel,Anjali Sarvaiya,Kishor P. Upla,Kiran B. Raja,Raghavendra Ramachandra,Christoph Busch,Etienne de Stoutz +40 more
TL;DR: In this article, an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs was developed.
Proceedings ArticleDOI
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Kim Byeoung-su,Radu Timofte,Angeline Pouget,Fenglong Song,Cheng Li,Shuai Xiao,Zhongqian Fu,Matteo Maggioni,Yibin Huang,Shen Cheng,Xin Lu,Yifeng Zhou,Liangyu Chen,Donghao Liu,Xiangyu Zhang,Haoqiang Fan,Jian Sun,Shuaicheng Liu,Minsu Kwon,Myungje Lee,Jaeyoon Yoo,Changbeom Kang,Shinjo Wang,Bin Huang,Tianbao Zhou,Shuai Liu,Lei Lei,Chaoyu Feng,Liguang Huang,Zhikun Lei,Feifei Chen +31 more
TL;DR: In this article, the authors introduced the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution that can demonstrate high efficiency on smartphone GPUs.
Proceedings ArticleDOI
Anchor-based Plain Net for Mobile Image Super-Resolution
TL;DR: Zhang et al. as discussed by the authors proposed anchor-based plain net (ABPN) for 8-bit quantization and deployed it on mobile device to improve the performance of super-resolution.
References
More filters
Journal ArticleDOI
Image Super-Resolution Using Deep Convolutional Networks
TL;DR: Zhang et al. as discussed by the authors proposed a deep learning method for single image super-resolution (SR), which directly learns an end-to-end mapping between the low/high-resolution images.
Journal ArticleDOI
Image Super-Resolution Via Sparse Representation
TL;DR: This paper presents a new approach to single-image superresolution, based upon sparse signal representation, which generates high-resolution images that are competitive or even superior in quality to images produced by other similar SR methods.
Book ChapterDOI
Learning a Deep Convolutional Network for Image Super-Resolution
TL;DR: This work proposes a deep learning method for single image super-resolution (SR) that directly learns an end-to-end mapping between the low/high-resolution images and shows that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network.
Proceedings ArticleDOI
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
TL;DR: In this article, a very deep convolutional network inspired by VGG-net was used for image superresolution, which achieved state-of-the-art performance in accuracy.
Journal ArticleDOI
Super-resolution image reconstruction: a technical overview
TL;DR: The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts to present the technical review of various existing SR methodologies which are often employed.