Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Andrés Romero,Heewon Kim,Radu Timofte,Chiu Man Ho,Zibo Meng,Kyoung Mu Lee,Yuxiang Chen,Yutong Wang,Zeyu Long,Chenhao Wang,Yifei Chen,Boshen Xu,Shuhang Gu,Lixin Duan,Wen Li,Wang Bofei,Zhang Diankai,Zheng Chengjian,Liu Shaoli,Gao Si,Zhang Xiaofeng,Lu Kaidi,Xu Tianyu,Zheng Hui,Xinbo Gao,Xiumei Wang,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan +30 more
- pp 2535-2544
Reads0
Chats0
TLDR
In this paper, the first Mobile AI challenge was introduced, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs.Abstract:
Video super-resolution has recently become one of the most important mobile-related problems due to the rise of video communication and streaming services. While many solutions have been proposed for this task, the majority of them are too computationally expensive to run on portable devices with limited hardware resources. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs. The participants were provided with the REDS dataset and trained their models to do an efficient 4X video upscaling. The runtime of all models was evaluated on the OPPO Find X2 smartphone with the Snapdragon 865 SoC capable of accelerating floating-point networks on its Adreno GPU. The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.read more
Citations
More filters
Proceedings ArticleDOI
Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Radu Timofte,Maurizio Denna,Abdel Younes,Andrew Lek,Mustafa Ayazoglu,Jie Liu,Zongcai Du,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan,Zexin Zhang,Yixin Chen,Yunbo Peng,Yue Lin,Xindong Zhang,Hui Zeng,Kun Zeng,Peirong Li,Zhihuang Liu,Shiqi Xue,Shengpeng Wang +22 more
TL;DR: In this paper, the authors introduced the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a realtime performance on mobile or edge NPUs.
Proceedings ArticleDOI
Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Cheng-Ming Chiang,Hsien-Kai Kuo,Anastasia Sycheva,Radu Timofte,Min-Hung Chen,Man-Yu Lee,Yu-Syuan Xu,Yu Tseng,Shusong Xu,Jin Guo,Chao-Hung Chen,Ming-Chun Hsyu,Wen-Chia Tsai,Chao-Wei Chen,Grigory Malivenko,Minsu Kwon,Myungje Lee,Jaeyoon Yoo,Changbeom Kang,Shinjo Wang,Zheng Shaolong,Hao Dejun,Xie Fen,Feng Zhuang,Yipeng Ma,Jingyang Peng,Tao Wang,Fenglong Song,Chih-Chung Hsu,Kwan-Lin Chen,Mei-Hsuang Wu,Vishal Chudasama,Kalpesh Prajapati,Heena Patel,Anjali Sarvaiya,Kishor P. Upla,Kiran B. Raja,Raghavendra Ramachandra,Christoph Busch,Etienne de Stoutz +40 more
TL;DR: In this article, an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs was developed.
Proceedings ArticleDOI
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Kim Byeoung-su,Radu Timofte,Angeline Pouget,Fenglong Song,Cheng Li,Shuai Xiao,Zhongqian Fu,Matteo Maggioni,Yibin Huang,Shen Cheng,Xin Lu,Yifeng Zhou,Liangyu Chen,Donghao Liu,Xiangyu Zhang,Haoqiang Fan,Jian Sun,Shuaicheng Liu,Minsu Kwon,Myungje Lee,Jaeyoon Yoo,Changbeom Kang,Shinjo Wang,Bin Huang,Tianbao Zhou,Shuai Liu,Lei Lei,Chaoyu Feng,Liguang Huang,Zhikun Lei,Feifei Chen +31 more
TL;DR: In this article, the authors introduced the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution that can demonstrate high efficiency on smartphone GPUs.
Journal ArticleDOI
Intrusion Detection System for IoT Based on Deep Learning and Modified Reptile Search Algorithm
Abdelghani Dahou,Mohamed Abd Elaziz,Samia Allaoua Chelloug,Mohammed A. Awadallah,Mohammed Azmi Al-Betar,Mohammed A. A. Al-qaness,Agostino Forestiero +6 more
TL;DR: A new feature selection mechanism is proposed based on a recently developed MH method, called Reptile Search Algorithm (RSA), which is inspired by the hunting behaviors of the crocodiles, which boosts the IDS system performance by selecting only the most important features from the extracted features using the CNN model.
Proceedings ArticleDOI
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Grigory Malivenko,Radu Timofte,Sheng Chen,Xin Xia,Zhaoyan Liu,Yuwei Zhang,Feng Zhu,Jiashi Li,Xuefeng Xiao,Yuan Tian,Xinglong Wu,Christos Kyrkou,Yixin Chen,Zexin Zhang,Yunbo Peng,Yue Lin,Saikat Dutta,Sourya Dipta Das,Nisarg Shah,Himanshu Kumar,Chao Ge,Pei-Lin Wu,Jin-Hua Du,Andrew Batutin,Juan Pablo Federico,Konrad Lyda,Levon Khojoyan,Abhishek Thanki,Sayak Paul,Shahid Siddiqui +30 more
TL;DR: In this article, the first Mobile AI challenge was introduced to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings ArticleDOI
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
Wenzhe Shi,Jose Caballero,Ferenc Huszar,Johannes Totz,Andrew Peter Aitken,Rob Bishop,Daniel Rueckert,Zehan Wang +7 more
TL;DR: This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
Proceedings ArticleDOI
Searching for MobileNetV3
Andrew Howard,Ruoming Pang,Hartwig Adam,Quoc V. Le,Mark Sandler,Bo Chen,Weijun Wang,Liang-Chieh Chen,Mingxing Tan,Grace Chu,Vijay K. Vasudevan,Yukun Zhu +11 more
TL;DR: MobileNetV3 as mentioned in this paper is the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design and achieves state-of-the-art results for mobile classification, detection and segmentation.
Proceedings ArticleDOI
MnasNet: Platform-Aware Neural Architecture Search for Mobile
TL;DR: In this article, the authors propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporates model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency.