Open AccessPosted Content
Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Andrés Romero,Heewon Kim,Radu Timofte,Chiu Man Ho,Zibo Meng,Kyoung Mu Lee,Yuxiang Chen,Yutong Wang,Zeyu Long,Chenhao Wang,Yifei Chen,Boshen Xu,Shuhang Gu,Lixin Duan,Wen Li,Wang Bofei,Zhang Diankai,Zheng Chengjian,Liu Shaoli,Gao Si,Zhang Xiaofeng,Lu Kaidi,Xu Tianyu,Zheng Hui,Xinbo Gao,Xiumei Wang,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan +30 more
TLDR
In this article, the first Mobile AI challenge was introduced, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs.Abstract:
Video super-resolution has recently become one of the most important mobile-related problems due to the rise of video communication and streaming services. While many solutions have been proposed for this task, the majority of them are too computationally expensive to run on portable devices with limited hardware resources. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs. The participants were provided with the REDS dataset and trained their models to do an efficient 4X video upscaling. The runtime of all models was evaluated on the OPPO Find X2 smartphone with the Snapdragon 865 SoC capable of accelerating floating-point networks on its Adreno GPU. The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.read more
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings ArticleDOI
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
Wenzhe Shi,Jose Caballero,Ferenc Huszar,Johannes Totz,Andrew Peter Aitken,Rob Bishop,Daniel Rueckert,Zehan Wang +7 more
TL;DR: This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
Proceedings ArticleDOI
Searching for MobileNetV3
Andrew Howard,Ruoming Pang,Hartwig Adam,Quoc V. Le,Mark Sandler,Bo Chen,Weijun Wang,Liang-Chieh Chen,Mingxing Tan,Grace Chu,Vijay K. Vasudevan,Yukun Zhu +11 more
TL;DR: MobileNetV3 as mentioned in this paper is the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design and achieves state-of-the-art results for mobile classification, detection and segmentation.
Proceedings ArticleDOI
MnasNet: Platform-Aware Neural Architecture Search for Mobile
TL;DR: In this article, the authors propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporates model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency.
Related Papers (5)
Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
Andrey Ignatov,Andrés Romero,Heewon Kim,Radu Timofte,Chiu Man Ho,Zibo Meng,Kyoung Mu Lee,Yuxiang Chen,Yutong Wang,Zeyu Long,Chenhao Wang,Yifei Chen,Boshen Xu,Shuhang Gu,Lixin Duan,Wen Li,Wang Bofei,Zhang Diankai,Zheng Chengjian,Liu Shaoli,Gao Si,Zhang Xiaofeng,Lu Kaidi,Xu Tianyu,Zheng Hui,Xinbo Gao,Xiumei Wang,Jiaming Guo,Xueyi Zhou,Hao Jia,Youliang Yan +30 more
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report.
Andrey Ignatov,Grigory Malivenko,David Plowman,Samarth Shukla,Radu Timofte,Ziyu Zhang,Yicheng Wang,Zilong Huang,Guozhong Luo,Gang Yu,Bin Fu,Yiran Wang,Xingyi Li,Min Shi,Ke Xian,Zhiguo Cao,Jin-Hua Du,Pei-Lin Wu,Chao Ge,Jiaoyang Yao,Fangwen Tu,Bo Li,Jung Eun Yoo,Kwanggyoon Seo,Jialei Xu,Zhenyu Li,Xianming Liu,Junjun Jiang,Wei-Chi Chen,Shayan Joya,Huanhuan Fan,Zhaobing Kang,Ang Li,Tianpeng Feng,Yang Liu,Chuannan Sheng,Jian Yin,Fausto T. Benavide +37 more