Removing multiple degradations, such as haze, rain, and blur, from real-world images poses a challenging and illposed problem. Recently, unified models that can handle different degradations have been proposed and yield promising results. However, these approaches focus on synthetic images and experience a significant performance drop when applied to realworld images. In this paper, we introduce Uni-Removal, a twostage semi-supervised framework for addressing the removal of multiple degradations in real-world images using a unified model and parameters. In the knowledge transfer stage, Uni-Removal leverages a supervised multi-teacher and student architecture in the knowledge transfer stage to facilitate learning from pretrained teacher networks specialized in different degradation types. A multi-grained contrastive loss is introduced to enhance learning from feature and image spaces. In the domain adaptation stage, unsupervised fine-tuning is performed by incorporating an adversarial discriminator on real-world images. The integration of an extended multi-grained contrastive loss and generative adversarial loss enables the adaptation of the student network from synthetic to real-world domains. Extensive experiments on real-world degraded datasets demonstrate the effectiveness of our proposed method. We compare our Uni-Removal framework with state-of-the-art supervised and unsupervised methods, showcasing its promising results in real-world image dehazing, deraining, and deblurring simultaneously.

Uni-Removal: A Semi-Supervised Framework for Simultaneously Addressing Multiple Degradations in Real-World Images

Single image blind deblurring is highly ill-posed as neither the latent sharp image nor the blur kernel is known. Even though considerable progress has been made, several major difficulties remain for blind deblurring, including the trade-off between high-performance deblurring and real-time processing. Besides, we observe that current single image blind deblurring networks cannot further improve or stabilize the performance but significantly degrades the performance when re-deblurring is repeatedly applied. This implies the limitation of these networks in modeling an ideal deblurring process. In this work, we make two contributions to tackle the above difficulties: (1) We introduce the idempotent constraint into the deblurring framework and present a deep idempotent network to achieve improved blind non-uniform deblurring performance with stable re-deblurring. (2) We propose a simple yet efficient deblurring network with lightweight encoder-decoder units and a recurrent structure that can deblur images in a progressive residual fashion. Extensive experiments on synthetic and realistic datasets prove the superiority of our proposed framework. Remarkably, our proposed network is nearly $6.5\times $ smaller and $6.4\times $ faster than the state-of-the-art while achieving comparable high performance.

Deep Idempotent Network for Efficient Single Image Blind Deblurring

Video stabilization is highly desirable when videos undergo severe jittering artifacts. The difficulty of obtaining sufficient training data obstructs the development of video stabilization. In this work, we address this issue by presenting a Sim2RealVS benchmark with more than 1,300 pairs of shaky and stable videos. Our benchmark is curated by an in-game simulator with diverse scenes and various jittering effects. Moreover, we propose a simple yet strong baseline approach, named Motion-Trajectory Smoothing Network (MTSNet), by fully exploiting our Sim2RealVS data. Our MTSNet consists of three main steps: motion estimation, global trajectory smoothing and frame warping. In motion estimation, we design a Motion Correction and Completion (MCC) module to rectify the optical flow with low confidence, such as in textureless regions, thus providing more consistent motion estimation for next steps. Benefiting from our synthetic data, we can explicitly learn a Trajectory Smoothing Transformer (TST) with ground-truth supervision to smooth global trajectories. In training TST, we propose two fully-supervised losses, i.e., a motion magnitude similarity loss and a motion tendency similarity loss. After training, our TST is able to produce smooth motion trajectories for the shaky input videos. Extensive qualitative and quantitative results demonstrate that our MTSNet achieves superior performance on both synthetic and real-world data.

Xin Yu

Papers

Deep Idempotent Network for Efficient Single Image Blind Deblurring

Sim2RealVS: A New Benchmark for Video Stabilization with a Strong Baseline