Upsampling

About: Upsampling is a research topic. Over the lifetime, 2426 publications have been published within this topic receiving 57613 citations.

3D display dependent quality evaluation and rate allocation using scalable video coding

Abstract: It is well known that the human visual system can perceive high frequency content in 3D, even if that information is present in only one of the views. Then, the best 3D perception quality may be achieved by allocating the rates of the reference (right) and auxiliary (left) views asymmetrically. However the question of whether the rate reduction for the auxiliary view should be achieved by spatial resolution reduction (coding a downsampled version of the video followed by upsampling after decoding) or quality (QP) reduction is an open issue. This paper shows that which approach should be preferred depends on the 3D display technology used at the receiver. Subjective tests indicate that users prefer lower quality (larger QP) coding of the auxiliary view over lower resolution coding if a “full spatial resolution” 3D display technology (such as polarized projection) is employed. On the other hand, users prefer lower resolution coding of the auxiliary view over lower quality coding if a “reduced spatial resolution” 3D display technology (such as parallax barrier - autostereoscopic) is used. Therefore, we conclude that for 3D IPTV services, while receiving full quality/resolution reference view, users should subscribe to differently scaled versions of the auxiliary view depending on their 3D display technology. We also propose an objective 3D video quality measure that takes the 3D display technology into account.

NAS-DIP: Learning Deep Image Prior with Neural Architecture Search

Abstract: Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior for solving various inverse image restoration tasks. Instead of using hand-designed architectures, we propose to search for neural architectures that capture stronger image priors. Building upon a generic U-Net architecture, our core contribution lies in designing new search spaces for (1) an upsampling cell and (2) a pattern of cross-scale residual connections. We search for an improved network by leveraging an existing neural architecture search algorithm (using reinforcement learning with a recurrent neural network controller). We validate the effectiveness of our method via a wide variety of applications, including image restoration, dehazing, image-to-image translation, and matrix factorization. Extensive experimental results show that our algorithm performs favorably against state-of-the-art learning-free approaches and reaches competitive performance with existing learning-based methods in some cases.

Improved YOLOv4 for Aerial Object Detection

Abstract: Drones equipped with cameras are being used for surveillance purposes. These surveillance systems need vision-based object detection of ground objects which look very small because of the altitude of drones. We propose an improved YOLOv4 model targeted for vision-based small object detection. We investigated the performance of state of the art YOLOv4 object detector on the VisDrone dataset. We enhanced the features of small objects by connecting Upsampling layers and concatenating the upsampled features with the original features to obtain more refined and grained features for small objects. Experiments showed that the modified YOLOv4 achieved 2 percent better mAP results as compared to the original YOLOv4 at different image resolutions on the VisDrone dataset while running at the same speed as the original YOLOv4.

Joint Object Segmentation and Depth Upsampling

Abstract: With the advent of powerful ranging and visual sensors, nowadays, it is convenient to collect sparse 3-D point clouds and aligned high-resolution images. Benefitted from such convenience, this letter proposes a joint method to perform both depth assisted object-level image segmentation and image guided depth upsampling. To this end, we formulate these two tasks together as a bi-task labeling problem, defined in a Markov random field. An alternating direction method (ADM) is adopted for the joint inference, solving each sub-problem alternatively. More specifically, the sub-problem of image segmentation is solved by Graph Cuts, which attains discrete object labels efficiently. Depth upsampling is addressed via solving a linear system that recovers continuous depth values. By this joint scheme, robust object segmentation results and high-quality dense depth maps are achieved. The proposed method is applied to the challenging KITTI vision benchmark suite, as well as the Leuven dataset for validation. Comparative experiments show that our method outperforms stand-alone approaches.

SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation

Abstract: Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution (SR). However, conventional methods still resort to some predetermined integer scaling factors, e.g., ×2 or ×4. Thus, they are difficult to be applied when arbitrary target resolutions are required. Recent approaches ex-tend the scope to real-valued upsampling factors, even with varying aspect ratios to handle the limitation. In this pa-per, we propose the SRWarp framework to further generalize the SR tasks toward an arbitrary image transformation. We interpret the traditional image warping task, specifically when the input is enlarged, as a spatially-varying SR problem. We also propose several novel formulations, including the adaptive warping layer and multiscale blending, to reconstruct visually favorable results in the transformation process. Compared with previous methods, we do not con-strain the SR model on a regular grid but allow numerous possible deformations for flexible and diverse image editing. Extensive experiments and ablation studies justify the necessity and demonstrate the advantage of the proposed SRWarp method under various transformations.