Upsampling

About: Upsampling is a research topic. Over the lifetime, 2426 publications have been published within this topic receiving 57613 citations.

GUN: Gradual Upsampling Network for Single Image Super-Resolution

Abstract: In this paper, an efficient super-resolution (SR) method based on deep convolutional neural network (CNN) is proposed, namely gradual upsampling network (GUN). Recent CNN-based SR methods often preliminarily magnify the low-resolution (LR) input to high-resolution (HR) input and then reconstruct the HR input, or directly reconstruct the LR input and then recover the HR result at the last layer. The proposed GUN utilizes a gradual process instead of these two commonly used frameworks. The GUN consists of an input layer, multiple upsampling and convolutional layers, and an output layer. By means of the gradual process, the proposed network can simplify the direct SR problem to multistep easier upsampling tasks with very small magnification factor in each step. Furthermore, a gradual training strategy is presented for the GUN. In the proposed training process, an initial network can be easily trained with edgelike samples, and then, the weights are gradually tuned with more complex samples. The GUN can recover fine and vivid results and is easy to be trained. The experimental results on several image sets demonstrate the effectiveness of the proposed network.

Implementation of a New Algorithm Using the STFT with Variable Frequency Resolution for the Time-Frequency Auditory Model

Abstract: A signal processing technique is proposed for the time-frequency analysis of unsteady sound signals considering the auditory perception model and is called VFR-STFT (short-time Fourier transform with variable frequency resolution). Conventional STFT, which is commonly used for the spectral analysis of unsteady sounds, is not suitable for the auditory model because the frequency resolution of the spectral analysis within the hearing system is not constant but varies with frequency. The frequency resolution of the VFR-STFT can be adjusted to a number of analyzed frequency ranges by introducing the downsampling technique. With the VFR-STFT, calculation schemes are presented for minimizing undesirable effects, such as the distortion of the overall sound level due to nonoverlapping of the analysis windows and the impairment of partial spectra due to the finite order of antialiasing filters. In addition a procedure for equalizing time grids at all frequency ranges is included in order to describe the two-dimensional time-frequency map (TFM) having different time grids. The proposed VFR-STFT is applied to the spectral analysis of the extraction of tonal components in an unsteady sound. The results are compared to those from other time-frequency analysis methods such as STFT, VFR-FFT (fast Fourier transform with variable frequency resolution), and the wavelet packet method.

Upsampling artifacts in neural audio synthesis

Abstract: A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied and are known as checkerboard artifacts (due to their characteristic visual pattern). However, their effect has been overlooked so far in audio processing. Here, we address this gap by studying this problem from the audio signal processing perspective. We first show that the main sources of upsampling artifacts are: (i) the tonal and filtering artifacts introduced by problematic upsampling operators, and (ii) the spectral replicas that emerge while upsampling. We then compare different upsampling layers, showing that nearest neighbor upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts.

Pyramid Attention Aggregation Network for Semantic Segmentation of Surgical Instruments

Abstract: Semantic segmentation of surgical instruments plays a critical role in computer-assisted surgery. However, specular reflection and scale variation of instruments are likely to occur in the surgical environment, undesirably altering visual features of instruments, such as color and shape. These issues make semantic segmentation of surgical instruments more challenging. In this paper, a novel network, Pyramid Attention Aggregation Network, is proposed to aggregate multi-scale attentive features for surgical instruments. It contains two critical modules: Double Attention Module and Pyramid Upsampling Module. Specifically, the Double Attention Module includes two attention blocks (i.e., position attention block and channel attention block), which model semantic dependencies between positions and channels by capturing joint semantic information and global contexts, respectively. The attentive features generated by the Double Attention Module can distinguish target regions, contributing to solving the specular reflection issue. Moreover, the Pyramid Upsampling Module extracts local details and global contexts by aggregating multi-scale attentive features. It learns the shape and size features of surgical instruments in different receptive fields and thus addresses the scale variation issue. The proposed network achieves state-of-the-art performance on various datasets. It achieves a new record of 97.10% mean IOU on Cata7. Besides, it comes first in the MICCAI EndoVis Challenge 2017 with 9.90% increase on mean IOU.