Showing papers on "Bicubic interpolation published in 2016"

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Abstract: Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods.

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Abstract: Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods.

End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks

Abstract: One impressive advantage of convolutional neural networks (CNNs) is their ability to automatically learn feature representation from raw pixels, eliminating the need for hand-designed procedures. However, recent methods for single image super-resolution (SR) fail to maintain this advantage. They utilize CNNs in two decoupled steps, i.e., first upsampling the low resolution (LR) image to the high resolution (HR) size with hand-designed techniques (e.g., bicubic interpolation), and then applying CNNs on the upsampled LR image to reconstruct HR results. In this paper, we seek an alternative and propose a new image SR method, which jointly learns the feature extraction, upsampling and HR reconstruction modules, yielding a completely end-to-end trainable deep CNN. As opposed to existing approaches, the proposed method conducts upsampling in the latent feature space with filters that are optimized for the task of image SR. In addition, the HR reconstruction is performed in a multi-scale manner to simultaneously incorporate both short- and long-range contextual information, ensuring more accurate restoration of HR images. To facilitate network training, a new training approach is designed, which jointly trains the proposed deep network with a relatively shallow network, leading to faster convergence and more superior performance. The proposed method is extensively evaluated on widely adopted data sets and improves the performance of state-of-the-art methods with a considerable margin. Moreover, in-depth ablation studies are conducted to verify the contribution of different network designs to image SR, providing additional insights for future research.

Shading-Aware Multi-view Stereo

Abstract: We present a novel multi-view reconstruction approach that effectively combines stereo and shape-from-shading energies into a single optimization scheme. Our method uses image gradients to transition between stereo-matching (which is more accurate at large gradients) and Lambertian shape-from-shading (which is more robust in flat regions). In addition, we show that our formulation is invariant to spatially varying albedo without explicitly modeling it. We show that the resulting energy function can be optimized efficiently using a smooth surface representation based on bicubic patches, and demonstrate that this algorithm outperforms both previous multi-view stereo algorithms and shading based refinement approaches on a number of datasets.

Generalizing bicubic splines for modeling and IGA with irregular layout

Abstract: Quad meshes can be interpreted as tensor-product spline control meshes as long as they form a regular grid, locally. We present a new option for complementing bi-3 splines by bi-4 splines near irregularities in the mesh layout, where less or more than four quadrilaterals join. These new generalized surface and IGA (isogeometric?analysis) elements have as their degrees of freedom the vertices of the irregular quad mesh. From a geometric design point of view, the new construction distinguishes itself from earlier work by a notably better distribution of highlight lines. From the IGA point of view, increased smoothness and reproduction at the irregular point?yield fast convergence. Bi-3 tensor-product splines are complemented by bi-4 splines near irregular points.The vertices of the irregular quad mesh serve as spline-like control points.The resulting surfaces have a good distribution of highlight lines.The resulting surfaces have a increased smoothness and reproduction at irregular points.

Four-Direction Residual Interpolation for Demosaicking

Abstract: In this paper, we propose a four-direction residual interpolation (FDRI) method for color filter array interpolation. The proposed algorithm exploits a guided filtering process to generate the tentative image. The residual image is generated by exploiting the tentative and original images. We use an FDRI algorithm to more accurately estimate the missing pixel values; the estimated image is adaptively combined with a joint inverse gradient weight. Based on the experimental results, the proposed method provides a superior performance in terms of objective and subjective quality compared with the conventional state-of-the-art demosaicking methods.

Noise-induced bias for convolution-based interpolation in digital image correlation

Abstract: In digital image correlation (DIC), the noise-induced bias is significant if the noise level is high or the contrast of the image is low. However, existing methods for the estimation of the noise-induced bias are merely applicable to traditional interpolation methods such as linear and cubic interpolation, but are not applicable to generalized interpolation methods such as BSpline and OMOMS. Both traditional interpolation and generalized interpolation belong to convolution-based interpolation. Considering the widely use of generalized interpolation, this paper presents a theoretical analysis of noise-induced bias for convolution-based interpolation. A sinusoidal approximate formula for noise-induced bias is derived; this formula motivates an estimating strategy which is with speed, ease, and accuracy; furthermore, based on this formula, the mechanism of sophisticated interpolation methods generally reducing noise-induced bias is revealed. The validity of the theoretical analysis is established by both numerical simulations and actual subpixel translation experiment. Compared to existing methods, formulae provided by this paper are simpler, briefer, and more general. In addition, a more intuitionistic explanation of the cause of noise-induced bias is provided by quantitatively characterized the position-dependence of noise variability in the spatial domain.

Soft-Then-Hard Subpixel Land Cover Mapping Based on Spatial-Spectral Interpolation

Abstract: In this letter, a novel subpixel sharpening for soft-then-hard subpixel mapping (SPM) is proposed. First, the fractional images for each class are, respectively, derived by spectral unmixing followed by spatial interpolation and by spectral interpolation followed by spectral unmixing. Bilinear and bicubic interpolation is used as the spatial and spectral interpolation methods. The fractional images for each class are then integrated together using the appropriate weighting parameter. Finally, the integrated finer fractional images are used to allocate hard class labels to subpixels. The proposed method is fast and does not need any prior spatial structure information. Experiments on two actual hyperspectral images show that the proposed method produces higher accuracy results than the existing algorithms. Moreover, both the spatial and spectral information is fully utilized to improve the accuracy of the SPM results.

Computational imaging with a highly parallel image-plane-coded architecture: challenges and solutions.

Abstract: This paper investigates a highly parallel extension of the single-pixel camera based on a focal plane array. It discusses the practical challenges that arise when implementing such an architecture and demonstrates that system-specific optical effects must be measured and integrated within the system model for accurate image reconstruction. Three different projection lenses were used to evaluate the ability of the system to accommodate varying degrees of optical imperfection. Reconstruction of binary and grayscale objects using system-specific models and Nesterov’s proximal gradient method produced images with higher spatial resolution and lower reconstruction error than using either bicubic interpolation or a theoretical system model that assumes ideal optical behavior. The high-quality images produced using relatively few observations suggest that higher throughput imaging may be achieved with such architectures than with conventional single-pixel cameras. The optical design considerations and quantitative performance metrics proposed here may lead to improved image reconstruction for similar highly parallel systems.

Image Interpolation Based on Non-local Geometric Similarities and Directional Gradients

Abstract: Image interpolation offers an efficient way to compose a high-resolution (HR) image from the observed low-resolution (LR) image. Advanced interpolation techniques design the interpolation weighting coefficients by solving a minimum mean-square-error (MMSE) problem in which the local geometric similarity is often considered. However, using local geometric similarities cannot usually make the MMSE-based interpolation as reliable as expected. To solve this problem, we propose a robust interpolation scheme by using the nonlocal geometric similarities to construct the HR image. In our proposed method, the MMSE-based interpolation weighting coefficients are generated by solving a regularized least squares problem that is built upon a number of dual-reference patches drawn from the given LR image and regularized by the directional gradients of these patches. Experimental results demonstrate that our proposed method offers a remarkable quality improvement as compared to some state-of-the-art methods, both objectively and subjectively.

Spatial Interpolation Algorithm For Error Concealment

Abstract: In this paper, we propose a new spatial interpolation algorithm for Intra-Frame error concealment. The method aims at interpolating areas in the image, which have been affected by packet loss. We have proposed an edge detection technique to aid the bilinear interpolation. The edge detection scheme is based on designing a robust Hough transform-based technique that is capable of systematically connecting edges irrespective of the number of edge points surrounding missing areas. The connected edges are used to divide the missing areas into different regions for interpolation along the directions of each detected line. Simulation results demonstrate that the proposed algorithm can recover the missing areas with a greater accuracy, when compared with the bilinear interpolation technique.

Image Resolution Enhancement by Using Interpolation Followed by Iterative Back Projection

Abstract: In this paper, we propose a new super resolution technique based on the interpolation followed by registering them using iterative back projection (IBP). Low resolution images are being interpolated and then the interpolated images are being registered in order to generate a sharper high resolution image. The proposed technique has been tested on Lena, Elaine, Pepper, and Baboon. The quantitative peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) results as well as the visual results show the superiority of the proposed technique over the conventional and state-of-art image super resolution techniques. For Lena's image, the PSNR is 6.52 dB higher than the bicubic interpolation.

Image super-resolution reconstruction method and system based on sparse representation

Abstract: The invention discloses an image super-resolution reconstruction method based on sparse representation and a system. The method comprises following steps: a complete high-low resolution dictionary pair is obtained by means of an image training library; bicubic amplification is performed on a low resolution image to be reconstructed to obtain an initial image of the super-resolution algorithm; the first order and second order gradient features of the initial image are extracted; overlapping partition is performed on the extracted features and sparse representation of the low frequency image blocks is obtained by means of the low resolution dictionary; the sparse representation coefficient of the obtained low resolution image blocks is approximately equal to the sparse representation coefficient of the high resolution image blocks to be solved in the high resolution dictionary and the initial estimate of a corresponding high resolution image is estimated by means of the sparse representation coefficient; reconstruction error of the high resolution image is reduced by means of a back projection filter.

Modification algorithm of Cubic B-spline curve interpolation

Abstract: Based on cubic B-Spline curve mathematical properties, theoretical analysis the cubic B-Spline curve recursive formula of Taylor development of first-order, derivation of two order in the interpolation cycle under the condition of certain interpolation increment only and interpolation speed, change the interpolation increments can be amended cubic times B-Spline curves purpose. So this paper presents such a modified interpolation algorithm, which is combined with specific examples of matlab7.0 verifies that the algorithm is correct; it is consistent with a cubic B-Spline curve interpolation requirements.

An edge-guided image interpolation method using Taylor series approximation

Abstract: In this paper, a novel real-time edge-guided interpolation method is presented to produce the high resolution image without any disturbing artifacts such as blurring, jagging, and overshooting. The proposed method computes the first- and second-order derivatives of an input image to measure the geometry of edges in the image. Based on these measures, the value of a pixel to be interpolated is estimated along four directions using Taylor series approximation. The four directional estimates are adaptively fused based on the orientation of a local edge to obtain the edge-guided interpolation output. Experimental results demonstrate that the proposed interpolation method outperforms the conventional ones both in terms of objective and subjective image qualities with low computational complexity1.

Method for realizing super resolution for image

Abstract: The invention relates to a method for realizing super resolution for an image and belongs to the computer vision field. The method includes the following steps of: A1, data preprocessing: a certain number of high-resolution natural images are adopted to form a data set, a certain number of image blocks are extracted from the data set, Bicubic interpolation down-sampling and up-sampling in three times are carried out on the image blocks, and low-resolution images can be obtained; A2, network structure design: a designed convolutional neural network has 4 layers altogether; A3, hyper parameter selection: parameters such as network learning rate, learning momentum and batch_size are determined; and A4, network training and super parameter optimization: the convolutional neural network of all images in the training set from low-resolution images to corresponding high-resolution images is trained, and after any one image is inputted into the trained network, a high-resolution image can be obtained, so that the super resolution of the image can be realized.

FPGA-Accelerated Particle-Grid Mapping

Abstract: Computing the forces derived from long-range electrostaticsis a critical application and also a central part of MolecularDynamics. Part of that computation, the transformation of a charge grid to a potential grid via a 3D FFT, has received some attentionrecently and has been found to work extremely well on FPGAs. Here we report on the rest of the computation, which consists oftwo mappings: charges onto a grid and a potential grid onto theparticles. These mappings are interesting in their own right as theyare far more compute intensive than the FFTs, each is typicallydone using tricubic interpolation. We believe that these mappingshave been studied only once previously for FPGAs and then foundto be exorbitantly expensive, i.e., only bicubic would fit on the chip. In the current work we find that, when using the Altera Arria 10, not only do both mappings fit, but also an appropriately sized 3DFFT. This enables the building of a balanced accelerator for theentire long-range electrostatics computation on a single FPGA. Thisdesign scales directly to FPGA clusters. Other contributions include a new mapping scheme based on table lookup and a measure of the utility of the floating point support of the Arria-10.

Calendar time interpolation of amplitude maps from 4D seismic data

Abstract: A calendar time interpolation method for 2D seismic amplitude maps, done in two steps, is presented. The contour interpolation part is formulated as a quadratic programming problem, whereas the amplitude value interpolation is based on a conditional probability formulation. The method is applied on field data from the Sleipner CO2 storage project. The output is a continuous image (movie) of the CO2 plume. Besides visualization, the output can be used to better couple 4D seismic to other types of data acquired. The interpolation uncertainty increases with the time gap between consecutive seismic surveys and is estimated by leaving a survey out (blind test). Errors from such tests can be used to identify problems in understanding the flow and possibly improve the interpolation scheme for a given case. Field-life cost of various acquisition systems and repeat frequencies are linked to the time-lapse interpolation errors. The error in interpolated amplitudes increased by 3%-4% per year of interpolation gap for the Sleipner case. Interpolation can never fully replace measurements.

Ultrafast imaging with anti-aliasing based on optical time-division multiplexing

Abstract: Serial time-encoded amplified microscopy (STEAM) is a novel ultrafast imaging technique that is based on space-to-time-to-wavelength mapping. Nevertheless, the technique requires a high-cost electronic digitizer of several tens of gigahertz sampling rate to read out sufficient image information. To acquire a large amount of image information by using a relatively low-sampling-rate electronic digitizer, an anti-aliasing technique based on optical time-division multiplexing is proposed. A 38.88 MHz line-scan imaging system is demonstrated experimentally. By using the proposed anti-aliasing technique, a 20 GS/s sampling rate is achieved by employing a 10 GS/s electronic digitizer. Defects and scratches on the target that were not identifiable originally can be clearly distinguished after using the proposed technique. Numerical analysis shows that the image quality can be improved by 4.16 dB, compared to that not using the anti-aliasing technique and at least 2.3 dB comparing to those obtained by bilinear, bicubic, and nearest-neighbor interpolation and Lanczos resampling techniques.

Nonlinear Interpolation on Manifold of Reduced-Order Models in Magnetodynamic Problems

Abstract: The proper orthogonal decomposition (POD) is an efficient model-order reduction (MOR) technique for linear problems in computational sciences, recently gaining popularity in electromagnetics. However, its efficiency has been shown to considerably degrade for nonlinear problems. In this paper, we propose a method to reduce nonlinear magnetodynamic problems by combining POD with an interpolation on manifolds, which interpolates the reduced bases to efficiently construct the appropriate reduced model. This method, new in electromagnetics, is applied on an inductor-core system and showed good results compared with the classical MOR approaches, e.g., direct POD reduction and a standard interpolation of pre-computed reduced bases (i.e., Lagrange interpolation).

Coupled dictionary learning for multimodal image super-resolution

Abstract: Real-world data processing problems often involve multiple data modalities, e.g., panchromatic and multispectral images, positron emission tomography (PET) and magnetic resonance imaging (MRI) images. As these modalities capture information associated with the same phenomenon, they must necessarily be correlated, although the precise relation is rarely known. In this paper, we propose a coupled dictionary learning (CDL) framework to automatically learn these relations. In particular, we propose a new data model to characterize both similarities and discrepancies between multimodal signals in terms of common and unique sparse representations with respect to a group of coupled dictionaries. However, learning these coupled dictionaries involves solving a highly non-convex structural dictionary learning problem. To address this problem, we design a coupled dictionary learning algorithm, referred to sequential recursive optimization (SRO) algorithm, to sequentially learn these dictionaries in a recursive manner. By capitalizing on our model and algorithm, we conceive a CDL based multimodal image super-resolution (SR) approach. Practical multispectral image SR experiments demonstrate that our SR approach outperforms the bicubic interpolation and the state-of-the-art dictionary learning based image SR approach, with Peak-SNR (PSNR) gains of up to 8.2 dB and 5.1 dB, respectively.

Fourier sampling of sun path for applications in solar energy

Abstract: A systematic approach is presented for the sampling and interpolation over sun path. The annual sun path is described in terms of the ecliptic longitude and hour angle so that all possible positions of the sun are mapped on a rectangular domain. This enables the use of many efficient algorithms, and the bicubic interpolation with Catmull–Rom splines is proposed. It is shown that a sufficiently good accuracy can be achieved with as little as 32 sampling points for a year.

Bilinear surface smoothing for spatial interpolation with optional incorporation of an explanatory variable. Part 2: Application to synthesized and rainfall data

Abstract: The non-parametric mathematical framework of bilinear surface smoothing (BSS) methodology provides flexible means for spatial (two dimensional) interpolation of variables. As presented in a companion paper, interpolation is accomplished by means of fitting consecutive bilinear surface into a regression model with known break points and adjustable smoothing terms defined by means of angles formed by those bilinear surface. Additionally, the second version of the methodology (BSSE) incorporates, in an objective manner, the influence of an explanatory variable available at a considerably denser dataset. In the present study, both versions are explored and illustrated using both synthesized and real world (hydrological) data, and practical aspects of their application are discussed. Also, comparison and validation against the results of commonly used spatial interpolation methods (inverse distance weighted, spline, ordinary kriging and ordinary cokriging) are performed in the context of the real world...