Showing papers on "Image scaling published in 2017"

Video Frame Interpolation via Adaptive Convolution

Abstract: Video frame interpolation typically involves two steps: motion estimation and pixel synthesis. Such a two-step approach heavily depends on the quality of motion estimation. This paper presents a robust video frame interpolation method that combines these two steps into a single process. Specifically, our method considers pixel synthesis for the interpolated frame as local convolution over two input frames. The convolution kernel captures both the local motion between the input frames and the coefficients for pixel synthesis. Our method employs a deep fully convolutional neural network to estimate a spatially-adaptive convolution kernel for each pixel. This deep neural network can be directly trained end to end using widely available video data without any difficult-to-obtain ground-truth data like optical flow. Our experiments show that the formulation of video interpolation as a single convolution process allows our method to gracefully handle challenges like occlusion, blur, and abrupt brightness change and enables high-quality video frame interpolation.

SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior

Abstract: The performance of traditional face recognition systems is sharply reduced when encountered with a low-resolution (LR) probe face image. To obtain much more detailed facial features, some face super-resolution (SR) methods have been proposed in the past decade. The basic idea of a face image SR is to generate a high-resolution (HR) face image from an LR one with the help of a set of training examples. It aims at transcending the limitations of optical imaging systems. In this paper, we regard face image SR as an image interpolation problem for domain-specific images. A missing intensity interpolation method based on smooth regression with a local structure prior (LSP), named SRLSP for short, is presented. In order to interpolate the missing intensities in a target HR image, we assume that face image patches at the same position share similar local structures, and use smooth regression to learn the relationship between LR pixels and missing HR pixels of one position patch. Performance comparison with the state-of-the-art SR algorithms on two public face databases and some real-world images shows the effectiveness of the proposed method for a face image SR in general. In addition, we conduct a face recognition experiment on the extended Yale-B face database based on the super-resolved HR faces. Experimental results clearly validate the advantages of our proposed SR method over the state-of-the-art SR methods in face recognition application.

Quantum realization of the bilinear interpolation method for NEQR.

Abstract: In recent years, quantum image processing is one of the most active fields in quantum computation and quantum information. Image scaling as a kind of image geometric transformation has been widely studied and applied in the classical image processing, however, the quantum version of which does not exist. This paper is concerned with the feasibility of the classical bilinear interpolation based on novel enhanced quantum image representation (NEQR). Firstly, the feasibility of the bilinear interpolation for NEQR is proven. Then the concrete quantum circuits of the bilinear interpolation including scaling up and scaling down for NEQR are given by using the multiply Control-Not operation, special adding one operation, the reverse parallel adder, parallel subtractor, multiplier and division operations. Finally, the complexity analysis of the quantum network circuit based on the basic quantum gates is deduced. Simulation result shows that the scaled-up image using bilinear interpolation is clearer and less distorted than nearest interpolation.

Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection

Abstract: In this paper, a Deep Learning system for accurate road detection is proposed using the ResNet-101 network with a fully convolutional architecture and multiple upscaling steps for image interpolation. It is demonstrated that significant generalization gains in the learning process are attained by randomly generating augmented training data using several geometric transformations and pixelwise changes, such as affine and perspective transformations, mirroring, image cropping, distortions, blur, noise, and color changes. In addition, this paper shows that the use of a 4-step upscaling strategy provides optimal learning results as compared to other similar techniques that perform data upscaling based on shallow layers with scarce representation of the scene data. The complete system is trained and tested on data from the KITTI benchmark and besides it is also tested on images recorded on the Campus of the University of Alcala (Spain). The improvement attained after performing data augmentation and conducting a number of training variants is really encouraging, showing the path to follow for enhanced learning generalization of road detection systems with a view to real deployment in self-driving cars.

On the robustness of constrained convolutional neural networks to JPEG post-compression for image resampling detection

Abstract: Detecting image resampling in re-compressed images is a very challenging problem. Existing approaches to image resampling detection operate by building pre-selected model to locate periodicities in linear predictor residues. Additionally, if an image was JPEG compressed before resampling, existing techniques detect tampering using the artifacts left by the pre-compression. However, state-of-the-art approaches cannot detect resampling in re-compressed images initially compressed with high quality factor. In this paper, we propose a novel deep learning approach to adaptively learn resampling detection features directly from data. To accomplish this, we use our recently proposed constrained convolutional layer. Through a set of experiments we evaluate the effectiveness of our proposed constrained convolutional neural network (CNN) to detect resampling in re-compressed images. The results of these experiments show that our constrained CNN can accurately detect resampling in re-compressed images in scenarios that previous approaches are unable to detect.

An Investigation of Interpolation Techniques to Generate 2D Intensity Image From LIDAR Data

Abstract: Light detection and ranging (LIDAR) has become a part and parcel of ongoing research in autonomous vehicles. LIDAR efficiently captures data during day and night alike; yet, data accuracy is affected in altered weather conditions. LIDAR data fusion with sensors, such as color camera, hyperspectral camera, and RADAR, proves to be a viable solution to improve the quality of data and add spectral information. LIDAR 3-D point cloud containing intensity data are transformed to 2-D intensity images for the said purpose. LIDAR produces large point cloud, but, while generating images for limited field of view, data sparsity results in poor quality images. Moreover, 3-D to 2-D data transformation also involves data reduction, which further deteriorates the quality of images. This paper focuses on generating intensity images from LIDAR data using interpolation techniques, including bi-linear, natural neighbor, bi-cubic, kriging, inverse distance, and weighted and nearest neighbor interpolation. The main focus is to test the suitability of interpolation methods for 2-D image generation, and analyze the quality of the generated 2-D image. Image similarity metrics, such as root mean square error, normalized least square error, peak signal-to-noise ratio, correlation, difference entropy, mutual information, and structural similarity index measurement, are utilized for camera and LIDAR image matching, and their ability to compare images from heterogeneous sensors is also analyzed. Generated images can further be used for data fusion purpose. Images generated using LIDAR points have a relevant distance matrix as well, which can be used to find the distance of any given pixel from the image. In addiiton, the accuracy of interpolated distance data is evaluated as well by comparing it with the original distance values of traffic cones placed in front of vehicle. Results show that the inverse distance weighted interpolation outperforms other selected methods in 2-D image quality, and images from nearest neighbor appear brighter subjectively.

PCA-based denoising method for division of focal plane polarimeters

Abstract: Division of focal plane (DoFP) polarimeters are composed of interlaced linear polarizers overlaid upon a focal plane array sensor. The interpolation is essential to reconstruct polarization information. However, current interpolation methods are based on the unrealistic assumption of noise-free images. Thus, it is advantageous to carry out denoising before interpolation. In this paper, we propose a principle component analysis (PCA) based denoising method, which works directly on DoFP images. Both simulated and real DoFP images are used to evaluate the denoising performance. Experimental results show that the proposed method can effectively suppress noise while preserving edges.

The Shannon Total Variation

Abstract: Discretization schemes commonly used for total variation regularization lead to images that are difficult to interpolate, which is a real issue for applications requiring subpixel accuracy and aliasing control. In the present work, we reconciliate total variation with Shannon interpolation and study a Fourier-based estimate that behaves much better in terms of grid invariance, isotropy, artifact removal and subpixel accuracy. We show that this new variant (called Shannon total variation) can be easily handled with classical primal–dual formulations and illustrate its efficiency on several image processing tasks, including deblurring, spectrum extrapolation and a new aliasing reduction algorithm.

InterpoNet, a Brain Inspired Neural Network for Optical Flow Dense Interpolation

Abstract: Sparse-to-dense interpolation for optical flow is a fundamental phase in the pipeline of most of the leading optical flow estimation algorithms. The current state-of-the-art method for interpolation, EpicFlow, is a local average method based on an edge aware geodesic distance. We propose a new data-driven sparse-to-dense interpolation algorithm based on a fully convolutional network. We draw inspiration from the filling-in process in the visual cortex and introduce lateral dependencies between neurons and multi-layer supervision into our learning process. We also show the importance of the image contour to the learning process. Our method is robust and outperforms EpicFlow on competitive optical flow benchmarks with several underlying matching algorithms. This leads to state-of-the-art performance on the Sintel and KITTI 2012 benchmarks.

An image interpolation based reversible data hiding scheme using pixel value adjusting feature

Abstract: In this paper, we propose an image interpolation based reversible data hiding scheme using pixel value adjusting feature. This scheme consists of two phases, namely: image interpolation and data hiding. In order to interpolate the original image, we propose a new image interpolation method which is based on the existing neighbor mean interpolation method. Our interpolation method takes into account all the neighboring pixels like the NMI method. However, it uses different weight-age as per their proximity. Thus, it provides the better quality interpolated image. In case of data hiding phase, secret data is embedded in the interpolated pixels in two passes. In the first pass, it embeds the secret data into the odd valued pixels and then in the second pass, the even valued pixels are used to embed the secret data. To ensure the reversibility of the proposed scheme, the location map is constructed for every pass. Basically, the proposed scheme only increases/decreases the pixel values during data hiding phase, which improves the performance of the proposed scheme in terms of computation complexity. Experimentally, our scheme is superior to the existing scheme in terms of data hiding capacity, image quality and computation complexity.

Visual Odometry for Pixel Processor Arrays

Abstract: We present an approach of estimating constrained egomotion on a Pixel Processor Array (PPA). These devices embed processing and data storage capability into the pixels of the image sensor, allowing for fast and low power parallel computation directly on the image-plane. Rather than the standard visual pipeline whereby whole images are transferred to an external general processing unit, our approach performs all computation upon the PPA itself, with the camera's estimated motion as the only information output. Our approach estimates 3D rotation and a 1D scale-less estimate of translation. We introduce methods of image scaling, rotation and alignment which are performed solely upon the PPA itself and form the basis for conducting motion estimation. We demonstrate the algorithms on a SCAMP-5 vision chip, achieving frame rates >1000Hz at ~2W power consumption.

Super-resolution convolutional neural network for the improvement of the image quality of magnified images in chest radiographs

Abstract: Single image super-resolution (SR) method can generate a high-resolution (HR) image from a low-resolution (LR) image by enhancing image resolution. In medical imaging, HR images are expected to have a potential to provide a more accurate diagnosis with the practical application of HR displays. In recent years, the super-resolution convolutional neural network (SRCNN), which is one of the state-of-the-art deep learning based SR methods, has proposed in computer vision. In this study, we applied and evaluated the SRCNN scheme to improve the image quality of magnified images in chest radiographs. For evaluation, a total of 247 chest X-rays were sampled from the JSRT database. The 247 chest X-rays were divided into 93 training cases with non-nodules and 152 test cases with lung nodules. The SRCNN was trained using the training dataset. With the trained SRCNN, the HR image was reconstructed from the LR one. We compared the image quality of the SRCNN and conventional image interpolation methods, nearest neighbor, bilinear and bicubic interpolations. For quantitative evaluation, we measured two image quality metrics, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). In the SRCNN scheme, PSNR and SSIM were significantly higher than those of three interpolation methods (p<0.001). Visual assessment confirmed that the SRCNN produced much sharper edge than conventional interpolation methods without any obvious artifacts. These preliminary results indicate that the SRCNN scheme significantly outperforms conventional interpolation algorithms for enhancing image resolution and that the use of the SRCNN can yield substantial improvement of the image quality of magnified images in chest radiographs.

Image interpolation based high capacity reversible data hiding scheme

Abstract: In this paper, we propose a new interpolation technique which considers all the neighboring pixels as well as their impact on the reference pixels to provide better quality interpolated image and a new data hiding scheme which embeds the secret data in the interpolated pixels by taking into account the human visual system so that quality of the resultant image is maintained. The proposed interpolation technique is an improvement of the existing neighbor mean interpolation (NMI) technique in such a way that the interpolated image would have more resemblance to the input image. The proposed interpolation technique has less computational cost like NMI as it does not perform much computation during estimation unlike B-Spline, Bilinear Interpolation etc. The proposed data hiding scheme comes into the category of reversible data hiding scheme as the input image can be reconstructed after extraction of the entire secret data at the receiver side. Thus, it reduces the communication cost. Furthermore, the proposed data hiding scheme identifies the smooth and complex regions of the interpolated (or cover) image by dividing the same into blocks. It then embeds more bits into the complex regions of the image so that data hiding capacity as well as the image quality can be enhanced. The experimental results shows that the percentage increment in the PSNR value and capacity of the proposed scheme with respect to Chang et al. method is in the range of 0.26 to 30.60% and 0.87 to 73.82%, respectively. Moreover, the modified NMI yields higher PSNRs than other interpolating methods such as NMI, BI, and ENMI.

An efficient reversible data hiding with reduplicated exploiting modification direction using image interpolation and edge detection

Abstract: Data hiding is a technique that embeds a cluster of secret messages into the original image. The image with secret messages can be distributed on the Internet while the message embedded would not be easily discovered by a third party. In this way, the secret message can be well protected. At the same time, an image might be made with complex and smooth textures. If changes are made on complex textures, it is less easy for human eyes to discern the differences in the image; but if changes are made on smooth textures, the changes are easier to be discerned by human eyes. This paper proposes a data hiding method based on reduplicated exploiting modification direction (REMD), image interpolation, and canny edge detection. It aims at fulfilling two goals. First, conduct difference-embedding on the image's feature information, distribute the image, and use image interpolation to accomplish reversibility. Second, check the pixels that are located at the edge and insert different data payload according to the application demands. This allows users to flexibly adjust data payload embedded into the image's edge pixels according to their practical conditions, effectively considering both the image quality and payload. The experimental results demonstrate that the proposed method can achieve the data payload of 3.01bpp, so this is a reversible hiding technique with a very high embedding capacity. In the mean time, the average image quality is kept at an acceptable level, about 33?±?1 dB.

An adaptive decision based interpolation scheme for the removal of high density salt and pepper noise in images

Abstract: An adaptive decision based inverse distance weighted interpolation (DBIDWI) algorithm for the elimination of high-density salt and pepper noise in images is proposed. The pixel is initially checked for salt and pepper noise. If classified as noisy pixel, replace it with an inverse distance weighted interpolation value. This interpolation estimates the values of corrupted pixels using the distance and values nearby non-noisy pixels in vicinity. Inverse distance weighted interpolation uses the contribution of non-noisy pixel to the interpolated value. The window size is varied adaptively depending upon the non-noisy content of the current processing window. The algorithm is tested on various images and found to exhibit good results both in terms of quantitative (PSNR, MSE, SSIM, Pratt’s FOM) and qualitative (visually) at high noise densities. The algorithm performs very well in restoring an image corrupted by high-density salt and pepper noise by preserving fine details of an image.

Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction.

Abstract: Image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction aims to recover detailed information from low-resolution images and reconstruct them into high-resolution images. Due to the limited amount of data and information retrieved from low-resolution images, it is difficult to restore clear, artifact-free images, while still preserving enough structure of the image such as the texture. This paper presents a new single image super-resolution method which is based on adaptive fractional-order gradient interpolation and reconstruction. The interpolated image gradient via optimal fractional-order gradient is first constructed according to the image similarity and afterwards the minimum energy function is employed to reconstruct the final high-resolution image. Fractional-order gradient based interpolation methods provide an additional degree of freedom which helps optimize the implementation quality due to the fact that an extra free parameter α-order is being used. The proposed method is able to produce a rich texture detail while still being able to maintain structural similarity even under large zoom conditions. Experimental results show that the proposed method performs better than current single image super-resolution techniques.

Fast and robust geometric correction for mosaicking UAV images with narrow overlaps

Abstract: Image mosaicking is essential for monitoring a wide target area using unmanned aircraft vehicle UAV images. An image mosaicking process requires accurate and robust geometric correction of individual images with respect to the reference plane. Tiepoint-based geometric correction methods developed so far usually assume wide overlaps between adjacent images. This article focuses on fast monitoring applications where UAVs fly very fast and image mosaics are to be generated immediately. In this case, wide overlaps might not be ensured. For this reason, we investigate a fast and robust geometric correction method for mosaicking UAV images with narrow overlaps. To ensure quickness in geometric correction, an image resampling approach using a resampling grid is presented. To ensure accuracy and robustness in geometric correction, existing transformation models are analysed in depth, and optimal models are proposed. Our proposed method shows the potential for fast monitoring applications. We also show that while existing transformation models work for images with a large overlap, perspective transformation models with full orientation parameters may suffer in images with a narrow overlap. We hope that our results can be useful when implementing an optimal solution that can simultaneously handle UAV images with different overlaps.

Comparative study of copy-move forgery detection techniques

Abstract: Digital images and their applications gained a huge interest around the world in several fields like newspapers, social media, defaming persons, and courts. There are two types of digital image authentication. The first type is active authentication, which uses digital signature and image watermarks. These techniques have certain constraints such as knowing the content of the digital image. They need special equipment like cameras and development software. The second type is passive authentication, which is used to detect digital image forgeries represented in image cloning, image splicing, image resampling, image retouching, and image morphing. Passive authentication has an advantage of not needing any previous knowledge of the image content to detect the forgery. Copy-move forgery is the most famous type, and it is widespread in all image forgeries. Copy-move forgery is easy to perform and the forged part has the same properties of the whole image that makes it difficult to detect. There are many algorithms used to detect copy-move forgery attacks depending on different techniques. This paper covers the directions of copy-move forgery detection and gives a wide coverage of earlier copy-move forgery detection algorithms and techniques.

Revealing Traces of Image Resampling and Resampling Antiforensics

Abstract: Image resampling is a common manipulation in image processing. The forensics of resampling plays an important role in image tampering detection, steganography, and steganalysis. In this paper, we proposed an effective and secure detector, which can simultaneously detect resampling and its forged resampling which is attacked by antiforensic schemes. We find that the interpolation operation used in the resampling and forged resampling makes these two kinds of image show different statistical behaviors from the unaltered images, especially in the high frequency domain. To reveal the traces left by the interpolation, we first apply multidirectional high-pass filters on an image and the residual to create multidirectional differences. Then, the difference is fit into an autoregressive (AR) model. Finally, the AR coefficients and normalized histograms of the difference are extracted as the feature. We assemble the feature extracted from each difference image to construct the comprehensive feature and feed it into support vector machines (SVM) to detect resampling and forged resampling. Experiments on a large image database show that the proposed detector is effective and secure. Compared with the state-of-the-art works, the proposed detector achieved significant improvements in the detection of downsampling or resampling under JPEG compression.

Deep learning for multisensor image resolution enhancement

Abstract: We describe a deep learning convolutional neural network (CNN) for enhancing low resolution multispectral satellite imagery without the use of a panchromatic image. For training, low resolution images are used as input and corresponding high resolution images are used as the target output (label). The CNN learns to automatically extract hierarchical features that can be used to enhance low resolution imagery. The trained network can then be effectively used for super-resolution enhancement of low resolution multispectral images where no corresponding high resolution image is available. The CNN enhances all four spectral bands of the low resolution image simultaneously and adjusts pixel values of the low resolution to match the dynamic range of the high resolution image. The CNN yields higher quality images than standard image resampling methods.

Quadratic trigonometric B-spline for image interpolation using GA.

Abstract: In this article, a new quadratic trigonometric B-spline with control parameters is constructed to address the problems related to two dimensional digital image interpolation. The newly constructed spline is then used to design an image interpolation scheme together with one of the soft computing techniques named as Genetic Algorithm (GA). The idea of GA has been formed to optimize the control parameters in the description of newly constructed spline. The Feature SIMilarity (FSIM), Structure SIMilarity (SSIM) and Multi-Scale Structure SIMilarity (MS-SSIM) indices along with traditional Peak Signal-to-Noise Ratio (PSNR) are employed as image quality metrics to analyze and compare the outcomes of approach offered in this work, with three of the present digital image interpolation schemes. The upshots show that the proposed scheme is better choice to deal with the problems associated to image interpolation.

Image demosaicking for Bayer-patterned CFA images using improved linear interpolation

Abstract: With the development of digital imaging technique, the demosaicking algorithm becomes a hot spot in the field of image processing. An efficient interpolation method with good visual quality and less calculation amount is urgently needed. In this paper, an improved linear interpolation for demosaicking of Bayer-patterned color filter array (CFA) images is proposed. Compared with bilinear interpolation, the proposed scheme gives full consideration to brightness information and edge information of the image. Since different color components need to be interpolated, different size linear filters with different gain parameters are designed. We correct the bilinear interpolation by a correction value to estimate the unknown color. Experimental results show that the improved scheme achieves better performance than other two methods both in subjective assessment and objective assessment. It incurs much fewer false colors in high-frequency regions. The zipper effect is also weakened to some extent. Besides, the improved scheme reduces the computational complexity. Due to good real-time adaptability, it is easily implemented in hardware.

Implementation of an IMU Aided Image Stacking Algorithm in a Digital Camera for Unmanned Aerial Vehicles.

Abstract: Images acquired with a long exposure time using a camera embedded on UAVs (Unmanned Aerial Vehicles) exhibit motion blur due to the erratic movements of the UAV. The aim of the present work is to be able to acquire several images with a short exposure time and use an image processing algorithm to produce a stacked image with an equivalent long exposure time. Our method is based on the feature point image registration technique. The algorithm is implemented on the light-weight IGN (Institut national de l’information geographique) camera, which has an IMU (Inertial Measurement Unit) sensor and an SoC (System on Chip)/FPGA (Field-Programmable Gate Array). To obtain the correct parameters for the resampling of the images, the proposed method accurately estimates the geometrical transformation between the first and the N-th images. Feature points are detected in the first image using the FAST (Features from Accelerated Segment Test) detector, then homologous points on other images are obtained by template matching using an initial position benefiting greatly from the presence of the IMU sensor. The SoC/FPGA in the camera is used to speed up some parts of the algorithm in order to achieve real-time performance as our ultimate objective is to exclusively write the resulting image to save bandwidth on the storage device. The paper includes a detailed description of the implemented algorithm, resource usage summary, resulting processing time, resulting images and block diagrams of the described architecture. The resulting stacked image obtained for real surveys does not seem visually impaired. An interesting by-product of this algorithm is the 3D rotation estimated by a photogrammetric method between poses, which can be used to recalibrate in real time the gyrometers of the IMU. Timing results demonstrate that the image resampling part of this algorithm is the most demanding processing task and should also be accelerated in the FPGA in future work.

Quantum multidimensional color image scaling using nearest-neighbor interpolation based on the extension of FRQI

Abstract: Reviewing past researches on quantum image scaling, only 2D images are studied. And, in a quantum system, the processing speed increases exponentially since parallel computation can be realized with superposition state when compared with classical computer. Consequently, this paper proposes quantum multidimensional color image scaling based on nearest-neighbor interpolation for the first time. Firstly, flexible representation of quantum images (FRQI) is extended to multidimensional color model. Meantime, the nearest-neighbor interpolation is extended to multidimensional color image and cycle translation operation is designed to perform scaling up operation. Then, the circuits are designed for quantum multidimensional color image scaling, including scaling up and scaling down, based on the extension of FRQI. In addition, complexity analysis shows that the circuits in the paper have lower complexity. Examples and simulation experiments are given to elaborate the procedure of quantum multidimensional scaling.

Variation learning guided convolutional network for image interpolation

Abstract: In this paper, we propose a variational learning model that effectively exploits the structural similarities for image representation, and construct a deep network based on this model for image interpolation. Based on the local dependency, our learning model represents an image as the three-dimensional features. Besides two coordinate dimensions, an additional neighboring variation dimension is added to encode every pixel as the variation to its nearest low-resolution pixel by the local similarity. This added dimension lowers the risk of over-fitting for learning approaches and constructs abundant structural correspondences for inferring the missing information lost in image degradation. Then, this three-dimensional features are naturally modeled, extracted and refined by an end-to-end trainable recurrent convolutional network for image interpolation. Comprehensive experiments demonstrate that our method leads to a surprisingly superior performance and offers new state-of-the-art benchmark.

Image processing system to detect objects of interest

Abstract: A method of detecting objects of interest in a vehicle image processing system comprising: a) capturing an image on a camera; b) providing a plurality of potential candidate windows by running a detection window at spatially different locations along said image, and repeating this at different image scaling relative to the detection window size; c) for each potential candidate window applying a candidate selection process adapted to select one or more candidates from said potential candidate windows; d) forwarding the candidates determined form step c) to a convolutional neural network (CNN) process; e) processing the candidates to identify objects of interest; characterized wherein the candidate input into the convolutional neural network (CNN) process have been resized by step b).