Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
TL;DR: In this article, the authors provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis, and provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI.
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
991 citations
TL;DR: This paper indicates how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction, and provides a starting point for people interested in experimenting and contributing to the field of deep learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
590 citations
Cites background from "Image super-resolution"
...Image super-resolution, reconstructing a higher-resolution image or image sequence from the observed low-resolution image [190], is an exciting application of deep learning methods....
[...]
TL;DR: In this article, a test-time augmentation-based aleatoric uncertainty was proposed to analyze the effect of different transformations of the input image on the segmentation output, and the results showed that the proposed test augmentation provides a better uncertainty estimation than calculating the testtime dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions.
305 citations
TL;DR: The proposed integrated fusion framework can achieve the integrated fusion of multisource observations to obtain high spatio-temporal-spectral resolution images, without limitations on the number of remote sensing sensors.
Abstract: Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio–temporal–spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio–spectral fusion, and spatio–temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio–temporal–spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Systeme Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.
240 citations
TL;DR: This letter proposes a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs, elaborately designed with its “multifork” structure to learn multilevel representations ofRemote sensing images including both local details and global environmental priors.
Abstract: Super-resolution is an image processing technology that recovers a high-resolution image from a single or sequential low-resolution images Recently deep convolutional neural networks (CNNs) have made a huge breakthrough in many tasks including super-resolution In this letter, we propose a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs Our LGCNet is elaborately designed with its “multifork” structure to learn multilevel representations of remote sensing images including both local details and global environmental priors Experimental results on a public remote sensing data set (UC Merced) demonstrate an overall improvement of both accuracy and visual performance over several state-of-the-art algorithms
203 citations
Cites background from "Image super-resolution"
...Instead of devoting to physical imaging technology, many researchers aim to recover highresolution images from low-resolution ones using an image processing technology called super-resolution [1]....
[...]
References
More filters
TL;DR: A line-scanner image matching method is proposed which represents deformation by the quadric function along the camera motion direction and bases on the deformation model for a relief terrain’s imaging on sensors of the satellite borne three-line scanner camera.
Abstract: This article intends to solve the matching problem of 2C level lunar images by Chang’E-1 (CE-1) lunar probe satellite. A line-scanner image matching method is proposed which represents deformation by the quadric function along the camera motion direction and bases on the deformation model for a relief terrain’s imaging on sensors of the satellite borne three-line scanner camera. A precise matching is carried out for the normal view, the frontward view, and the backward view images of the CE-1 by combining the proposed method with the standard correlation method. A super-resolution (SR) reconstruction algorithm based on the wavelet interpolation of non-uniformly sampled data is also adopted to realize SR reconstruction of CE-1 lunar images, which adds the recognizable targets and explores CE-1 lunar images to the full.
14 citations
TL;DR: A Bayesian formulation is employed to incorporate the two regularizations into one Maximum a Posteriori (MAP) estimation model, where the HR image and the motion estimation can be refined progressively in an alternative and iterative manner.
Abstract: Video super-resolution (SR) is a process for reconstructing high-resolution (HR) images by utilizing complementary information among multiple low-resolution (LR) images. Accurate estimation of the motion among the LR images significantly affects the quality of the reconstructed HR image. In this paper, we analyze the possible reasons for the inaccuracy of motion estimation and then propose a multi-lateral filter to regularize the process of motion estimation. This filter can adaptively correct motion estimation according to the estimation reliability, image intensity discontinuity, and motion dissimilarity. Furthermore, we introduce a non-local prior to solve the ill-posed problem of HR image reconstruction. This prior can fully utilize the self-similarities existing in natural images to regularize the HR image reconstruction. Finally, we employ a Bayesian formulation to incorporate the two regularizations into one Maximum a Posteriori (MAP) estimation model, where the HR image and the motion estimation can be refined progressively in an alternative and iterative manner. In addition, an algorithm that estimates the blur kernel by analyzing edges in an image is also presented in this paper. Experimental results demonstrate that the proposed approaches are highly effective and compare favorably to state-of-the-art SR algorithms.
14 citations
06 Sep 2014
TL;DR: It is shown that recording multiple images, transformed in the octic group, with a sensor of asymmetric sub-pixel layout increases the spatial sampling compared to a conventional sensor with a rectilinear grid of pixels and hence increases the image resolution.
Abstract: This paper presents a novel super-resolution framework by exploring the properties of non-conventional pixel layouts and shapes. We show that recording multiple images, transformed in the octic group, with a sensor of asymmetric sub-pixel layout increases the spatial sampling compared to a conventional sensor with a rectilinear grid of pixels and hence increases the image resolution. We further prove a theoretical bound for achieving well-posed super-resolution with a designated magnification factor w.r.t. the number and distribution of sub-pixels. We also propose strategies for selecting good sub-pixel layouts and effective super-resolution algorithms for our setup. The experimental results validate the proposed theory and solution, which have the potential to guide the future CCD layout design with super-resolution functionality.
13 citations
"Image super-resolution" refers background or methods in this paper
...Since the two CCD detectors can capture images at the same time, a set of data can therefore be acquired at a half-pixel shift in the imaging position....
[...]
...Moreover, some CCD pixels comprise sub-pixels with different shapes and spatial locations [29]....
[...]
...Compared with CMOSs, CCDs have advantages in sensor sensitivity, imaging resolution, noise suppression and technology maturity [20]....
[...]
...Furthermore, nonrectangular pixel layouts, as in the hexagonal Fujifilm super CCD and the orthogonal-transfer CCD [18,19], have been used to increase the spatial sampling rate, as shown in Fig....
[...]
...Among them, the resolution of the panchromatic image acquired by SPOT-5 can reach 2.5 m through the SR of two 5-m images obtained by shifting a double CCD array by half a sampling interval (Fig....
[...]
TL;DR: The proposed algorithm constructs the superresolution bands corresponding to the low-resolution bands to enhance the resolution using a proposed local enhancement technique based on the least squares regression and the local correlation to improve the estimated interpolation of the spatial resolution.
Abstract: Hyperspectral imagery is used for a wide variety of applications, including target detection, tracking, agricultural monitoring, and natural resources exploration. The main reason for using hyperspectral imagery is that images reveal spectral information about the scene that is not available in a single band. Unfortunately, many factors, such as the limitations of focal plane array technology, the inherent tradeoff in spatial versus spectral resolution, and the desire to achieve area coverage, degrade the spatial quality of these images. Recently, many algorithms are introduced in the literature to improve the resolution of hyperspectral images using coregistered high spatial resolution imagery such as panchromatic imagery. In this letter, we propose a new algorithm to enhance the spatial resolution of low-resolution hyperspectral bands using strongly correlated and coregistered high spatial resolution panchromatic imagery. The proposed algorithm constructs the superresolution bands corresponding to the low-resolution bands to enhance the resolution using a proposed local enhancement technique. The local enhancement is based on the least squares regression and the local correlation to improve the estimated interpolation of the spatial resolution. The introduced algorithm is considered as an improvement for Price's algorithm which uses the global correlation for the spatial resolution enhancement. In addition, numerous studies are conducted to investigate the effect of spatial window size for achieving the local enhancement in the estimation process. The experimental results, which are obtained using hyperspectral data derived from an airborne imaging sensor, are presented to verify the improvement by the proposed algorithm.
13 citations
TL;DR: The connection of concepts from structured dyadic-tree complexity measures, wavelet shrinkage, morphological wavelets, and smoothness regularization in Besov space into a single coherent image regularization framework is connection.
Abstract: Despite the tremendous success of wavelet-based image regularization, we still lack a comprehensive understanding of the exact factor that controls edge preservation and a principled method to determine the wavelet decomposition structure for dimensions greater than 1. We address these issues from a machine learning perspective by using tree classifiers to underpin a new image regularizer that measures the complexity of an image based on the complexity of the dyadic-tree representations of its sublevel sets. By penalizing unbalanced dyadic trees less, the regularizer preserves sharp edges. The main contribution of this paper is the connection of concepts from structured dyadic-tree complexity measures, wavelet shrinkage, morphological wavelets, and smoothness regularization in Besov space into a single coherent image regularization framework. Using the new regularizer, we also provide a theoretical basis for the data-driven selection of an optimal dyadic wavelet decomposition structure. As a specific application example, we give a practical regularized image denoising algorithm that uses this regularizer and the optimal dyadic wavelet decomposition structure.
13 citations
"Image super-resolution" refers background in this paper
...In addition to the above regularizations, there have been many other studies of prior models, such as regularization based on sparsity [117], along with morphological theory [98]....
[...]