Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
TL;DR: In this article, the authors provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis, and provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI.
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
991 citations
TL;DR: This paper indicates how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction, and provides a starting point for people interested in experimenting and contributing to the field of deep learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
590 citations
Cites background from "Image super-resolution"
...Image super-resolution, reconstructing a higher-resolution image or image sequence from the observed low-resolution image [190], is an exciting application of deep learning methods....
[...]
TL;DR: In this article, a test-time augmentation-based aleatoric uncertainty was proposed to analyze the effect of different transformations of the input image on the segmentation output, and the results showed that the proposed test augmentation provides a better uncertainty estimation than calculating the testtime dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions.
305 citations
TL;DR: The proposed integrated fusion framework can achieve the integrated fusion of multisource observations to obtain high spatio-temporal-spectral resolution images, without limitations on the number of remote sensing sensors.
Abstract: Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio–temporal–spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio–spectral fusion, and spatio–temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio–temporal–spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Systeme Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.
240 citations
TL;DR: This letter proposes a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs, elaborately designed with its “multifork” structure to learn multilevel representations ofRemote sensing images including both local details and global environmental priors.
Abstract: Super-resolution is an image processing technology that recovers a high-resolution image from a single or sequential low-resolution images Recently deep convolutional neural networks (CNNs) have made a huge breakthrough in many tasks including super-resolution In this letter, we propose a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs Our LGCNet is elaborately designed with its “multifork” structure to learn multilevel representations of remote sensing images including both local details and global environmental priors Experimental results on a public remote sensing data set (UC Merced) demonstrate an overall improvement of both accuracy and visual performance over several state-of-the-art algorithms
203 citations
Cites background from "Image super-resolution"
...Instead of devoting to physical imaging technology, many researchers aim to recover highresolution images from low-resolution ones using an image processing technology called super-resolution [1]....
[...]
References
More filters
14 Nov 2005
TL;DR: It is shown that the usual iterative-warping scheme encounters divergence problems and a modified scheme with better behavior is proposed, which yields good results with a much lower cost than the exact dense LK algorithm, on simulated and real sequences.
Abstract: We study dense optical flow estimation using iterative registration of local window, also known as iterative Lucas-Kanade (LK) [B. Lucas et al, 1981]. We show that the usual iterative-warping scheme encounters divergence problems and propose a modified scheme with better behavior. It yields good results with a much lower cost than the exact dense LK algorithm, on simulated and real sequences.
78 citations
Additional excerpts
...However, the optical flow based methods are computationally expensive [141] and are sensitive to noise, large displacements, and illumination variation [142]....
[...]
06 May 2001
TL;DR: Current research is focused on simultaneous blur identification and robust superresolution, and a formulation involving a periodically shift-variant system model is proposed.
Abstract: Subsequent to the work of Kim, Bose, and Valenzuela in 1990 on the simultaneous filtering and interpolation of a registered sequence of undersampled noisy and shift-invariant blur degraded images, Bose and Boo tackled in 1998 the problem of reconstructing a high-resolution image from multiple undersampled, shifted, degraded frames with subpixel displacement errors. This led to a formulation involving a periodically shift-variant system model. Lertrattanapanich and Bose advanced in 1999 a procedure for construction of a high-resolution video mosaic following the estimation of motion parameters between successive frames in a video sequence generated from a video camera. Current research is focused on simultaneous blur identification and robust superresolution. The blur is not restricted to be linear shift-invariant and could not only be of the linear shift-variant type but also some nonlinear blurs could be accommodated. The optimal tuning parameter may, if desired, be calculated analytically and not by trial-and-error.
78 citations
"Image super-resolution" refers background or methods in this paper
...The popular methods include the L-curve method [124], generalized cross-validation (GCV) [35], and the U-curve method [49]....
[...]
...Further details can be found in the related works [49,124]....
[...]
07 Oct 2014
TL;DR: The SkySat-1 mission as discussed by the authors is the first microsatellite-class commercial earth-observation system to generate sub-meter resolution panchromatic imagery, in addition to sub-metric resolution 4-band pan-sharpened imagery.
Abstract: This paper presents details of the SkySat-1 mission, which is the first microsatellite-class commercial earth- observation system to generate sub-meter resolution panchromatic imagery, in addition to sub-meter resolution 4-band pan-sharpened imagery. SkySat-1 was built and launched for an order of magnitude lower cost than similarly performing missions. The low-cost design enables the deployment of a large imaging constellation that can provide imagery with both high temporal resolution and high spatial resolution. One key enabler of the SkySat-1 mission was simplifying the spacecraft design and instead relying on ground- based image processing to achieve high-performance at the system level. The imaging instrument consists of a custom-designed high-quality optical telescope and commercially-available high frame rate CMOS image sen- sors. While each individually captured raw image frame shows moderate quality, ground-based image processing algorithms improve the raw data by combining data from multiple frames to boost image signal-to-noise ratio (SNR) and decrease the ground sample distance (GSD) in a process Skybox calls digital TDI". Careful qual-ity assessment and tuning of the spacecraft, payload, and algorithms was necessary to generate high-quality panchromatic, multispectral, and pan-sharpened imagery. Furthermore, the framing sensor configuration en- abled the first commercial High-Definition full-frame rate panchromatic video to be captured from space, with approximately 1 meter ground sample distance. Details of the SkySat-1 imaging instrument and ground-based image processing system are presented, as well as an overview of the work involved with calibrating and validating the system. Examples of raw and processed imagery are shown, and the raw imagery is compared to pre-launch simulated imagery used to tune the image processing algorithms.
77 citations
TL;DR: A novel dictionary training method for sparse reconstruction for enhancing the similarity of sparse representations between the low resolution and high resolution MRI block pairs through simultaneous training two dictionaries.
73 citations
TL;DR: Adapt weighted super-resolution reconstruction is proposed to alleviate the limitations of the different resolutions of the multi-angle remote sensing images by using the angle between the imaging angle of the current image and that of the nadir image.
Abstract: Multi-angle remote sensing images are acquired over the same imaging scene from different angles, and share similar but not identical information. It is therefore possible to enhance the spatial resolution of the multi-angle remote sensing images by the super-resolution reconstruction technique. However, different sensor shooting angles lead to different resolutions for each angle image, which affects the effectiveness of the super-resolution reconstruction of the multi-angle images. In view of this, we propose utilizing adaptive weighted super-resolution reconstruction to alleviate the limitations of the different resolutions. This paper employs two adaptive weighting themes. The first approach uses the angle between the imaging angle of the current image and that of the nadir image. The second is closely related to the residual error of each low-resolution angle image. The experimental results confirm the feasibility of the proposed method and demonstrate the effectiveness of the proposed adaptive weighted super-resolution approach.
73 citations