Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
TL;DR: In this article, the authors provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis, and provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI.
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
991 citations
TL;DR: This paper indicates how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction, and provides a starting point for people interested in experimenting and contributing to the field of deep learning for medical imaging.
Abstract: What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
590 citations
Cites background from "Image super-resolution"
...Image super-resolution, reconstructing a higher-resolution image or image sequence from the observed low-resolution image [190], is an exciting application of deep learning methods....
[...]
TL;DR: In this article, a test-time augmentation-based aleatoric uncertainty was proposed to analyze the effect of different transformations of the input image on the segmentation output, and the results showed that the proposed test augmentation provides a better uncertainty estimation than calculating the testtime dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions.
305 citations
TL;DR: The proposed integrated fusion framework can achieve the integrated fusion of multisource observations to obtain high spatio-temporal-spectral resolution images, without limitations on the number of remote sensing sensors.
Abstract: Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio–temporal–spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio–spectral fusion, and spatio–temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio–temporal–spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Systeme Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.
240 citations
TL;DR: This letter proposes a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs, elaborately designed with its “multifork” structure to learn multilevel representations ofRemote sensing images including both local details and global environmental priors.
Abstract: Super-resolution is an image processing technology that recovers a high-resolution image from a single or sequential low-resolution images Recently deep convolutional neural networks (CNNs) have made a huge breakthrough in many tasks including super-resolution In this letter, we propose a new single-image super-resolution algorithm named local–global combined networks (LGCNet) for remote sensing images based on the deep CNNs Our LGCNet is elaborately designed with its “multifork” structure to learn multilevel representations of remote sensing images including both local details and global environmental priors Experimental results on a public remote sensing data set (UC Merced) demonstrate an overall improvement of both accuracy and visual performance over several state-of-the-art algorithms
203 citations
Cites background from "Image super-resolution"
...Instead of devoting to physical imaging technology, many researchers aim to recover highresolution images from low-resolution ones using an image processing technology called super-resolution [1]....
[...]
References
More filters
29 Nov 2010
TL;DR: In this article, an improved variational level set method for the Chan-Vese model is proposed to drive level set function to become fast and stably close to signed distance function.
Abstract: In this paper, an improved variational level set method for the Chan-Vese model is proposed to drive level set function to become fast and stably close to signed distance function. A restriction item that is a nonlinear heat equation with balanced diffusion rate is added to the traditional Chan-Vese model, and therefore the costly re-initialization procedure is completely eliminated. The proposed variational level set formulation is implemented by numerical scheme with spatial rotation-invariance gradient and divergence operator. Consequently it computes more efficiently. The proposed algorithm has been applied to medical images with desired results.
10 citations
10 Dec 2002
TL;DR: A regularization functional is introduced, not only to reflect the relative amount of registration error in each low-resolution image, but also to determine the regularization parameter without any prior knowledge in the reconstruction procedure.
Abstract: The problem of recovering a high-resolution image from a sequence of low-resolution DCT-based compressed images is considered. The presence of the compression system complicates the recovery problem, as the operation reduces the amount of frequency aliasing in the low-resolution frames and introduces a non-linear quantization process, The effect of the quantization error and resulting inaccurate sub-pixel motion information is modeled as a zero-mean additive correlated Gaussian noise. A regularization functional is introduced, not only to reflect the relative amount of registration error in each low-resolution image, but also to determine the regularization parameter without any prior knowledge in the reconstruction procedure. The effectiveness of the proposed algorithm is demonstrated experimentally.
10 citations
TL;DR: In this article, a super-resolution approach was proposed to detect and map compositional variability using thermal infrared (TIR) data at scales below 100 m. The technique was applied to the THEMIS TIR and VIS datasets and demonstrated its ability using existing THEMIS IR and VIS data.
9 citations
21 Feb 2014
TL;DR: In this article, the authors reviewed the sub-pixel imaging technology principles, characteristics, the current development status at home and abroad and the latest research developments, and the application prospect is very extensive.
Abstract: This paper reviews the Sub-pixel imaging technology principles, characteristics, the current development status at home and abroad and the latest research developments. As Sub-pixel imaging technology has achieved the advantages of high resolution of optical remote sensor, flexible working ways and being miniaturized with no moving parts. The imaging system is suitable for the application of space remote sensor. Its application prospect is very extensive. It is quite possible to be the research development direction of future space optical remote sensing technology.
7 citations
"Image super-resolution" refers background or methods in this paper
...Among them, the resolution of the panchromatic image acquired by SPOT-5 can reach 2.5 m through the SR of two 5-m images obtained by shifting a double CCD array by half a sampling interval (Fig....
[...]
...In the French space agency's SPOT-5 satellite system, a specially developed CCD detector was used which packages two 12,000-pixel CCDs in one structure....
[...]
...3), which was the most successful case [27,192]....
[...]
...Leica ADS40 aerial cameras have adopted a similar imaging mechanism to SPOT-5 [27,28]....
[...]
01 Jan 2010
TL;DR: A novel iris segmentation algorithm is proposed which is robust in dealing with specular reflections and image blur and is also computationally efficient, and a probabilistic, learning-based approach where the system can learn about the pixel distribution from a training data set and create masks for a test data set in an efficient way.
Abstract: Iris recognition has been developing for over 20 years, but, only in recent years has it been more accessible and widely accepted as one of the most accurate and un-obtrusive biometric modalities. Over the past few years, many companies have developed iris acquisition systems that are more user-friendly. Iris-On-the-Move (IOM) is one such system, offering significant stand-off acquisition distance (3m), which is extremely convenient for users and very suitable for deployment at airports to check passenger identifications and to control access. However, iris images acquired by the IOM and other long range systems are, in most cases, considerably blurred, of low contrast, and lacking detail in the iris texture compared to images from very close proximity sensors (5cm stand-off). This thesis focuses on how to deal with the three most challenging problems in long-range iris recognition: (1) iris segmentation from long range systems, (2) automatic iris mask generation of occluded regions, (3) iris matching performance enhancement using multiple irises from a video sequence. In particular, we emphasize solutions in the context of the IOM system and those that take advantage of an iris image video stream.
If an image of the eye is clear and has strong contrast, it is very easy to find the boundaries of the pupil and iris. However, most images acquired by the long-range iris acquisition system are blurred and noisy, which is the first problem that we propose a solution to: iris segmentation. Even worse, there are always strong specular reflections either in the pupil or on the iris region, which increases the difficulty in achieving good iris segmentation results. For this problem, we propose a novel iris segmentation algorithm which is robust in dealing with specular reflections and image blur and is also computationally efficient.
For the second problem, automatic detection of iris occluded regions from eyelashes and specular reflections, we focus on estimating a mask for the iris texture in the polar coordinate system. Unlike most current methods, we propose a probabilistic, learning-based approach where the system can learn about the pixel distribution from a training data set and create masks for a test data set in an efficient way. We further search the parameter space of Gabor filters in order to optimize the features set that the proposed algorithm learns, for the purpose of minimizing global error rate for large-scale iris recognition.
Iris matching performance enhancement for images captured by long-range iris acquisition devices, the third problem we address, deals with iris images that are blurred, defocused, and noisy due to low quantum efficiency of long-range iris sensors imaging in near-Infrared wavelengths. By exploiting the multi-frame video capture of the long-range iris acquisition system, it is possible to enhance the recognition performance by improving the low-quality iris images acquired from the system using super-resolution methods designed specifically for irises. We show a comprehensive set of empirical results demonstrating the effectiveness of our proposed approach designed for the IOM system that also apply to any video sequence of iris images captured by long-range iris acquisition devices.
6 citations
"Image super-resolution" refers background in this paper
...SR is also important in biometric recognition, including resolution enhancement for faces [24,201,202], fingerprints [203], and iris images [65,204]....
[...]