Showing papers on "Image resolution published in 2016"
27 Jun 2016
TL;DR: This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
Abstract: Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods.
4,770 citations
TL;DR: A novel 10-m spatial resolution MNDWI is produced from Sentinel-2 images by downscaling the 20-m resolution SWIR band to 10 m based on pan-sharpening, which shows that MND WI can enhance water bodies and suppressbuilt-up features more efficiently than NDWI.
Abstract: Monitoring open water bodies accurately is an important and basic application in remote sensing. Various water body mapping approaches have been developed to extract water bodies from multispectral images. The method based on the spectral water index, especially the Modified Normalized Difference Water Index (MDNWI) calculated from the green and Shortwave-Infrared (SWIR) bands, is one of the most popular methods. The recently launched Sentinel-2 satellite can provide fine spatial resolution multispectral images. This new dataset is potentially of important significance for regional water bodies’ mapping, due to its free access and frequent revisit capabilities. It is noted that the green and SWIR bands of Sentinel-2 have different spatial resolutions of 10 m and 20 m, respectively. Straightforwardly, MNDWI can be produced from Sentinel-2 at the spatial resolution of 20 m, by upscaling the 10-m green band to 20 m correspondingly. This scheme, however, wastes the detailed information available at the 10-m resolution. In this paper, to take full advantage of the 10-m information provided by Sentinel-2 images, a novel 10-m spatial resolution MNDWI is produced from Sentinel-2 images by downscaling the 20-m resolution SWIR band to 10 m based on pan-sharpening. Four popular pan-sharpening algorithms, including Principle Component Analysis (PCA), Intensity Hue Saturation (IHS), High Pass Filter (HPF) and A Trous Wavelet Transform (ATWT), were applied in this study. The performance of the proposed method was assessed experimentally using a Sentinel-2 image located at the Venice coastland. In the experiment, six water indexes, including 10-m NDWI, 20-m MNDWI and 10-m MNDWI, produced by four pan-sharpening algorithms, were compared. Three levels of results, including the sharpened images, the produced MNDWI images and the finally mapped water bodies, were analysed quantitatively. The results showed that MNDWI can enhance water bodies and suppressbuilt-up features more efficiently than NDWI. Moreover, 10-m MNDWIs produced by all four pan-sharpening algorithms can represent more detailed spatial information of water bodies than 20-m MNDWI produced by the original image. Thus, MNDWIs at the 10-m resolution can extract more accurate water body maps than 10-m NDWI and 20-m MNDWI. In addition, although HPF can produce more accurate sharpened images and MNDWI images than the other three benchmark pan-sharpening algorithms, the ATWT algorithm leads to the best 10-m water bodies mapping results. This is no necessary positive connection between the accuracy of the sharpened MNDWI image and the map-level accuracy of the resultant water body maps.
498 citations
TL;DR: A miniature flat camera integrating a monolithic metasurface lens doublet corrected for monochromatic aberrations, and an image sensor is demonstrated with nearly diffraction-limited image quality, indicating the potential of this technology in the development of optical systems for microscopy, photography, and computer vision.
Abstract: Optical metasurfaces are two-dimensional arrays of nano-scatterers that modify optical wavefronts at subwavelength spatial resolution They are poised to revolutionize optics by enabling complex low-cost systems where multiple metasurfaces are lithographically stacked and integrated with electronics For imaging applications, metasurface stacks can perform sophisticated image corrections and can be directly integrated with image sensors Here we demonstrate this concept with a miniature flat camera integrating a monolithic metasurface lens doublet corrected for monochromatic aberrations, and an image sensor The doublet lens, which acts as a fisheye photographic objective, has a small f-number of 09, an angle-of-view larger than 60° × 60°, and operates at 850 nm wavelength with 70% focusing efficiency The camera exhibits nearly diffraction-limited image quality, which indicates the potential of this technology in the development of optical systems for microscopy, photography, and computer vision
495 citations
TL;DR: Shake-The-Box as discussed by the authors is a Lagrangian tracking method that uses a prediction of the particle distribution for the subsequent time-step as a mean to seize the temporal domain.
Abstract: A Lagrangian tracking method is introduced, which uses a prediction of the particle distribution for the subsequent time-step as a mean to seize the temporal domain. Errors introduced by the prediction process are corrected by an image matching technique (‘shaking’ the particle in space), followed by an iterative triangulation of particles newly entering the measurement domain. The scheme was termed ‘Shake-The-Box’ and previously characterized as ‘4D-PTV’ due to the strong interaction with the temporal dimension. Trajectories of tracer particles are identified at high spatial accuracy due to a nearly complete suppression of ghost particles; a temporal filtering scheme further improves on accuracy and allows for the extraction of local velocity and acceleration as derivatives of a continuous function. Exploiting the temporal information enables the processing of densely seeded flows (beyond 0.1 particles per pixel, ppp), which were previously reserved for tomographic PIV evaluations. While TOMO-PIV uses statistical means to evaluate the flow (building an ‘anonymous’ voxel space with subsequent spatial averaging of the velocity information using correlation), the Shake-The-Box approach is able to identify and track individual particles at numbers of tens or even hundreds of thousands per time-step. The method is outlined in detail, followed by descriptions of applications to synthetic and experimental data. The synthetic data evaluation reveals that STB is able to capture virtually all true particles, while effectively suppressing the formation of ghost particles. For the examined four-camera set-up particle image densities N
I up to 0.125 ppp could be processed. For noise-free images, the attained accuracy is very high. The addition of synthetic noise reduces usable particle image density (N
I ≤ 0.075 ppp for highly noisy images) and accuracy (still being significantly higher compared to tomographic reconstruction). The solutions remain virtually free of ghost particles. Processing an experimental data set on a transitional jet in water demonstrates the benefits of advanced Lagrangian evaluation in describing flow details—both on small scales (by the individual tracks) and on larger structures (using an interpolation onto an Eulerian grid). Comparisons to standard TOMO-PIV processing for synthetic and experimental evaluations show distinct benefits in local accuracy, completeness of the solution, ghost particle occurrence, spatial resolution, temporal coherence and computational effort.
450 citations
11 Nov 2016
TL;DR: In this paper, a learning-based approach is proposed to synthesize new views from a sparse set of input views using two sequential convolutional neural networks to model disparity and color estimation components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images.
Abstract: With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in either spatial or angular domain. In this paper, we use machine learning to mitigate this trade-off. Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views. We build upon existing view synthesis techniques and break down the process into disparity and color estimation components. We use two sequential convolutional neural networks to model these two components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images. We show the performance of our approach using only four corner sub-aperture views from the light fields captured by the Lytro Illum camera. Experimental results show that our approach synthesizes high-quality images that are superior to the state-of-the-art techniques on a variety of challenging real-world scenes. We believe our method could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.
435 citations
TL;DR: This paper proposes a novel learning-based approach to synthesize new views from a sparse set of input views that could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.
Abstract: With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in either spatial or angular domain. In this paper, we use machine learning to mitigate this trade-off. Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views. We build upon existing view synthesis techniques and break down the process into disparity and color estimation components. We use two sequential convolutional neural networks to model these two components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images. We show the performance of our approach using only four corner sub-aperture views from the light fields captured by the Lytro Illum camera. Experimental results show that our approach synthesizes high-quality images that are superior to the state-of-the-art techniques on a variety of challenging real-world scenes. We believe our method could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.
427 citations
TL;DR: A new spatiotemporal data fusion method that uses simple principles and needs only one fine-resolution image as input has the potential to increase the availability of high-resolution time-series data that can support studies of rapid land surface dynamics.
418 citations
TL;DR: A modified time-of-flight three-dimensional imaging system, which can use compressed sensing techniques to reduce acquisition times, whilst distributing the optical illumination over the full field of view, is shown.
Abstract: A three-dimensional imaging system which distributes the optical illumination over the full field-of-view is sought after. Here, the authors demonstrate the capability of reconstructing 128 × 128 pixel resolution three-dimensional scenes to an accuracy of 3 mm as well as real-time video with a frame-rate up to 12 Hz.
409 citations
TL;DR: A new hyperspectral image super-resolution method from a low-resolution (LR) image and a HR reference image of the same scene to improve the accuracy of non-negative sparse coding and to exploit the spatial correlation among the learned sparse codes.
Abstract: Hyperspectral imaging has many applications from agriculture and astronomy to surveillance and mineralogy. However, it is often challenging to obtain high-resolution (HR) hyperspectral images using existing hyperspectral imaging techniques due to various hardware limitations. In this paper, we propose a new hyperspectral image super-resolution method from a low-resolution (LR) image and a HR reference image of the same scene. The estimation of the HR hyperspectral image is formulated as a joint estimation of the hyperspectral dictionary and the sparse codes based on the prior knowledge of the spatial-spectral sparsity of the hyperspectral image. The hyperspectral dictionary representing prototype reflectance spectra vectors of the scene is first learned from the input LR image. Specifically, an efficient non-negative dictionary learning algorithm using the block-coordinate descent optimization technique is proposed. Then, the sparse codes of the desired HR hyperspectral image with respect to learned hyperspectral basis are estimated from the pair of LR and HR reference images. To improve the accuracy of non-negative sparse coding, a clustering-based structured sparse coding method is proposed to exploit the spatial correlation among the learned sparse codes. The experimental results on both public datasets and real LR hypspectral images suggest that the proposed method substantially outperforms several existing HR hyperspectral image recovery techniques in the literature in terms of both objective quality metrics and computational efficiency.
404 citations
08 Oct 2016
TL;DR: This paper proposes a learning-based approach to construct a joint filter based on Convolutional Neural Networks that can selectively transfer salient structures that are consistent in both guidance and target images and validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.
Abstract: Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or hand-designed objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.
267 citations
TL;DR: This work presents fairSIM, an easy-to-use plugin that provides SR-SIM reconstructions for a wide range ofSR-SIM platforms directly within ImageJ, and can easily be adapted, automated and extended as the field of SR- SIM progresses.
Abstract: Super-resolved structured illumination microscopy (SR-SIM) is an important tool for fluorescence microscopy. SR-SIM microscopes perform multiple image acquisitions with varying illumination patterns, and reconstruct them to a super-resolved image. In its most frequent, linear implementation, SR-SIM doubles the spatial resolution. The reconstruction is performed numerically on the acquired wide-field image data, and thus relies on a software implementation of specific SR-SIM image reconstruction algorithms. We present fairSIM, an easy-to-use plugin that provides SR-SIM reconstructions for a wide range of SR-SIM platforms directly within ImageJ. For research groups developing their own implementations of super-resolution structured illumination microscopy, fairSIM takes away the hurdle of generating yet another implementation of the reconstruction algorithm. For users of commercial microscopes, it offers an additional, in-depth analysis option for their data independent of specific operating systems. As a modular, open-source solution, fairSIM can easily be adapted, automated and extended as the field of SR-SIM progresses.
TL;DR: The proposed integrated fusion framework can achieve the integrated fusion of multisource observations to obtain high spatio-temporal-spectral resolution images, without limitations on the number of remote sensing sensors.
Abstract: Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio–temporal–spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio–spectral fusion, and spatio–temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio–temporal–spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Systeme Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.
TL;DR: A plasmonic filter set with polarization-switchable color properties, based upon arrays of asymmetric cross-shaped nanoapertures in an aluminum thin-film, used to create micro image displays containing duality in their optical information states.
Abstract: Color filters based upon nanostructured metals have garnered significant interest in recent years, having been positioned as alternatives to the organic dye-based filters which provide color selectivity in image sensors, as nonfading “printing” technologies for producing images with nanometer pixel resolution, and as ultra-high-resolution, small foot-print optical storage and encoding solutions. Here, we demonstrate a plasmonic filter set with polarization-switchable color properties, based upon arrays of asymmetric cross-shaped nanoapertures in an aluminum thin-film. Acting as individual color-emitting nanopixels, the plasmonic cavity-apertures have dual-color selectivity, transmitting one of two visible colors, controlled by the polarization of the white light incident on the rear of the pixel and tuned by varying the critical dimensions of the geometry and periodicity of the array. This structural approach to switchable optical filtering enables a single nanoaperture to encode two information states wi...
TL;DR: This paper proposes a new edge preserving image fusion method for infrared and visible sensor images that outperforms the existing methods and is compared with the traditional and recent image fusion algorithms.
Abstract: Image fusion is a process of generating a more informative image from a set of source images. Major applications of image fusion are in navigation and military. Here, infrared and visible sensors are used to capture complementary images of the targeted scene. The complementary information of these source images has to be integrated into a single image using some fusion algorithms. The aim of any fusion method is to transfer maximum information from the source images to the fused image with a minimum information loss. It has to minimize the artifacts in the fused image. In this paper, we propose a new edge preserving image fusion method for infrared and visible sensor images. Anisotropic diffusion is used to decompose the source images into approximation and detail layers. Final detail and approximation layers are calculated with the help of Karhunen-Loeve transform and weighted linear superposition, respectively. A fused image is generated from the linear combination of final detail and approximation layers. Performance of the proposed algorithm is assessed with the help of petrovic metrics. The results of the proposed algorithm are compared with the traditional and recent image fusion algorithms. Results reveal that the proposed method outperforms the existing methods.
20 Aug 2016
TL;DR: Unlike competing methods, this technique quantitatively measures the volumetric refractive index of primarily transparent and contiguous sample features without the need for interferometry or any moving parts.
Abstract: This paper presents a technique to image the complex index of refraction of a sample across three dimensions. The only required hardware is a standard microscope and an array of LEDs. The method, termed Fourier ptychographic tomography (FPT), first captures a sequence of intensity-only images of a sample under angularly varying illumination. Then, using principles from ptychography and diffraction tomography, it computationally solves for the sample structure in three dimensions. The experimental microscope demonstrates a lateral spatial resolution of 0.39 μm and an axial resolution of 3.7 μm at the Nyquist–Shannon sampling limit (0.54 and 5.0 μm at the Sparrow limit, respectively) across a total imaging depth of 110 μm. Unlike competing methods, this technique quantitatively measures the volumetric refractive index of primarily transparent and contiguous sample features without the need for interferometry or any moving parts. Wide field-of-view reconstructions of thick biological specimens suggest potential applications in pathology and developmental biology.
TL;DR: In this article, scattered light images of the TW Hya disk performed with SPHERE in PDI mode at 063, 079, 124 and 162 micron were presented.
Abstract: We present scattered light images of the TW Hya disk performed with SPHERE in PDI mode at 063, 079, 124 and 162 micron We also present H2/H3-band ADI observations Three distinct radial depressions in the polarized intensity distribution are seen, around 85, 21, and 6~au The overall intensity distribution has a high degree of azimuthal symmetry; the disk is somewhat brighter than average towards the South and darker towards the North-West The ADI observations yielded no signifiant detection of point sources in the disk
Our observations have a linear spatial resolution of 1 to 2au, similar to that of recent ALMA dust continuum observations The sub-micron sized dust grains that dominate the light scattering in the disk surface are strongly coupled to the gas We created a radiative transfer disk model with self-consistent temperature and vertical structure iteration and including grain size-dependent dust settling This method may provide independent constraints on the gas distribution at higher spatial resolution than is feasible with ALMA gas line observations
We find that the gas surface density in the "gaps" is reduced by 50% to 80% relative to an unperturbed model Should embedded planets be responsible for carving the gaps then their masses are at most a few 10 Mearth The observed gaps are wider, with shallower flanks, than expected for planet-disk interaction with such low-mass planets If forming planetary bodies have undergone collapse and are in the "detachted phase" then they may be directly observable with future facilities such as METIS at the E-ELT
TL;DR: It was shown that the use of SAR imagery allows to use optical data without gap-filling yielding results which are equivalent to theUse of gap- filling in the case of perfect cloud screening, and better results in the cases of cloud screening errors.
Abstract: High temporal and spatial resolution optical image time series have been proven efficient for crop type mapping at the end of the agricultural season. However, due to cloud cover and image availability, crop identification earlier in the season is difficult. The recent availability of high temporal and spatial resolution SAR image time series, opens the possibility of improving early crop type mapping. This paper studies the impact of such SAR image time series when used in complement of optical imagery. The pertinent SAR image features, the optimal working resolution, the effect of speckle filtering and the use of temporal gap-filling of the optical image time series are assessed. SAR image time series as those provided by the Sentinel-1 satellites allow significant improvements in terms of land cover classification, both in terms of accuracy at the end of the season and for early crop identification. Haralik textures (Entropy, Inertia), the polarization ratio and the local mean together with the VV imagery were found to be the most pertinent features. Working at at 10 m resolution and using speckle filtering yield better results than other configurations. Finally it was shown that the use of SAR imagery allows to use optical data without gap-filling yielding results which are equivalent to the use of gap-filling in the case of perfect cloud screening, and better results in the case of cloud screening errors.
Posted Content•
TL;DR: A novel, simple approach to non-parametric vignette calibration, which requires minimal set-up and is easy to reproduce and thoroughly evaluate two existing methods (ORB-SLAM and DSO) on the dataset.
Abstract: We present a dataset for evaluating the tracking accuracy of monocular visual odometry and SLAM methods. It contains 50 real-world sequences comprising more than 100 minutes of video, recorded across dozens of different environments -- ranging from narrow indoor corridors to wide outdoor scenes. All sequences contain mostly exploring camera motion, starting and ending at the same position. This allows to evaluate tracking accuracy via the accumulated drift from start to end, without requiring ground truth for the full sequence. In contrast to existing datasets, all sequences are photometrically calibrated. We provide exposure times for each frame as reported by the sensor, the camera response function, and dense lens attenuation factors. We also propose a novel, simple approach to non-parametric vignette calibration, which requires minimal set-up and is easy to reproduce. Finally, we thoroughly evaluate two existing methods (ORB-SLAM and DSO) on the dataset, including an analysis of the effect of image resolution, camera field of view, and the camera motion direction.
TL;DR: Downscaled ten-band 10 m Sentinel-2 datasets represent important and promising products for a wide range of applications in remote sensing and have potential for blending with the upcoming Sentinel-3 data for fine spatio-temporal resolution monitoring at the global scale.
TL;DR: In this paper, the performance of the EUDET-type beam telescopes originally developed within the DESY project was evaluated using test beam measurements at the test beam facilities of DESY and the mean intrinsic resolution over the six sensors used was found to be (3.24 ± 0.09) µm.
Abstract: Test beam measurements at the test beam facilities of DESY have been conducted to characterise the performance of the EUDET-type beam telescopes originally developed within the EUDET project. The beam telescopes are equipped with six sensor planes using MIMOSA 26 monolithic active pixel devices. A programmable Trigger Logic Unit provides trigger logic and time stamp information on particle passage. Both data acquisition framework and offline reconstruction software packages are available. User devices are easily integrable into the data acquisition framework via predefined interfaces. The biased residual distribution is studied as a function of the beam energy, plane spacing and sensor threshold. Its standard deviation at the two centre pixel planes using all six planes for tracking in a 6 GeV electron/positron-beam is measured to be (2.88 ± 0.08) µm. Iterative track fits using the formalism of General Broken Lines are performed to estimate the intrinsic resolution of the individual pixel planes. The mean intrinsic resolution over the six sensors used is found to be (3.24 ± 0.09) µm. With a 5 GeV electron/positron beam, the track resolution halfway between the two inner pixel planes using an equidistant plane spacing of 20 mm is estimated to (1.83 ± 0.03) µm assuming the measured intrinsic resolution. Towards lower beam energies the track resolution deteriorates due to increasing multiple scattering. Threshold studies show an optimal working point of the MIMOSA 26 sensors at a sensor threshold of between five and six times their RMS noise. Measurements at different plane spacings are used to calibrate the amount of multiple scattering in the material traversed and allow for corrections to the predicted angular scattering for electron beams.
TL;DR: The potential and limitations of tracking glacier flow from repeat Sentinel-2 data are characterized using a set of typical glaciers in different environments, finding a high radiometric performance, but with slight striping effects under certain conditions.
Abstract: With its temporal resolution of 10 days (five days with two satellites, and significantly more at high latitudes), its swath width of 290 km, and its 10 m and 20 m spatial resolution bands from the visible to the shortwave infrared, the European Sentinel-2 satellites have significant potential for glacier remote sensing, in particular mapping of glacier outlines and facies, and velocity measurements. Testing Level 1C commissioning and ramp-up phase data for initial sensor quality experiences, we find a high radiometric performance, but with slight striping effects under certain conditions. Through co-registration of repeat Sentinal-2 data we also find lateral offset patterns and noise on the order of a few metres. Neither of these issues will complicate most typical glaciological applications. Absolute geo-location of the data investigated was on the order of one pixel at the time of writing. The most severe geometric problem stems from vertical errors of the DEM used for ortho-rectifying Sentinel-2 data. These errors propagate into locally varying lateral offsets in the images, up to several pixels with respect to other georeferenced data, or between Sentinel-2 data from different orbits. Finally, we characterize the potential and limitations of tracking glacier flow from repeat Sentinel-2 data using a set of typical glaciers in different environments: Aletsch Glacier, Swiss Alps; Fox Glacier, New Zealand; Jakobshavn Isbree, Greenland; Antarctic Peninsula at the Larsen C ice shelf.
TL;DR: A novel framework for the single depth image superresolution is proposed that is guided by a high-resolution edge map, which is constructed from the edges of the low-resolution depth image through a Markov random field optimization in a patch synthesis based manner.
Abstract: Recently, consumer depth cameras have gained significant popularity due to their affordable cost. However, the limited resolution and the quality of the depth map generated by these cameras are still problematic for several applications. In this paper, a novel framework for the single depth image superresolution is proposed. In our framework, the upscaling of a single depth image is guided by a high-resolution edge map, which is constructed from the edges of the low-resolution depth image through a Markov random field optimization in a patch synthesis based manner. We also explore the self-similarity of patches during the edge construction stage, when limited training data are available. With the guidance of the high-resolution edge map, we propose upsampling the high-resolution depth image through a modified joint bilateral filter. The edge-based guidance not only helps avoiding artifacts introduced by direct texture prediction, but also reduces jagged artifacts and preserves the sharp edges. Experimental results demonstrate the effectiveness of our method both qualitatively and quantitatively compared with the state-of-the-art methods.
TL;DR: This work proposes and experimentally demonstrate an ultimately optimized distributed fiber sensor capable of resolving 2100000 independent points, which corresponds to a one-order-of-magnitude improvement compared to the state- of-the-art.
Abstract: Distributed fiber sensing possesses the unique ability to measure the distributed profile of an environmental quantity along many tens of kilometers with spatial resolutions in the meter or even centimeter scale. This feature enables distributed sensors to provide a large number of resolved points using a single optical fiber. However, in current systems, this number has remained constrained to a few hundreds of thousands due to the finite signal-to-noise ratio (SNR) of the measurements, which imposes significant challenges in the development of more performing sensors. Here, we propose and experimentally demonstrate an ultimately optimized distributed fiber sensor capable of resolving 2100000 independent points, which corresponds to a one-order-of-magnitude improvement compared to the state-of-the-art. Using a Brillouin distributed fiber sensor based on phase-modulation correlation-domain analysis combined with temporal gating of the pump and time-domain acquisition, a spatial resolution of 8.3 mm is demonstrated over a distance of 17.5 km. The sensor design addresses the most relevant factors impacting the SNR and the performance of medium-to-long range sensors as well as of sub-meter spatial resolution schemes. This step record in the number of resolved points could be reached due to two theoretical models proposed and experimentally validated in this study: one model describes the spatial resolution of the system and its relation with the sampling interval, and the other describes the amplitude response of the sensor, providing an accurate estimation of the SNR of the measurements. A distributed fiber sensor has been developed that can resolve more than 2 million points along its length. The system, built by Andrey Denisov and co-workers from Ecole Polytechnique Federale de Lausanne, offers a spatial resolution of just 8.3 mm over a distance of 17.5 km. This has been achieved by combining phase-modulation correlation-domain analysis with temporal gating of the pump signal and a theoretical model to optimize the signal-to-noise ratio. The measurement time varies cubically with the number of sample points. Consequently, a measurement of 300 000 points corresponding to a resolution of 5 cm takes only 4 min. Thus, the researchers say that a coarse scan of the whole fiber can be quickly made prior to a finer resolution scan of the region of interest.
TL;DR: A complete pansharpening scheme based on the use of morphological half gradient operators is designed and the suitability of this algorithm is demonstrated through the comparison with the state-of-the-art approaches.
Abstract: Nonlinear decomposition schemes constitute an alternative to classical approaches for facing the problem of data fusion. In this paper, we discuss the application of this methodology to a popular remote sensing application called pansharpening, which consists in the fusion of a low resolution multispectral image and a high-resolution panchromatic image. We design a complete pansharpening scheme based on the use of morphological half gradient operators and demonstrate the suitability of this algorithm through the comparison with the state-of-the-art approaches. Four data sets acquired by the Pleiades, Worldview-2, Ikonos, and Geoeye-1 satellites are employed for the performance assessment, testifying the effectiveness of the proposed approach in producing top-class images with a setting independent of the specific sensor.
10 Jul 2016
TL;DR: A convolutional neural network model for remote sensing image classification using CNNs provides a means of learning contextual features for large-scale image labeling and produces more accurate classifications in lower computational time.
Abstract: We propose a convolutional neural network (CNN) model for remote sensing image classification. Using CNNs provides us with a means of learning contextual features for large-scale image labeling. Our network consists of four stacked convolutional layers that downsample the image and extract relevant features. On top of these, a deconvolutional layer upsamples the data back to the initial resolution, producing a final dense image labeling. Contrary to previous frameworks, our network contains only convolution and deconvolution operations. Experiments on aerial images show that our network produces more accurate classifications in lower computational time.
TL;DR: In this paper, a high-resolution micro-computed tomography (micro-CT) scanner was used to image and reconstruct three-dimensional pore-space images of five sandstones (Bentheimer, Berea, Clashach, Doddington and Stainton) and five complex carbonates (Ketton, Estaillades, Middle Eastern sample 3,Middle Eastern sample 5 and Indiana Limestone 1) at four different voxel resolutions (4.4
TL;DR: Experimental evaluation on the real world challenging databases and comparison with the state-of-the-art super-resolution, classifier based and cross modal synthesis techniques show the effectiveness of the proposed algorithm.
Abstract: We propose a completely automatic approach for recognizing low resolution face images captured in uncontrolled environment. The approach uses multidimensional scaling to learn a common transformation matrix for the entire face which simultaneously transforms the facial features of the low resolution and the high resolution training images such that the distance between them approximates the distance had both the images been captured under the same controlled imaging conditions. Stereo matching cost is used to obtain the similarity of two images in the transformed space. Though this gives very good recognition performance, the time taken for computing the stereo matching cost is significant. To overcome this limitation, we propose a reference-based approach in which each face image is represented by its stereo matching cost from a few reference images. Experimental evaluation on the real world challenging databases and comparison with the state-of-the-art super-resolution, classifier based and cross modal synthesis techniques show the effectiveness of the proposed algorithm.
TL;DR: The results show that images with good spatial resolution and contrast can be obtained from highly sub-sampled PAT data if variational image reconstruction techniques that describe the tissues structures with suitable sparsity-constraints are used.
Abstract: Current 3D photoacoustic tomography (PAT) systems offer either high image quality or high frame rates but are not able to deliver high spatial and temporal resolution simultaneously, which limits their ability to image dynamic processes in living tissue (4D PAT). A particular example is the planar Fabry-Perot (FP) photoacoustic scanner, which yields high-resolution 3D images but takes several minutes to sequentially map the incident photoacoustic field on the 2D sensor plane, point-by-point. However, as the spatio-temporal complexity of many absorbing tissue structures is rather low, the data recorded in such a conventional, regularly sampled fashion is often highly redundant. We demonstrate that combining model-based, variational image reconstruction methods using spatial sparsity constraints with the development of novel PAT acquisition systems capable of sub-sampling the acoustic wave field can dramatically increase the acquisition speed while maintaining a good spatial resolution: first, we describe and model two general spatial sub-sampling schemes. Then, we discuss how to implement them using the FP interferometer and demonstrate the potential of these novel compressed sensing PAT devices through simulated data from a realistic numerical phantom and through measured data from a dynamic experimental phantom as well as from in vivo experiments. Our results show that images with good spatial resolution and contrast can be obtained from highly sub-sampled PAT data if variational image reconstruction techniques that describe the tissues structures with suitable sparsity-constraints are used. In particular, we examine the use of total variation (TV) regularization enhanced by Bregman iterations. These novel reconstruction strategies offer new opportunities to dramatically increase the acquisition speed of photoacoustic scanners that employ point-by-point sequential scanning as well as reducing the channel count of parallelized schemes that use detector arrays.
TL;DR: Time-resolved femtosecond x-ray diffraction patterns from laser-excited molecular iodine are used to create a movie of intramolecular motion with a temporal and spatial resolution of 30 fs and 0.3 Å, demonstrating the stunning sensitivity of heterodyne methods.
Abstract: Time-resolved femtosecond x-ray diffraction patterns from laser-excited molecular iodine are used to create a movie of intramolecular motion with a temporal and spatial resolution of 30 fs and 0.3 A. This high fidelity is due to interference between the nonstationary excitation and the stationary initial charge distribution. The initial state is used as the local oscillator for heterodyne amplification of the excited charge distribution to retrieve real-space movies of atomic motion on angstrom and femtosecond scales. This x-ray interference has not been employed to image internal motion in molecules before. In conclusion, coherent vibrational motion and dispersion, dissociation, and rotational dephasing are all clearly visible in the data, thereby demonstrating the stunning sensitivity of heterodyne methods.
TL;DR: A new approach of time-reversal imaging is developed that reduces the computational cost and the scanning dimensions from 4D to 3D (no time) and increases the spatial resolution of the source image.
Abstract: Time reversal is a powerful tool used to image directly the location and mechanism of passive seismic sources. This technique assumes seismic velocities in the medium and propagates time-reversed observations of ground motion at each receiver location. Assuming an accurate velocity model and adequate array aperture, the waves will focus at the source location. Because we do not know the location and the origin time a priori, we need to scan the entire 4D image (3D in space and 1D in time) to localize the source, which makes time-reversal imaging computationally demanding. We have developed a new approach of time-reversal imaging that reduces the computational cost and the scanning dimensions from 4D to 3D (no time) and increases the spatial resolution of the source image. We first individually extrapolate wavefields at each receiver, and then we crosscorrelate these wavefields (the product in the frequency domain: geometric mean). This crosscorrelation creates another imaging condition, and focusi...