Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
TL;DR: In this article , the authors provide a systematic review of 2.5D/3D Digital Elevation Model (DEM) fusion, including preprocessing workflow, methods and applications, enhanced with a meta-analysis.
Abstract: The remote sensing community has identified data fusion as one of the key challenging topics of the 21st century. The subject of image fusion in two-dimensional (2D) space has been covered in several published reviews. However, the special case of 2.5D/3D Digital Elevation Model (DEM) fusion has not been addressed till date. DEM fusion is a key application of data fusion in remote sensing. It takes advantage of the complementary characteristics of multi-source DEMs to deliver a more complete, accurate, and reliable elevation dataset. Although several methods for fusing DEMs have been developed, the absence of a well-rounded review has limited their proliferation among researchers and end-users. Combining knowledge from multiple studies is often required to inform a holistic perspective and guide further research. In response, this paper provides a systematic review of DEM fusion: the pre-processing workflow, methods and applications, enhanced with a meta-analysis. Through the discussion and comparative analysis, unresolved challenges and open issues are identified, and future directions for research are proposed. This review is a timely solution and an invaluable source of information for researchers within the fields of remote sensing and spatial information science, and the data fusion community at large.
13 citations
11 Jul 2021
TL;DR: In this article, several deep networks were proposed to enhance Proba-V images, and in this paper, they focus on employing them to super-resolve the Sentinel-2 images, in particular, investigate the influence of the training data, including real and simulated low-resolution images, on the final SR outcome.
Abstract: Super-resolution (SR) reconstruction is a common term for techniques aimed at generating a high-resolution image from a single low-resolution image or multiple images showing the same scene. Multiple-image SR benefits from data fusion which allows for more accurate reconstruction of the underlying high-resolution information. Deep learning is extensively used for single-image SR, but its application to multiple-image SR is much less explored. Recently, several deep networks were proposed to enhance Proba-V images, and in this paper, we focus on employing them to super-resolve the Sentinel-2 images. In particular, we investigate the influence of the training data, including real and simulated low-resolution images, on the final SR outcome. Also, we make the simulated data publicly available.
12 citations
TL;DR: This study established the first workflow for SBP monitoring based on remotely sensed data, suggesting an operational approach forSBP identification and can be applied to other types of pastures in agroforestry regions to reduce the effects of shadows and flowering in classification problems.
Abstract: Sown Biodiverse Pastures (SBP) are the basis of a high-yield grazing system tailored for Mediterranean ecosystems and widely implemented in Southern Portugal. The application of precision farming methods in SBP requires cost-effective monitoring using remote sensing (RS). The main hurdle for the remote monitoring of SBP is the fact that the bulk of the pastures are installed in open Montado agroforestry systems. Sparsely distributed trees cast shadows that hinder the identification of the underlaying pasture using Unmanned Aerial Vehicles (UAV) imagery. Image acquisition in the Spring is made difficult by the presence of flowers that mislead the classification algorithms. Here, we tested multiple procedures for the geographical, object-based image classification (GEOBIA) of SBP, aiming to reduce the effects of tree shadows and flowers in open Montado systems. We used remotely sensed data acquired between November 2017 and May 2018 in three Portuguese farms. We used three machine learning supervised classification algorithms: Random Forests (RF), Support Vector Machine (SVM) and Artificial Neural Networks (ANN). We classified SBP based on: (1) a single-period image for the maximum Normalized Difference Vegetation Index (NDVI) epoch in each of the three farms, and (2) multi-temporal image stacking. RF, SVM and ANN were trained using some visible (red, green and blue bands) and near-infrared (NIR) reflectance bands, plus NDVI and a Digital Surface Model (DSM). We obtained high overall accuracy and kappa index (higher than 79% and 0.60, respectively). The RF algorithm had the highest overall accuracy (more than 92%) for all farms. Multitemporal image classification increased the accuracy of the algorithms. as it helped to correctly identify as SBP the areas covered by tree shadows and flower patches, which would be misclassified using single image classification. This study thus established the first workflow for SBP monitoring based on remotely sensed data, suggesting an operational approach for SBP identification. The workflow can be applied to other types of pastures in agroforestry regions to reduce the effects of shadows and flowering in classification problems.
12 citations
Posted Content•
TL;DR: Deep learning has achieved great success in various fields, including bioinformatics as discussed by the authors, and concrete examples and implementations of its representative applications in bio-informatic applications are provided.
Abstract: Deep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at \url{this https URL}.
12 citations
TL;DR: A new video super-resolution method that can generate high-quality and temporally coherent high-resolution (HR) videos by incorporating deep features generated by VGG16 into the authors' sparse reconstruction based video SR method.
12 citations
References
More filters
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Abstract: Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
40,609 citations
Book•
23 May 2011TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
17,433 citations
TL;DR: It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.
Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the
16,538 citations
"Image super-resolution" refers background in this paper
...As the l2 norm represents a smoothing prior and the l1 norm tends to preserve the edges, the lp ( ≤ ≤ p 1 2) norm achieves a balance between them, thereby avoiding the staircase effect [110]....
[...]
TL;DR: In this article, a constrained optimization type of numerical algorithm for removing noise from images is presented, where the total variation of the image is minimized subject to constraints involving the statistics of the noise.
15,225 citations
"Image super-resolution" refers background in this paper
...[93,103], based on the fact that an image is naturally “blocky” and discontinuous....
[...]
Book•
01 Jan 1977
8,009 citations
"Image super-resolution" refers background in this paper
...In the early years, the smoothness of natural images was mainly considered, which leads to the quadratic property of the regularizations [99,100]....
[...]