The PASCAL Visual Object Classes Challenge

2005년 대한 생화학·분자생물학회 하계학술대회를 다녀와서

我的台灣, 看見心靈的故鄉 =2009林磐聳藝術與設計展

Sparse coding has become an increasingly popular method in learning and vision for a variety of classification, reconstruction and coding tasks. The canonical approach intrinsically assumes independence between observations during learning. For many natural signals however, sparse coding is applied to sub-elements ( i.e. patches) of the signal, where such an assumption is invalid. Convolutional sparse coding explicitly models local interactions through the convolution operator, however the resulting optimization problem is considerably more complex than traditional sparse coding. In this paper, we draw upon ideas from signal processing and Augmented Lagrange Methods (ALMs) to produce a fast algorithm with globally optimal sub problems and super-linear convergence.

Fast convolutional sparse coding

Image fusion meets deep learning: A survey and perspective

Fusing a low spatial resolution hyperspectral images (HSIs) with an high spatial resolution conventional (e.g., RGB) image has underpinned much of recent progress in HSIs super-resolution. However, such a scheme requires this pair of images to be well registered, which is often difficult to be complied with in real applications. To address this problem, we present a novel single HSI super-resolution method, termed Grouped Deep Recursive Residual Network (GDRRN), which learns to directly map an input low resolution HSI to a high resolution HSI with a specialized deep neural network. To well depict the complicated non-linear mapping function with a compact network, a grouped recursive module is embedded into the global residual structure to transform the input HSIs. In addition, we conjoin the traditional mean squared error (MSE) loss with the spectral angle mapper (SAM) loss together to learn the network parameters, which enables to reduce both the numerical error and spectral distortion in the super-resolution results, and ultimately improve the performance. Sufficient experiments on the benchmark HSI dataset demonstrate the effectiveness of the proposed method in terms of single HSI super-resolution.

Single Hyperspectral Image Super-Resolution with Grouped Deep Recursive Residual Network

The production of a high spatial resolution (HR) hyperspectral image (HSI) through the fusion of a low spatial resolution (LR) HSI with an HR multispectral image (MSI) has underpinned much of the recent progress in HSI super-resolution. The premise of these signs of progress is that both the degeneration from the HR HSI to LR HSI in the spatial domain and the degeneration from the HR HSI to HR MSI in the spectral domain are assumed to be known in advance. However, such a premise is difficult to achieve in practice. To address this problem, we propose to incorporate degeneration estimation into HSI super-resolution and present an unsupervised deep framework for “blind” HSIs super-resolution where the degenerations in both domains are unknown. In this framework, we model the latent HR HSI and the unknown degenerations with deep network structures to regularize them instead of using handcrafted (or shallow) priors. Specifically, we generate the latent HR HSI with an image-specific generator network and structure the degenerations in spatial and spectral domains through a convolution layer and a fully connected layer, respectively. By doing this, the proposed framework can be formulated as an end-to-end deep network learning problem, which is purely supervised by those two input images (i.e., LR HSI and HR MSI) and can be effectively solved by the backpropagation algorithm. Experiments on both natural scene and remote sensing HSI data sets show the superior performance of the proposed method in coping with unknown degeneration either in the spatial domain, spectral domain, or even both of them.

Deep Blind Hyperspectral Image Super-Resolution

Fusion based hyperspectral image (HSI) super-resolution method, which obtains a spatially high-resolution (HR) HSI by fusing a low-resolution (LR) HSI and an HR conventional image, has been a prevalent method for HSI super-resolution. One effective fusion based method is to cast HSI super-resolution into a unified optimization problem, where handcrafted priors such as sparse prior or low rank prior are always adopted to regularize the latent HR HSI to be optimized. However, these priors show limitations in generalizing to challenging cases due to the heuristic assumption on image statistics as well as the restricted expressiveness capacity of the shallow structure. Taking advantages of the powerful expression ability of deep learning based method, a new HSI super-resolution network is proposed which implicitly incorporates a deep structure as the regularizer/prior. Specifically, we reformulate the original unified optimization problem into three sub-optimization problems, one is related with the regularizer and the others are without. Thanks to the fact that the one related with the regularizer naturally equals to a denoising problem, a recursive residual network is proposed for this sub-optimization problem. In addition, we unfold the other sub-optimization problems into network representations, with which the original unified optimization problem can be represented into a fully end-to-end network. Experimental results shows the superiority of the proposed method for HSI super-resolution on three benchmark datasets.

Deep Recursive Network for Hyperspectral Image Super-Resolution

Salient object detection in hyperspectral imagery has drawn people's attention in recent years. Some detection methods which focus on extending Itti's visual saliency model into spectral domain have been proposed. However, these methods are sensitive to high-contrast edges and cannot preserve boundary of salient object well. To address these shortcomings, we propose a region-based spectral gradient contrast method for salient object detection in this paper. First, we calculate gradient along each spectral vector of the input hyperspectral imagery. Then over segmentation and clustering methods are applied on gradient data to get a group of image regions. Finally, center prior and local contrast are employed to compute the saliency score of each region, with which the salient object can be obtained. Experimental results on four datasets demonstrate that the proposed method outperforms several competing methods on detection accuracy.

Salient object detection in hyperspectral imagery using spectral gradient contrast

Image denoising is the problem that aims at recovering a clean image from a noisy counterpart. A promising solution for image denoising is to employ an appropriate deep neural network to learn a hierarchical mapping function from the noisy image to its clean counterpart. This mapping function, however, is generally difficult to learn since the potential feature space of the noisy patterns can be huge. To overcome this difficulty, we propose a separation–aggregation strategy to decompose the noisy image into multiple bands, each of which exhibits one kind of pattern. Then a deep mapping function is learned for each band and the mapping results are ultimately assembled to the clean image. By doing so, the network only needs to deal with the compositing components of the noisy image, thus makes it easier to learn an effective mapping function. Moreover, as any image can be viewed as a composition of some basic patterns, our strategy is expected to better generalize to unseen images. Inspired by this idea, we develop a separation–aggregation network. The proposed network consists of three blocks, namely a convolutional separation block that decomposes the input into multiple bands, a deep mapping block that learns the mapping function for each band, and a band aggregation block that assembles the mapping results. Experimental results demonstrate the superiority of our strategy over counterparts without image decomposition.

Yong Li

Papers

Single Hyperspectral Image Super-Resolution with Grouped Deep Recursive Residual Network

Deep Blind Hyperspectral Image Super-Resolution

Deep Recursive Network for Hyperspectral Image Super-Resolution

Salient object detection in hyperspectral imagery using spectral gradient contrast

A separation–aggregation network for image denoising