Showing papers by "Yannick Berthoumieu published in 2022"

Patch-based image Super Resolution using generalized Gaussian mixture model

Abstract: les patchs consid´erable. La technique de minimum de l’erreur qua-dratique moyenne est une m´ethode de restauration d’images qui utilise un mod`ele gaussien de probabilit´es les patchs d’images. article propose un algorithme d’apprentissage d’un mod ` ele conjoint de m ´ elange gaussien g ´ en ´ eralis ´ e (GGMM) ` a partir des paires de patchs ` a basse r´esolution et des patchs correspondants `a haute r´esolution d’une image de r´ef´erence. `A partir de ce mod`ele GGMM, l’image haute r´esolution en utilisant la m´ethode MMSE. Nos ´evaluations num´eriques indiquent que la m´ethode MMSE-GGMM comporte rapport `a Abstract – Single Image Super Resolution (SISR) methods aim to recover the clean images in high resolution from low resolution observations. A family of patch-based approaches have received considerable attention and development. The minimum mean square error (MMSE) method is a powerful image restoration method that uses a probability model on the patches of images. This paper proposes an algorithm to learn a joint generalized Gaussian mixture model (GGMM) from a pair of the low resolution patches and the corresponding high resolution patches from the reference data. We then reconstruct the high resolution image based on the MMSE method. Our numerical evaluations indicate that the MMSE-GGMM method competes with other state of the art methods.

Remote Sensing Scene Classification Based on Covariance Pooling of Multi-layer CNN Features Guided by Saliency Maps

Abstract: The new generation of remote sensing imaging sensors enables high spatial, spectral and temporal resolution images with high revisit frequencies. These sensors allow the acquisition of multi-spectral and multi-temporal images. The availability of these data has raised the interest of the remote sensing community to develop novel machine learning strategies for supervised classification. This paper aims at introducing a novel supervised classification algorithm based on covariance pooling of multi-layer convolutional neural network (CNN) features. The basic idea consists in an ensemble learning approach based on covariance matrices estimation from CNN features. Then, after being projected on the log-Euclidean space, an SVM classifier is used to make a decision. In order to give more strength to relatively small objects of interest in the scene, we propose to incorporate the visual saliency map in the process. For that, inspired by the theory of robust statistics, a weighted covariance matrix estimator is considered. Larger weights are given to more salient regions. Finally, some experiments on remote sensing classification are conducted on the UC Merced land use dataset. The obtained results confirm the potential of the proposed approach in terms of classification scene accuracy. It demonstrates, besides the interest of exploiting second order statistics and adopting an ensemble learning approach, the benefit of incorporating visual saliency maps.

Deep Ensemble Learning Model Based on Covariance Pooling of Multi-Layer CNN Features

Abstract: Compared to standard deep convolutional neural networks (CNN) which include a global average pooling operator, second-order neural networks have a global covariance pooling operator which allows to capture richer statistics of CNN features. They have been shown to improve representation and generalization abilities. However, this covariance pooling is performed only on the deepest CNN feature maps. To benefit from different levels of abstraction, we propose to extend these models by using a multi-layer approach. In addition, to obtain better predictive performance, an end-to-end ensemble learning architecture is proposed. Experiments are conducted on four datasets and have confirmed the potential of the proposed model for various image processing applications such as remote sensing scene classification, indoor scene recognition and texture classification.

Automatic selection of latent variables in variational auto-encoders

Abstract: Variational auto-encoders (VAEs) are powerful generative neural networks based on latent variables. They aim to capture the distribution of a dataset, by building an informative space composed of a reduced number of variables. However, the size of this latent space is both sensitive and difficult to adjust. Thus, most state-of-the-art architectures experience either dis-entanglement issues, or, at the opposite, posterior collapse. Both phenomena impair the interpretability of the latent variables. In this paper, we propose a variant of the VAE which is able to automatically determine the informative components of the latent space. It consists in augmenting the vanilla VAE with auxiliary variables and defining a hierarchical model which favors that only a subset of the latent variables are used for the encoding. We refer to it as NGVAE. We compare its performance with other auto-encoder based architectures.