Showing papers on "Markov random field published in 2016"

Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

Abstract: This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.

Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

Abstract: This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.

Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs

Abstract: Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving. We build on recent work [32] that trained a convolutional neural net to predict instance labeling in local image patches, extracted exhaustively in a stride from an image. A simple Markov random field model using several heuristics was then proposed in [32] to derive a globally consistent instance labeling of the image. In this paper, we formulate the global labeling problem with a novel densely connected Markov random field and show how to encode various intuitive potentials in a way that is amenable to efficient mean field inference [15]. Our potentials encode the compatibility between the global labeling and the patch-level predictions, contrast-sensitive smoothness as well as the fact that separate regions form different instances. Our experiments on the challenging KITTI benchmark [8] demonstrate that our method achieves a significant performance boost over the baseline [32].

Edge-Guided Single Depth Image Super Resolution

Abstract: Recently, consumer depth cameras have gained significant popularity due to their affordable cost. However, the limited resolution and the quality of the depth map generated by these cameras are still problematic for several applications. In this paper, a novel framework for the single depth image superresolution is proposed. In our framework, the upscaling of a single depth image is guided by a high-resolution edge map, which is constructed from the edges of the low-resolution depth image through a Markov random field optimization in a patch synthesis based manner. We also explore the self-similarity of patches during the edge construction stage, when limited training data are available. With the guidance of the high-resolution edge map, we propose upsampling the high-resolution depth image through a modified joint bilateral filter. The edge-based guidance not only helps avoiding artifacts introduced by direct texture prediction, but also reduces jagged artifacts and preserves the sharp edges. Experimental results demonstrate the effectiveness of our method both qualitatively and quantitatively compared with the state-of-the-art methods.

Lidar Waveform-Based Analysis of Depth Images Constructed Using Sparse Single-Photon Data

Abstract: This paper presents a new Bayesian model and algorithm used for depth and reflectivity profiling using full waveforms from the time-correlated single-photon counting measurement in the limit of very low photon counts. The proposed model represents each Lidar waveform as a combination of a known impulse response, weighted by the target reflectivity, and an unknown constant background, corrupted by Poisson noise. Prior knowledge about the problem is embedded through prior distributions that account for the different parameter constraints and their spatial correlation among the image pixels. In particular, a gamma Markov random field (MRF) is used to model the joint distribution of the target reflectivity, and a second MRF is used to model the distribution of the target depth, which are both expected to exhibit significant spatial correlations. An adaptive Markov chain Monte Carlo algorithm is then proposed to perform Bayesian inference. This algorithm is equipped with a stochastic optimization adaptation mechanism that automatically adjusts the parameters of the MRFs by maximum marginal likelihood estimation. Finally, the benefits of the proposed methodology are demonstrated through a series of experiments using real data.

Joint Recovery of Dense Correspondence and Cosegmentation in Two Images

Abstract: We propose a new technique to jointly recover cosegmentation and dense per-pixel correspondence in two images. Our method parameterizes the correspondence field using piecewise similarity transformations and recovers a mapping between the estimated common "foreground" regions in the two images allowing them to be precisely aligned. Our formulation is based on a hierarchical Markov random field model with segmentation and transformation labels. The hierarchical structure uses nested image regions to constrain inference across multiple scales. Unlike prior hierarchical methods which assume that the structure is given, our proposed iterative technique dynamically recovers the structure along with the labeling. This joint inference is performed in an energy minimization framework using iterated graph cuts. We evaluate our method on a new dataset of 400 image pairs with manually obtained ground truth, where it outperforms state-of-the-art methods designed specifically for either cosegmentation or correspondence estimation.

Sub-cortical brain structure segmentation using F-CNN'S

Abstract: In this paper we propose a deep learning approach for segmenting sub-cortical structures of the human brain in Magnetic Resonance (MR) image data. We draw inspiration from a state-of-the-art Fully-Convolutional Neural Network (F-CNN) architecture for semantic segmentation of objects in natural images, and adapt it to our task. Unlike previous CNN-based methods that operate on image patches, our model is applied on a full blown 2D image, without any alignment or registration steps at testing time. We further improve segmentation results by interpreting the CNN output as potentials of a Markov Random Field (MRF), whose topology corresponds to a volumetric grid. Alpha-expansion is used to perform approximate inference imposing spatial volumetric homogeneity to the CNN priors. We compare the performance of the proposed pipeline with a similar system using Random Forest-based priors, as well as state-of-art segmentation algorithms, and show promising results on two different brain MRI datasets.

Deep Learning Markov Random Field for Semantic Segmentation

Abstract: Semantic segmentation tasks can be well modeled by Markov Random Field (MRF). This paper addresses semantic segmentation by incorporating high-order relations and mixture of label contexts into MRF. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms. It has several appealing properties. First, different from the recent works that required many iterations of MF during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing models as its special cases. Furthermore, pairwise terms in DPN provide a unified framework to encode rich contextual information in high-dimensional data, such as images and videos. Third, DPN makes MF easier to be parallelized and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated on standard semantic image/video segmentation benchmarks, where a single DPN model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012, Cityscapes dataset and CamVid dataset.

A Field Model for Repairing 3D Shapes

Abstract: This paper proposes a field model for repairing 3D shapes constructed from multi-view RGB data. Specifically, we represent a 3D shape in a Markov random field (MRF) in which the geometric information is encoded by random binary variables and the appearance information is retrieved from a set of RGB images captured at multiple viewpoints. The local priors in the MRF model capture the local structures of object shapes and are learnt from 3D shape templates using a convolutional deep belief network. Repairing a 3D shape is formulated as the maximum a posteriori (MAP) estimation in the corresponding MRF. Variational mean field approximation technique is adopted for the MAP estimation. The proposed method was evaluated on both artificial data and real data obtained from reconstruction of practical scenes. Experimental results have shown the robustness and efficiency of the proposed method in repairing noisy and incomplete 3D shapes.

Fast Image-Based Obstacle Detection From Unmanned Surface Vehicles

Abstract: Obstacle detection plays an important role in unmanned surface vehicles (USVs). The USVs operate in a highly diverse environments in which an obstacle may be a floating piece of wood, a scuba diver, a pier, or a part of a shoreline, which presents a significant challenge to continuous detection from images taken on board. This paper addresses the problem of online detection by constrained, unsupervised segmentation. To this end, a new graphical model is proposed that affords a fast and continuous obstacle image-map estimation from a single video stream captured on board a USV. The model accounts for the semantic structure of marine environment as observed from USV by imposing weak structural constraints. A Markov random field framework is adopted and a highly efficient algorithm for simultaneous optimization of model parameters and segmentation mask estimation is derived. Our approach does not require computationally intensive extraction of texture features and comfortably runs in real time. The algorithm is tested on a new, challenging, dataset for segmentation, and obstacle detection in marine environments, which is the largest annotated dataset of its kind. Results on this dataset show that our model outperforms the related approaches, while requiring a fraction of computational effort.

Efficient 3D Room Shape Recovery from a Single Panorama

Abstract: We propose a method to recover the shape of a 3D room from a full-view indoor panorama. Our algorithm can automatically infer a 3D shape from a collection of partially oriented superpixel facets and line segments. The core part of the algorithm is a constraint graph, which includes lines and superpixels as vertices, and encodes their geometric relations as edges. A novel approach is proposed to perform 3D reconstruction based on the constraint graph by solving all the geometric constraints as constrained linear least-squares. The selected constraints used for reconstruction are identified using an occlusion detection method with a Markov random field. Experiments show that our method can recover room shapes that can not be addressed by previous approaches. Our method is also efficient, that is, the inference time for each panorama is less than 1 minute.

Infrared Ship Target Segmentation Based on Spatial Information Improved FCM

Abstract: Segmentation of infrared (IR) ship images is always a challenging task, because of the intensity inhomogeneity and noise. The fuzzy C-means (FCM) clustering is a classical method widely used in image segmentation. However, it has some shortcomings, like not considering the spatial information or being sensitive to noise. In this paper, an improved FCM method based on the spatial information is proposed for IR ship target segmentation. The improvements include two parts: 1) adding the nonlocal spatial information based on the ship target and 2) using the spatial shape information of the contour of the ship target to refine the local spatial constraint by Markov random field. In addition, the results of ${K}$ -means are used to initialize the improved FCM method. Experimental results show that the improved method is effective and performs better than the existing methods, including the existing FCM methods, for segmentation of the IR ship images.

Supervised Band Selection Using Local Spatial Information for Hyperspectral Image

Abstract: In order to alleviate the subsequent computation burden and storage requirement, band selection has been widely adopted to reduce the dimensionality of hyperspectral images, and the current methods mainly consist of the supervised and the unsupervised. Although these supervised methods have better performance, those unsupervised methods dominate the band selection field. In this letter, based on the unique properties of hyperspectral images, we propose a very simple but effective supervised band selection algorithm based on the local spatial information of the hyperspectral image and wrapper method. By using both the information of labeled and unlabeled pixels of the hyperspectral image, our proposed algorithm consistently outperforms the classical wrapper method. We use five widely used real hyperspectral data to demonstrate the effectiveness of our proposed algorithms. We also analyze the relationship between our band selection algorithm and the well-known Markov random field classifier.

MRF-based segmentation and unsupervised classification for building and road detection in peri-urban areas of high-resolution satellite images

Abstract: We present in this article a new method on unsupervised semantic parsing and structure recognition in peri-urban areas using satellite images. The automatic “building” and “road” detection is based on regions extracted by an unsupervised segmentation method. We propose a novel segmentation algorithm based on a Markov random field model and we give an extensive data analysis for determining relevant features for the classification problem. The novelty of the segmentation algorithm lies on the class-driven vector data quantization and clustering and the estimation of the likelihoods given the resulting clusters. We have evaluated the reachability of a good classification rate using the Random Forest method. We found that, with a limited number of features, among them some new defined in this article, we can obtain good classification performance. Our main contribution lies again on the data analysis and the estimation of likelihoods. Finally, we propose a new method for completing the road network exploiting its connectivity, and the local and global properties of the road network.

Robust Low-Rank Matrix Factorization Under General Mixture Noise Distributions

Abstract: Many computer vision problems can be posed as learning a low-dimensional subspace from high-dimensional data. The low rank matrix factorization (LRMF) represents a commonly utilized subspace learning strategy. Most of the current LRMF techniques are constructed on the optimization problems using $L_{1}$ -norm and $L_{2}$ -norm losses, which mainly deal with the Laplace and Gaussian noises, respectively. To make LRMF capable of adapting more complex noise, this paper proposes a new LRMF model by assuming noise as mixture of exponential power (MoEP) distributions and then proposes a penalized MoEP (PMoEP) model by combining the penalized likelihood method with MoEP distributions. Such setting facilitates the learned LRMF model capable of automatically fitting the real noise through MoEP distributions. Each component in this mixture distribution is adapted from a series of preliminary super- or sub-Gaussian candidates. Moreover, by facilitating the local continuity of noise components, we embed Markov random field into the PMoEP model and then propose the PMoEP-MRF model. A generalized expectation maximization (GEM) algorithm and a variational GEM algorithm are designed to infer all parameters involved in the proposed PMoEP and the PMoEP-MRF model, respectively. The superiority of our methods is demonstrated by extensive experiments on synthetic data, face modeling, hyperspectral image denoising, and background subtraction.

Classification of Polarimetric SAR Images Based on Modeling Contextual Information and Using Texture Features

Abstract: This paper proposes a novel contextual method for classification of polarimetric synthetic aperture radar data. The method combines support vector machine (SVM) and Wishart classifiers to benefit from both parametric and nonparametric methods. This method computes the energy function of a Markov random field (MRF) in the neighborhoods of the pixel using Wishart distribution. It then relates the Markovian energydifference function to the SVM classifier. Therefore, the salt-and-pepper effect on the classified map is reduced using a contextual classifier. Moreover, to achieve the full advantage of spatial information, texture features are added into the contextual classification. Texture features are extracted from SPAN images and are added to the SVM classifier. In this paper, two Radarsat-2 polarimetric images acquired in the leaf-off and leaf-on seasons are used from a forest area. Efficient multitemporal information is exploited using composite kernels in SVM. Comparison of the proposed algorithm with the Wishart, Wishart-MRF, SVM, and SVM with composite kernel classifiers shows a 21.72%, 16.17%, 11.29%, and 8.19% improvement in overall accuracy, respectively. Moreover, incorporating texture features into classification results significant increase in the average accuracy in forest species compared with the use of only polarimetric features.

Inferring species interactions from co-occurrence data with Markov networks.

Abstract: Inferring species interactions from co-occurrence data is one of the most controversial tasks in community ecology. One difficulty is that a single pairwise Interaction can ripple through an ecological network and produce surprising indirect consequences. For example, the negative correlation between two competing species can be reversed in the presence of a third species that outcompetes both of them. Here, I apply models from statistical physics, called Markov networks or Markov random fields, that can predict the direct and indirect consequences of any possible species interaction matrix. Interactions in these models can be estimated from observed co-occurrence rates via maximum likelihood, controlling for indirect effects. Using simulated landscapes with known interactions, I evaluated Markov networks and six existing approaches. Markov networks consistently outperformed the other methods, correctly isolating direct interactions between species pairs even when indirect interactions or abiotic factors largely overpowered them. Two computationally efficient approximations, which controlled for indirect effects with partial correlations or generalized linear models, also performed well. Null models showed no evidence of being able to control for indirect effects, and reliably yielded incorrect inferences when such effects were present. This article is protected by copyright. All rights reserved.

Deep Discrete Flow

Abstract: Motivated by the success of deep learning techniques in matching problems, we present a method for learning context-aware features for solving optical flow using discrete optimization. Towards this goal, we present an efficient way of training a context network with a large receptive field size on top of a local network using dilated convolutions on patches. We perform feature matching by comparing each pixel in the reference image to every pixel in the target image, utilizing fast GPU matrix multiplication. The matching cost volume from the network’s output forms the data term for discrete MAP inference in a pairwise Markov random field. We provide an extensive empirical investigation of network architectures and model parameters. At the time of submission, our method ranks second on the challenging MPI Sintel test set.

Integrating Hierarchical Segmentation Maps With MRF Prior for Classification of Hyperspectral Images in a Bayesian Framework

Abstract: This paper presents a new spectral-spatial classification method for hyperspectral (HS) images. The proposed method is based on integrating hierarchical segmentation results into Markov random field (MRF) spatial prior in the Bayesian framework. This work includes two main contributions. First, statistical region merging (SRM) segmentation algorithm is extended to a hierarchical version, HSRM. Second, a method for extracting a multilevel “fuzzy no-border/border” map from HSRM segmentation hierarchy is proposed, which are then exploited as weighting coefficients to modify the spatial prior of MRF-based multilevel logistic (MLL) model. The proposed method, named as MRF + HSRM, addresses the common problem of MRF-based methods, i.e., over-smoothing of classification result. Several experiments are conducted using real HS images to evaluate the performance of the proposed method in comparison with conventional MRF, and some state-of-the-art weighted MRF and object-based classifiers. To estimate the class conditional probability distribution in Bayesian framework, probabilistic support vector machine (SVM) and subspace multinomial logistic regression (MLRsub) classifiers are used. The experimental results demonstrate that the proposed method is able to generate more homogeneous regions similar to MRF-based methods, while preserving class boundaries as accurately as segmentation-based methods. The overhead computational burden of the proposed hierarchical segmentation stage is negligible considering the improvement it offers in classification results.

Manhattan-World Urban Reconstruction from Point Clouds

Abstract: Manhattan-world urban scenes are common in the real world. We propose a fully automatic approach for reconstructing such scenes from 3D point samples. Our key idea is to represent the geometry of the buildings in the scene using a set of well-aligned boxes. We first extract plane hypothesis from the points followed by an iterative refinement step. Then, candidate boxes are obtained by partitioning the space of the point cloud into a non-uniform grid. After that, we choose an optimal subset of the candidate boxes to approximate the geometry of the buildings. The contribution of our work is that we transform scene reconstruction into a labeling problem that is solved based on a novel Markov Random Field formulation. Unlike previous methods designed for particular types of input point clouds, our method can obtain faithful reconstructions from a variety of data sources. Experiments demonstrate that our method is superior to state-of-the-art methods.

Spectral-Spatial Hyperspectral Image Classification Using Subspace-Based Support Vector Machines and Adaptive Markov Random Fields

Abstract: This paper introduces a new supervised classification method for hyperspectral images that combines spectral and spatial information. A support vector machine (SVM) classifier, integrated with a subspace projection method to address the problems of mixed pixels and noise, is first used to model the posterior distributions of the classes based on the spectral information. Then, the spatial information of the image pixels is modeled using an adaptive Markov random field (MRF) method. Finally, the maximum posterior probability classification is computed via the simulated annealing (SA) optimization algorithm. The combination of subspace-based SVMs and adaptive MRFs is the main contribution of this paper. The resulting methods, called SVMsub-eMRF and SVMsub-aMRF, were experimentally validated using two typical real hyperspectral data sets. The obtained results indicate that the proposed methods demonstrate superior performance compared with other classical hyperspectral image classification methods.

Object Depth Profile and Reflectivity Restoration from Sparse Single-Photon Data Acquired in Underwater Environments

Abstract: This paper presents two new algorithms for the joint restoration of depth and reflectivity (DR) images constructed from time-correlated single-photon counting (TCSPC) measurements. Two extreme cases are considered: (i) a reduced acquisition time that leads to very low photon counts and (ii) a highly attenuating environment (such as a turbid medium) which makes the reflectivity estimation more difficult at increasing range. Adopting a Bayesian approach, the Poisson distributed observations are combined with prior distributions about the parameters of interest, to build the joint posterior distribution. More precisely, two Markov random field (MRF) priors enforcing spatial correlations are assigned to the DR images. Under some justified assumptions, the restoration problem (regularized likelihood) reduces to a convex formulation with respect to each of the parameters of interest. This problem is first solved using an adaptive Markov chain Monte Carlo (MCMC) algorithm that approximates the minimum mean square parameter estimators. This algorithm is fully automatic since it adjusts the parameters of the MRFs by maximum marginal likelihood estimation. However, the MCMC-based algorithm exhibits a relatively long computational time. The second algorithm deals with this issue and is based on a coordinate descent algorithm. Results on single-photon depth data from laboratory based underwater measurements demonstrate the benefit of the proposed strategy that improves the quality of the estimated DR images.

Highway traffic flow prediction using support vector regression and Bayesian classifier

Abstract: With the vast availability of traffic sensing data on highway, real-time traffic flow prediction is essential part of transportation, traffic control, reports of accidents and intelligent transportation systems. To satisfy the demand of traffic flow prediction, this paper presents the method of real-time traffic flow prediction based on Bayesian classifier and support vector regression (SVR). We first model the traffic flow and its relations on the roads using 3D Markov random fields in spatiotemporal domain. Based on their relations, we define cliques as combination of current road and its neighbors. The dependencies on the defined cliques are estimated by using multiple linear regression and SVR. Finally, the traffic flow at next time stamp is predicted by finding the speed level with decreasing the energy function. To evaluate the performance of the proposed method, it was tested on traffic data obtained from Gyeongbu expressway. The experimental results showed that the approach using SVR-based estimation had superior accuracy than linear-based regression.