scispace - formally typeset
Search or ask a question

Showing papers on "Markov random field published in 2018"


Proceedings ArticleDOI
Linchao Bao1, Baoyuan Wu1, Wei Liu1
18 Jun 2018
TL;DR: A novel CNN-embedded algorithm to perform approximate inference in the MRF, which outperforms the winning entries of the DAVIS 2017 Challenge, without resorting to model ensembling or any dedicated detectors.
Abstract: This paper addresses the problem of video object segmentation, where the initial object mask is given in the first frame of an input video. We propose a novel spatiotemporal Markov Random Field (MRF) model defined over pixels to handle this problem. Unlike conventional MRF models, the spatial dependencies among pixels in our model are encoded by a Convolutional Neural Network (CNN). Specifically, for a given object, the probability of a labeling to a set of spatially neighboring pixels can be predicted by a CNN trained for this specific object. As a result, higher-order, richer dependencies among pixels in the set can be implicitly modeled by the CNN. With temporal dependencies established by optical flow, the resulting MRF model combines both spatial and temporal cues for tackling video object segmentation. However, performing inference in the MRF model is very difficult due to the very high-order dependencies. To this end, we propose a novel CNN-embedded algorithm to perform approximate inference in the MRF. This algorithm proceeds by alternating between a temporal fusion step and a feed-forward CNN step. When initialized with an appearance-based one-shot segmentation CNN, our model outperforms the winning entries of the DAVIS 2017 Challenge, without resorting to model ensembling or any dedicated detectors.

197 citations


Journal ArticleDOI
Ruijin Cang1, Hechao Li1, Hope Yao1, Yang Jiao1, Yi Ren1 
TL;DR: In this paper, a generative machine learning model was proposed to generate an arbitrary amount of artificial material samples with negligible computation cost, when trained on only a limited amount of authentic samples.

146 citations


Journal ArticleDOI
TL;DR: This work treats the small-dim targets as a special sparse noise component of the complex background noise and adopt Mixture of Gaussians (MoG) with Markov random field (MRF) with MRF to model the small target detection problem.

146 citations


Book ChapterDOI
Song Tao, Leiyu Sun, Di Xie, Sun Haiming, Shiliang Pu 
08 Sep 2018
TL;DR: A novel method integrated with somatic topological line localization (TLL) and temporal feature aggregation for detecting multi-scale pedestrians and a post-processing scheme based on Markov Random Field (MRF) is introduced to eliminate ambiguities in occlusion cases.
Abstract: A critical issue in pedestrian detection is to detect small-scale objects that will introduce feeble contrast and motion blur in images and videos, which in our opinion should partially resort to deep-rooted annotation bias. Motivated by this, we propose a novel method integrated with somatic topological line localization (TLL) and temporal feature aggregation for detecting multi-scale pedestrians, which works particularly well with small-scale pedestrians that are relatively far from the camera. Moreover, a post-processing scheme based on Markov Random Field (MRF) is introduced to eliminate ambiguities in occlusion cases. Applying with these methodologies comprehensively, we achieve best detection performance on Caltech benchmark and improve performance of small-scale objects significantly (miss rate decreases from 74.53% to 60.79%). Beyond this, we also achieve competitive performance on CityPersons dataset and show the existence of annotation bias in KITTI dataset.

135 citations


Journal ArticleDOI
TL;DR: This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on a pairwise Markov random field (MRF) that effectively combines recently proposed slanted patch matching and curvature regularization terms.
Abstract: We present an accurate stereo matching method using local expansion moves based on graph cuts. This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on a pairwise Markov random field (MRF) that effectively combines recently proposed slanted patch matching and curvature regularization terms. The local expansion moves are presented as many $\alpha$ -expansions defined for small grid regions. The local expansion moves extend traditional expansion moves by two ways: localization and spatial propagation. By localization, we use different candidate $\alpha$ -labels according to the locations of local $\alpha$ -expansions. By spatial propagation, we design our local $\alpha$ -expansions to propagate currently assigned labels for nearby regions. With this localization and spatial propagation, our method can efficiently infer MRF models with a continuous label space using randomized search. Our method has several advantages over previous approaches that are based on fusion moves or belief propagation; it produces submodular moves deriving a subproblem optimality ; it helps find good, smooth, piecewise linear disparity maps; it is suitable for parallelization; it can use cost-volume filtering techniques for accelerating the matching cost computations. Even using a simple pairwise MRF, our method is shown to have best performance in the Middlebury stereo benchmark V2 and V3.

123 citations


Journal ArticleDOI
TL;DR: The objective of this work is to detect shadows in images by posing this as the problem of labeling image regions, where each region corresponds to a group of superpixels, and training a kernel Least-Squares SVM for separating shadow and non-shadow regions.
Abstract: The objective of this work is to detect shadows in images. We pose this as the problem of labeling image regions, where each region corresponds to a group of superpixels. To predict the label of each region, we train a kernel Least-Squares Support Vector Machine (LSSVM) for separating shadow and non-shadow regions. The parameters of the kernel and the classifier are jointly learned to minimize the leave-one-out cross validation error. Optimizing the leave-one-out cross validation error is typically difficult, but it can be done efficiently in our framework. Experiments on two challenging shadow datasets, UCF and UIUC, show that our region classifier outperforms more complex methods. We further enhance the performance of the region classifier by embedding it in a Markov Random Field (MRF) framework and adding pairwise contextual cues. This leads to a method that outperforms the state-of-the-art for shadow detection. In addition we propose a new method for shadow removal based on region relighting. For each shadow region we use a trained classifier to identify a neighboring lit region of the same material. Given a pair of lit-shadow regions we perform a region relighting transformation based on histogram matching of luminance values between the shadow region and the lit region. Once a shadow is detected, we demonstrate that our shadow removal approach produces results that outperform the state of the art by evaluating our method using a publicly available benchmark dataset.

116 citations


Journal ArticleDOI
TL;DR: An automatic railway visual detection system (RVDS) for surface defects and focuses on several key issues of RVDS, which enables identification and segmentation of the defects from rail surface, achieving detection performance with 92% precision and 88.8% recall rate on average.
Abstract: Rails are among the most important components of railway transportation, and real-time defects detection of the railway is an important and challenging task because of intensity inhomogeneity, low contrast, and noise. This paper presents an automatic railway visual detection system (RVDS) for surface defects and focuses on several key issues of RVDS. First, in view of challenges such as complex condition and orbital reflectance inequality, we put forward a region-of-interest detection region extraction algorithm by vertical projection and gray contrast algorithm. In addition, a curvature filter equipped with implicit computing and surface preserving power is studied to eliminate noise and keep only the details. Then, an improved fast and robust Gaussian mixture model based on Markov random field is established for accurate and rapid surface defect segmentation. Additionally, an expectation–maximization algorithm is applied to optimize the parameters. The experimental results demonstrate that the proposed method performs well with both noisy and railway images, which enables identification and segmentation of the defects from rail surface, achieving detection performance with 92% precision and 88.8% recall rate on average, and is robust compared with the related well-established approaches.

111 citations


Journal ArticleDOI
TL;DR: Deep Parsing Network (DPN) as discussed by the authors extends a CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms.
Abstract: Semantic segmentation tasks can be well modeled by Markov Random Field (MRF). This paper addresses semantic segmentation by incorporating high-order relations and mixture of label contexts into MRF. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms. It has several appealing properties. First, different from the recent works that required many iterations of MF during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing models as its special cases. Furthermore, pairwise terms in DPN provide a unified framework to encode rich contextual information in high-dimensional data, such as images and videos. Third, DPN makes MF easier to be parallelized and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated on standard semantic image/video segmentation benchmarks, where a single DPN model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012, Cityscapes dataset and CamVid dataset.

110 citations


Book
01 Jan 2018
TL;DR: Medical Imaging Systems Fundamental Tools for Image Processing and Analysis Probability Theory for Stochastic Modeling of Images Two-Dimensional Fourier Transform Nonlinear Diffusion Filtering Intensity-Based Image Segmentation image segmentation by Markov Random Field Modeling Deformable Models Image Analysis.
Abstract: Medical Imaging Systems Fundamental Tools for Image Processing and Analysis Probability Theory for Stochastic Modeling of Images Two-Dimensional Fourier Transform Nonlinear Diffusion Filtering Intensity-Based Image Segmentation Image Segmentation by Markov Random Field Modeling Deformable Models Image Analysis Application 1: Quantification of Green Fluorescent Protein eXpression in Live Cells: ProXcell Application 2: Calculation of Performance Parameters of Gamma Cameras and SPECT Systems Application 3: Analysis of Islet Cells Using Automated Color Image Analysis Appendix A: Notation Appendix B: Working with DICOM Images Appendix C: Medical Image Processing Toolbox Appendix D: Description of Image Data

102 citations


Journal ArticleDOI
TL;DR: This work proposes a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup and proposes a RANSAC-based robust illumination estimation technique.
Abstract: Faces in natural images are often occluded by a variety of objects We propose a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup The key idea is to segment the image into regions explained by separate models Our framework includes a 3D morphable face model, a prototype-based beard model and a simple model for occlusions and background regions The segmentation and all the model parameters have to be inferred from the single target image Face model adaptation and segmentation are solved jointly using an expectation–maximization-like procedure During the E-step, we update the segmentation and in the M-step the face model parameters are updated For face model adaptation we apply a stochastic sampling strategy based on the Metropolis–Hastings algorithm For segmentation, we apply loopy belief propagation for inference in a Markov random field Illumination estimation is critical for occlusion handling Our combined segmentation and model adaptation needs a proper initialization of the illumination parameters We propose a RANSAC-based robust illumination estimation technique By applying this method to a large face image database we obtain a first empirical distribution of real-world illumination conditions The obtained empirical distribution is made publicly available and can be used as prior in probabilistic frameworks, for regularization or to synthesize data for deep learning methods

88 citations


Journal ArticleDOI
TL;DR: This review is devoted to cut-based medical segmentation methods, including graph cuts and graph search for region and surface segmentation, which include graph-cuts-based methods, model integrated graph cuts methods, graph-search-based Methods, and graph Search/graph cuts based methods.
Abstract: Medical image segmentation is a fundamental and challenging problem for analyzing medical images. Among different existing medical image segmentation methods, graph-based approaches are relatively new and show good features in clinical applications. In the graph-based method, pixels or regions in the original image are interpreted into nodes in a graph. By considering Markov random field to model the contexture information of the image, the medical image segmentation problem can be transformed into a graph-based energy minimization problem. This problem can be solved by the use of minimum s-t cut/ maximum flow algorithm. This review is devoted to cut-based medical segmentation methods, including graph cuts and graph search for region and surface segmentation. Different varieties of cut-based methods, including graph-cuts-based methods, model integrated graph cuts methods, graph-search-based methods, and graph search/graph cuts based methods, are systematically reviewed. Graph cuts and graph search with deep learning technique are also discussed.

Journal ArticleDOI
TL;DR: A novel cluster sparsity field based HSI reconstruction framework which explicitly models both the intrinsic correlation between measurements within the spectrum for a particular pixel, and the similarity between pixels due to the spatial structure of the HSI, thus combating the effects of noise corruption or undersampling.
Abstract: Hyperspectral images (HSIs) have significant advantages over more traditional image types for a variety of computer vision applications dues to the extra information available. The practical reality of capturing and transmitting HSIs however, means that they often exhibit large amounts of noise, or are undersampled to reduce the data volume. Methods for combating such image corruption are thus critical to many HSIs applications. Here we devise a novel cluster sparsity field (CSF) based HSI reconstruction framework which explicitly models both the intrinsic correlation between measurements within the spectrum for a particular pixel, and the similarity between pixels due to the spatial structure of the HSI. These two priors have been shown to be effective previously, but have been always considered separately. By dividing pixels of the HSI into a group of spatial clusters on the basis of spectrum characteristics, we define CSF, a Markov random field based prior. In CSF, a structured sparsity potential models the correlation between measurements within each spectrum, and a graph structure potential models the similarity between pixels in each spatial cluster. Then, we integrate the CSF prior learning and image reconstruction into a unified variational framework for optimization, which makes the CSF prior image-specific, and robust to noise. It also results in more accurate image reconstruction compared with existing HSI reconstruction methods, thus combating the effects of noise corruption or undersampling. Extensive experiments on HSI denoising and HSI compressive sensing demonstrate the effectiveness of the proposed method.

Journal ArticleDOI
TL;DR: This paper introduces a new adaptive spatial regularizer that well exploits the local spatial information, while a nonlocal regularizer is also used to search for global patch-pair similarities in the whole image.
Abstract: Hyperspectral images (HSIs) provide invaluable information in both spectral and spatial domains for image classification tasks. In this paper, we use semantic representation as a middle-level feature to describe image pixels’ characteristics. Deriving effective semantic representation is critical for achieving good classification performance. Since different image descriptors depict characteristics from different perspectives, combining multiple features in the same semantic space makes semantic representation more meaningful. First, a probabilistic support vector machine is used to generate semantic representation-based multifeatures. In order to derive better semantic representation, we introduce a new adaptive spatial regularizer that well exploits the local spatial information, while a nonlocal regularizer is also used to search for global patch-pair similarities in the whole image. We combine multiple features with local and nonlocal spatial constraints using an extended Markov random field model in the semantic space. Experimental results on three hyperspectral data sets show that the proposed method provides better performance than several state-of-the-art techniques in terms of region uniformity, overall accuracy, average accuracy, and Kappa statistics.

Journal ArticleDOI
TL;DR: The experimental study established that the proposed two stage approach extracted efficiently the contrast enhanced regions from the MRA and T1C brain images.

Journal ArticleDOI
TL;DR: A novel single image Bayesian super-resolution algorithm where the hyperspectral image (HSI) is the only source of information is proposed and it is shown that the proposed method outperforms the state of the art methods in terms of quality while preserving the spectral consistency.
Abstract: In this paper, we propose a novel single image Bayesian super-resolution (SR) algorithm where the hyperspectral image (HSI) is the only source of information. The main contribution of the proposed approach is to convert the ill-posed SR reconstruction problem in the spectral domain to a quadratic optimization problem in the abundance map domain. In order to do so, Markov random field based energy minimization approach is proposed and proved that the solution is quadratic. The proposed approach consists of five main steps. First, the number of endmembers in the scene is determined using virtual dimensionality. Second, the endmembers and their low resolution abundance maps are computed using simplex identification via the splitted augmented Lagrangian and fully constrained least squares algorithms. Third, high resolution (HR) abundance maps are obtained using our proposed maximum a posteriori based energy function. This energy function is minimized subject to smoothness, unity, and boundary constraints. Fourth, the HR abundance maps are further enhanced with texture preserving methods. Finally, HR HSI is reconstructed using the extracted endmembers and the enhanced abundance maps. The proposed method is tested on three real HSI data sets; namely the Cave, Harvard, and Hyperspectral Remote Sensing Scenes and compared with state-of-the-art alternative methods using peak signal to noise ratio, structural similarity, spectral angle mapper, and relative dimensionless global error in synthesis metrics. It is shown that the proposed method outperforms the state of the art methods in terms of quality while preserving the spectral consistency.

Journal ArticleDOI
TL;DR: A novel structure-guided framework for exemplar-based image inpainting to maintain the neighborhood consistence and structure coherence of an inpainted region using the Markov random field model is presented.
Abstract: In this paper, we present a novel structure-guided framework for exemplar-based image inpainting to maintain the neighborhood consistence and structure coherence of an inpainted region. The proposed method consists of a data term for pixel validity and boundary continuity, a smoothness term to depict the compatibility of neighboring pixels for contextual continuity, and a coherence term to investigate image inherent regularities to ensure image self-similarity. To better reconstruct image structures, the method utilizes image regularity statistics to extract dominant linear structures of the target image. Guided by these structures, homography transformations are estimated and combined to globally repair the missing region using the Markov random field model. To reduce computational complexity, a hierarchical process is implemented to utilize the regularity effectively. The experimental results demonstrate that our method yields better results for various real-world scenes than existing state-of-the-art image inpainting techniques.

Journal ArticleDOI
TL;DR: A regional decision fusion framework within which to gain the advantages of model-based CNN, while overcoming the problem of losing effective resolution and uncertain prediction at object boundaries, which is especially pertinent for complex VFSR image classification.
Abstract: Recent advances in computer vision and pattern recognition have demonstrated the superiority of deep neural networks using spatial feature representation, such as convolutional neural networks (CNNs), for image classification. However, any classifier, regardless of its model structure (deep or shallow), involves prediction uncertainty when classifying spatially and spectrally complicated very fine spatial resolution (VFSR) imagery. We propose here to characterize the uncertainty distribution of CNN classification and integrate it into a regional decision fusion to increase classification accuracy. Specifically, a variable precision rough set (VPRS) model is proposed to quantify the uncertainty within CNN classifications of VFSR imagery and partition this uncertainty into positive regions (correct classifications) and nonpositive regions (uncertain or incorrect classifications). Those “more correct” areas were trusted by the CNN, whereas the uncertain areas were rectified by a multilayer perceptron (MLP)-based Markov random field (MLP-MRF) classifier to provide crisp and accurate boundary delineation. The proposed MRF-CNN fusion decision strategy exploited the complementary characteristics of the two classifiers based on VPRS uncertainty description and classification integration. The effectiveness of the MRF-CNN method was tested in both urban and rural areas of southern England as well as semantic labeling data sets. The MRF-CNN consistently outperformed the benchmark MLP, support vector machine, MLP-MRF, CNN, and the baseline methods. This paper provides a regional decision fusion framework within which to gain the advantages of model-based CNN, while overcoming the problem of losing effective resolution and uncertain prediction at object boundaries, which is especially pertinent for complex VFSR image classification.

Posted Content
Song Tao, Leiyu Sun, Di Xie, Sun Haiming, Shiliang Pu 
TL;DR: A novel method integrated with somatic topological line localization (TLL) and temporal feature aggregation for detecting multi-scale pedestrians and a post-processing scheme based on Markov Random Field (MRF) is introduced to eliminate ambiguities in occlusion cases.
Abstract: A critical issue in pedestrian detection is to detect small-scale objects that will introduce feeble contrast and motion blur in images and videos, which in our opinion should partially resort to deep-rooted annotation bias. Motivated by this, we propose a novel method integrated with somatic topological line localization (TLL) and temporal feature aggregation for detecting multi-scale pedestrians, which works particularly well with small-scale pedestrians that are relatively far from the camera. Moreover, a post-processing scheme based on Markov Random Field (MRF) is introduced to eliminate ambiguities in occlusion cases. Applying with these methodologies comprehensively, we achieve best detection performance on Caltech benchmark and improve performance of small-scale objects significantly (miss rate decreases from 74.53% to 60.79%). Beyond this, we also achieve competitive performance on CityPersons dataset and show the existence of annotation bias in KITTI dataset.

Journal ArticleDOI
TL;DR: Experimental results on the CDnet dataset indicate that the proposed robust change detection method, named $M^{4}CD$ is robust under complex environments and ranks among the top methods.
Abstract: In this paper, we propose a robust change detection method for intelligent visual surveillance. This method, named $M^{4}CD$ , includes three major steps. First, a sample-based background model that integrates color and texture cues is built and updated over time. Second, multiple heterogeneous features (including brightness variation, chromaticity variation, and texture variation) are extracted by comparing the input frame with the background model, and a multi-view learning strategy is designed to online estimate the probability distributions for both foreground and background. The three features are approximately conditionally independent, making multi-view learning feasible. Pixel-wise foreground posteriors are then estimated with Bayes rule. Finally, the Markov random field (MRF) optimization and heuristic post-processing techniques are used sequentially to improve accuracy. In particular, a two-layer MRF model is constructed to represent pixel-based and superpixel-based contextual constraints compactly. Experimental results on the CDnet dataset indicate that $M^{4}CD$ is robust under complex environments and ranks among the top methods.

Journal ArticleDOI
TL;DR: A quantitative measurement on the inconsistency between the depth edge map and the color edge map is proposed and explicitly embeds it into the smoothness term of the MRF model.
Abstract: Color-guided depth enhancement is used to refine depth maps according to the assumption that the depth edges and the color edges at the corresponding locations are consistent. In methods on such low-level vision tasks, the Markov random field (MRF), including its variants, is one of the major approaches that have dominated this area for several years. However, the assumption above is not always true. To tackle the problem, the state-of-the-art solutions are to adjust the weighting coefficient inside the smoothness term of the MRF model. These methods lack an explicit evaluation model to quantitatively measure the inconsistency between the depth edge map and the color edge map, so they cannot adaptively control the efforts of the guidance from the color image for depth enhancement, leading to various defects such as texture-copy artifacts and blurring depth edges. In this paper, we propose a quantitative measurement on such inconsistency and explicitly embed it into the smoothness term. The proposed method demonstrates promising experimental results compared with the benchmark and state-of-the-art methods on the Middlebury ToF-Mark, and NYU data sets.

Journal ArticleDOI
TL;DR: A voxel-based method for automatically extracting the transmission lines from airborne LiDAR point cloud data that provides higher detection correctness rate and efficiency and robustness compared with other existing methods.
Abstract: The safety of the electricity infrastructure significantly affects both our daily life and industrial activities. Timely and accurate monitoring of the safety of electricity network can prevent dangerous situations effectively. Thus, we, in this paper, develop a voxel-based method for automatically extracting the transmission lines from airborne LiDAR point cloud data. The method proposed in this paper uses three-dimensional (3-D) voxels as primitives and consist of the following steps: First, skeleton structure extraction using Laplacian smoothing; second, feature construction of a 3-D voxel using Latent Dirichlet allocation topic model; and third Markov random field model-based extraction for generating locally continuous and globally optimal results. To evaluate the effectiveness and robustness of the proposed method, experiments were conducted on four different types of power line scenes with flat and complex terrains from helicopter-borne LiDAR point cloud data. Experimental results demonstrate that our proposed method is efficient and robust for automatically detecting both the single conductor and the bundled conductors, with precision, recall, and quality of over 96.78%, 98.67%, and 96.66%, respectively. Moreover, compared with other existing methods, our proposed method provides higher detection correctness rate.

Journal ArticleDOI
TL;DR: This paper proposes a dense correspondence-based transfer learning (DCTL) approach, which consists of extracting deep representations of traffic scene images via a fine-tuned convolutional neural network and constructing compact and effective representations via cross-domain metric learning and subspace alignment for cross- domain retrieval.
Abstract: Understanding traffic scene images taken from vehicle mounted cameras is important for high-level tasks, such as advanced driver assistance systems and autonomous driving It is a challenging problem due to large variations under different weather or illumination conditions In this paper, we tackle the problem of traffic scene understanding from a cross-domain perspective We attempt to understand the traffic scene from images taken from the same location but under different weather or illumination conditions (eg, understanding the same traffic scene from images on a rainy night with the help of images taken on a sunny day) To this end, we propose a dense correspondence-based transfer learning (DCTL) approach, which consists of three main steps: 1) extracting deep representations of traffic scene images via a fine-tuned convolutional neural network; 2) constructing compact and effective representations via cross-domain metric learning and subspace alignment for cross-domain retrieval; and 3) transferring the annotations from the retrieved best matching image to the test image based on cross-domain dense correspondences and a probabilistic Markov random field To verify the effectiveness of our DCTL approach, we conduct extensive experiments on a challenging data set, which contains 1828 images from six weather or illumination conditions

Proceedings ArticleDOI
01 Jul 2018
TL;DR: This work proposes a new method of model-based clustering, which it is called Toeplitz Inverse Covariance-based Clustering (TICC), and solves the TICC problem through a scalable algorithm that is able to efficiently solve for tens of millions of observations.
Abstract: Subsequence clustering of multivariate time series is a useful tool for discovering repeated patterns in temporal data Once these patterns have been discovered, seemingly complicated datasets can be interpreted as a temporal sequence of only a small number of states, or clusters For example, raw sensor data from a fitness-tracking application can be expressed as a timeline of a select few actions (ie, walking, sitting, running) However, discovering these patterns is challenging because it requires simultaneous segmentation and clustering of the time series Furthermore, interpreting the resulting clusters is difficult, especially when the data is high-dimensional Here we propose a new method of model-based clustering, which we call Toeplitz Inverse Covariance-based Clustering (TICC) Each cluster in the TICC method is defined by a correlation network, or Markov random field (MRF), characterizing the interdependencies between different observations in a typical subsequence of that cluster Based on this graphical representation, TICC simultaneously segments and clusters the time series data We solve the TICC problem through alternating minimization, using a variation of the expectation maximization (EM) algorithm We derive closed-form solutions to efficiently solve the two resulting subproblems in a scalable way, through dynamic programming and the alternating direction method of multipliers (ADMM), respectively We validate our approach by comparing TICC to several state-of-the-art baselines in a series of synthetic experiments, and we then demonstrate on an automobile sensor dataset how TICC can be used to learn interpretable clusters in real-world scenarios

Proceedings ArticleDOI
19 Apr 2018
TL;DR: Plug-and-Play (PnP) framework is used and the state-of-the-art deep residual learning for the image denoising operator is adopted which represents the prior model in MBIR, reducing the noise and artifacts compared to analytical reconstruction and standard MBIr with MRF prior.
Abstract: Model-Based Iterative Reconstruction (MBIR) has shown promising results in clinical studies as they allow significant dose reduction during CT scans while maintaining the diagnostic image quality. MBIR improves the image quality over analytical reconstruction by modeling both the sensor (e.g., forward model) and the image being reconstructed (e.g., prior model). While the forward model is typically based on the physics of the sensor, accurate prior modeling remains a challenging problem. Markov Random Field (MRF) has been widely used as prior models in MBIR due to simple structure, but they cannot completely capture the subtle characteristics of complex images. To tackle this challenge, we generate a prior model by learning the desirable image property from a large dataset. Toward this, we use Plug-and-Play (PnP) framework which decouples the forward model and the prior model in the optimization procedure, replacing the prior model optimization by a image denoising operator. Then, we adopt the state-of-the-art deep residual learning for the image denoising operator which represents the prior model in MBIR. Experimental results on real CT scans demonstrate that our PnP MBIR with deep residual learning prior significantly reduces the noise and artifacts compared to analytical reconstruction and standard MBIR with MRF prior.

Journal ArticleDOI
TL;DR: An improved Fuzzy C-Means clustering method, which incorporates geometric symmetry information, is proposed for infrared pedestrian segmentation, which performs better and obtains better segmentation results compared with other state-of-the-art methods.
Abstract: Pedestrian detection in infrared images is always a challenging task. Segmentation is an important step of pedestrian detection. An accurate segmentation could provide more information for further analysis. In this paper, an improved Fuzzy C-Means clustering method, which incorporates geometric symmetry information, is proposed for infrared pedestrian segmentation. In the proposed method, symmetry information is introduced by Markov random field theory. Moreover, a new metric is utilized to handle the weak symmetry of pedestrian. In addition, a whole procedure is proposed to extract infrared pedestrians. The experimental results indicate that our method performs better for infrared pedestrian segmentation and obtains better segmentation results compared with other state-of-the-art methods.

Journal ArticleDOI
Chao Zhao, Jingchi Jiang1, Yi Guan1, Xitong Guo1, Bin He1 
TL;DR: In this paper, the authors proposed a general system that can extract and represent knowledge contained in EMRs to support three clinical decision support tasks (test recommendation, initial diagnosis, and treatment plan recommendation) given the condition of a patient.

Posted Content
Linchao Bao1, Baoyuan Wu1, Wei Liu1
TL;DR: In this paper, a spatio-temporal Markov Random Field (MRF) model is proposed over pixels to handle the problem of video object segmentation, where the initial object mask is given in the first frame of an input video.
Abstract: This paper addresses the problem of video object segmentation, where the initial object mask is given in the first frame of an input video. We propose a novel spatio-temporal Markov Random Field (MRF) model defined over pixels to handle this problem. Unlike conventional MRF models, the spatial dependencies among pixels in our model are encoded by a Convolutional Neural Network (CNN). Specifically, for a given object, the probability of a labeling to a set of spatially neighboring pixels can be predicted by a CNN trained for this specific object. As a result, higher-order, richer dependencies among pixels in the set can be implicitly modeled by the CNN. With temporal dependencies established by optical flow, the resulting MRF model combines both spatial and temporal cues for tackling video object segmentation. However, performing inference in the MRF model is very difficult due to the very high-order dependencies. To this end, we propose a novel CNN-embedded algorithm to perform approximate inference in the MRF. This algorithm proceeds by alternating between a temporal fusion step and a feed-forward CNN step. When initialized with an appearance-based one-shot segmentation CNN, our model outperforms the winning entries of the DAVIS 2017 Challenge, without resorting to model ensembling or any dedicated detectors.

Journal ArticleDOI
TL;DR: A new segmentation framework based on supervoxels is proposed to solve the existing challenges of previous methods and relieves the dependency on the non‐linear pairwise registration.

Journal ArticleDOI
TL;DR: The experiments show that the proposed CDFFCM is superior to three comparative approaches in terms of higher accuracy, fewer parameters, and shorter execution time.
Abstract: Change detection approaches based on image segmentation are often used for landslide mapping (LM) from very high-resolution (VHR) remote sensing images. However, these approaches usually have two limitations. One is that they are sensitive to thresholds used for image segmentation and require too many parameters. The other one is that the computational complexity of these approaches depends on the image size, and thus they require a long execution time for very high-resolution (VHR) remote sensing images. In this paper, an unsupervised change detection using fast fuzzy c-means clustering (CDFFCM) for LM is proposed. The proposed CDFFCM has two contributions. The first is that we employ a Gaussian pyramid-based fast fuzzy c-means (FCM) clustering algorithm to obtain candidate landslide regions that have a better visual effect due to the utilization of image spatial information. The second is that we use the difference of image structure information instead of grayscale difference to obtain more accurate landslide regions. Three comparative approaches, edge-based level-set (ELSE), region-based level-set (RLSE), and change detection-based Markov random field (CDMRF), and the proposed CDFFCM are evaluated in three true landslide cases in the Lantau area of Hong Kong. The experiments show that the proposed CDFFCM is superior to three comparative approaches in terms of higher accuracy, fewer parameters, and shorter execution time.

Journal ArticleDOI
TL;DR: The proposed inversion method can effectively remove the anisotropic features of the model gradients and significantly improve the inversion results, especially for geologically layered formations.
Abstract: Prestack seismic inversion is an ill-posed problem and must be regularized to stabilize the inverted results. In particular, edge-preserving regularization with prior constraints based on Markov random field (MRF) has proved to be an effective technique for reconstructing subsurface models. However, regularized seismic inversion, based on the standard MRF scheme, typically makes use of isotropic MRF neighborhoods, in which the weighting coefficients of the model gradients are equivalent in all directions. Considering real geological conditions, subsurface formations are expected to be laterally continuous and vertically stratified. Therefore, the anisotropic effects caused by model gradients which vary along different directions should not be ignored. In this paper, we proposed a new prestack seismic inversion method based on anisotropic MRF (AMRF). In this method, AMRF coefficients are incorporated into the standard MRF scheme. These coefficients demonstrate directional variations and gradient dependencies, intended to directly correct the errors caused by the anisotropic model gradients on the prior constraints. In particular, we introduced the anisotropic diffusion method to calculate the AMRF coefficients. The proposed inversion method can effectively remove the anisotropic features of the model gradients and significantly improve the inversion results, especially for geologically layered formations. We demonstrated the effectiveness of the inversion method by both 2-D synthetic test and field data example, which presented encouraging results.