Showing papers on "Variable-order Bayesian network published in 2018"

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications.

Abstract: Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).

Bayesian data analysis for newcomers

Abstract: This article explains the foundational concepts of Bayesian data analysis using virtually no mathematical notation. Bayesian ideas already match your intuitions from everyday reasoning and from traditional data analysis. Simple examples of Bayesian data analysis are presented that illustrate how the information delivered by a Bayesian analysis can be directly interpreted. Bayesian approaches to null-value assessment are discussed. The article clarifies misconceptions about Bayesian methods that newcomers might have acquired elsewhere. We discuss prior distributions and explain how they are not a liability but an important asset. We discuss the relation of Bayesian data analysis to Bayesian models of mind, and we briefly discuss what methodological problems Bayesian data analysis is not meant to solve. After you have read this article, you should have a clear sense of how Bayesian data analysis works and the sort of information it delivers, and why that information is so intuitive and useful for drawing conclusions from data.

Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau

Abstract: Modern imaging methods rely strongly on Bayesian inference techniques to solve challenging imaging problems. Currently, the predominant Bayesian computation approach is convex optimization, which scales very efficiently to high-dimensional image models and delivers accurate point estimation results. However, in order to perform more complex analyses, for example, image uncertainty quantification or model selection, it is necessary to use more computationally intensive Bayesian computation techniques such as Markov chain Monte Carlo methods. This paper presents a new and highly efficient Markov chain Monte Carlo methodology to perform Bayesian computation for high-dimensional models that are log-concave and nonsmooth, a class of models that is central in imaging sciences. The methodology is based on a regularized unadjusted Langevin algorithm that exploits tools from convex analysis, namely, Moreau--Yoshida envelopes and proximal operators, to construct Markov chains with favorable convergence properties. ...

A guide to Bayesian model checking for ecologists

Abstract: Checking that models adequately represent data is an essential component of applied statistical inference. Ecologists increasingly use hierarchical Bayesian statistical models in their research. The appeal of this modeling paradigm is undeniable, as researchers can build and fit models that embody complex ecological processes while simultaneously controlling observation error. However, ecologists tend to be less focused on checking model assumptions and assessing potential lack-of-fit when applying Bayesian methods than when applying more traditional modes of inference such as maximum likelihood. There are also multiple ways of assessing the fit of Bayesian models, each of which has strengths and weaknesses. For instance, Bayesian p-values are relatively easy to compute, but are well known to be conservative, producing p-values biased toward 0.5. Alternatively, lesser known approaches to model checking, such as prior predictive checks, cross-validation probability integral transforms, and pivot discrepancy measures may produce more accurate characterizations of goodness-of-fit but are not as well known to ecologists. In addition, a suite of visual and targeted diagnostics can be used to examine violations of different model assumptions and lack-of-fit at different levels of the modeling hierarchy, and to check for residual temporal or spatial autocorrelation. In this review, we synthesize existing literature to guide ecologists through the many available options for Bayesian model checking. We illustrate methods and procedures with several ecological case studies, including i) analysis of simulated spatio-temporal count data, (ii) N-mixture models for estimating abundance and detection probability of sea otters from an aircraft, and (iii) hidden Markov modeling to describe attendance patterns of California sea lion mothers on a rookery. We find that commonly used procedures based on posterior predictive p-values detect extreme model inadequacy, but often do not detect more subtle cases of lack of fit. Tests based on cross-validation and pivot discrepancy measures (including the ``sampled predictive p-value'') appear to be better suited to model checking and to have better overall statistical performance. We conclude that model checking is an essential component of scientific discovery and learning that should accompany most Bayesian analyses presented in the literature.

Bayesian Inference in the Presence of Intractable Normalizing Functions

Abstract: Models with intractable normalizing functions arise frequently in statistics. Common examples of such models include exponential random graph models for social networks and Markov point processes f...

A primer on Bayesian distributional regression

Abstract: :Bayesian methods have become increasingly popular in the past two decades. With the constant rise of computational power, even very complex models can be estimated on virtually any modern computer. Moreover, interest has shifted from conditional mean models to probabilistic distributional models capturing location, scale, shape and other aspects of a response distribution, where covariate effects can have flexible forms, for example, linear, non-linear, spatial or random effects. This tutorial article discusses how to select models in the Bayesian distributional regression setting, how to monitor convergence of the Markov chains and how to use simulation-based inference also for quantities derived from the original model parametrization. We exemplify the workflow using daily weather data on (a) temperatures on Germany's highest mountain and (b) extreme values of precipitation for the whole of Germany.

Scalable Bayesian Modeling, Monitoring, and Analysis of Dynamic Network Flow Data

Abstract: Traffic flow count data in networks arise in many applications, such as automobile or aviation transportation, certain directed social network contexts, and Internet studies. Using an example of In...

Optimal Bayesian estimators for latent variable cluster models.

Abstract: In cluster analysis interest lies in probabilistically capturing partitions of individuals, items or observations into groups, such that those belonging to the same group share similar attributes or relational profiles. Bayesian posterior samples for the latent allocation variables can be effectively obtained in a wide range of clustering models, including finite mixtures, infinite mixtures, hidden Markov models and block models for networks. However, due to the categorical nature of the clustering variables and the lack of scalable algorithms, summary tools that can interpret such samples are not available. We adopt a Bayesian decision theoretical approach to define an optimality criterion for clusterings and propose a fast and context-independent greedy algorithm to find the best allocations. One important facet of our approach is that the optimal number of groups is automatically selected, thereby solving the clustering and the model-choice problems at the same time. We consider several loss functions to compare partitions and show that our approach can accommodate a wide range of cases. Finally, we illustrate our approach on both artificial and real datasets for three different clustering models: Gaussian mixtures, stochastic block models and latent block models for networks.

Bayesian correction for covariate measurement error: A frequentist evaluation and comparison with regression calibration

Abstract: Bayesian approaches for handling covariate measurement error are well established and yet arguably are still relatively little used by researchers. For some this is likely due to unfamiliarity or disagreement with the Bayesian inferential paradigm. For others a contributory factor is the inability of standard statistical packages to perform such Bayesian analyses. In this paper, we first give an overview of the Bayesian approach to handling covariate measurement error, and contrast it with regression calibration, arguably the most commonly adopted approach. We then argue why the Bayesian approach has a number of statistical advantages compared to regression calibration and demonstrate that implementing the Bayesian approach is usually quite feasible for the analyst. Next, we describe the closely related maximum likelihood and multiple imputation approaches and explain why we believe the Bayesian approach to generally be preferable. We then empirically compare the frequentist properties of regression calibration and the Bayesian approach through simulation studies. The flexibility of the Bayesian approach to handle both measurement error and missing data is then illustrated through an analysis of data from the Third National Health and Nutrition Examination Survey.

Bayesian analysis of the piecewise diffusion decision model.

Abstract: Most past research on sequential sampling models of decision-making have assumed a time homogeneous process (i.e., parameters such as drift rates and boundaries are constant and do not change during the deliberation process). This has largely been due to the theoretical difficulty in testing and fitting more complex models. In recent years, the development of simulation-based modeling approaches matched with Bayesian fitting methodologies has opened the possibility of developing more complex models such as those with time-varying properties. In the present work, we discuss a piecewise variant of the well-studied diffusion decision model (termed pDDM) that allows evidence accumulation rates to change during the deliberation process. Given the complex, time-varying nature of this model, standard Bayesian parameter estimation methodologies cannot be used to fit the model. To overcome this, we apply a recently developed simulation-based, hierarchal Bayesian methodology called the probability density approximation (PDA) method. We provide an analysis of this methodology and present results of parameter recovery experiments to demonstrate the strengths and limitations of this approach. With those established, we fit pDDM to data from a perceptual experiment where information changes during the course of trials. This extensible modeling platform opens the possibility of applying sequential sampling models to a range of complex non-stationary decision tasks.

Fitting growth curve models in the Bayesian framework

Abstract: Growth curve modeling is a popular methodological tool due to its flexibility in simultaneously analyzing both within-person effects (e.g., assessing change over time for one person) and between-person effects (e.g., comparing differences in the change trajectories across people). This paper is a practical exposure to fitting growth curve models in the hierarchical Bayesian framework. First the mathematical formulation of growth curve models is provided. Then we give step-by-step guidelines on how to fit these models in the hierarchical Bayesian framework with corresponding computer scripts (JAGS and R). To illustrate the Bayesian GCM approach, we analyze a data set from a longitudinal study of marital relationship quality. We provide our computer code and example data set so that the reader can have hands-on experience fitting the growth curve model.

Bayesian inference for psychology, part III: Parameter estimation in nonstandard models

Abstract: We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked examples; the examples involve a simple univariate linear regression and fitting a multinomial processing tree model to data from a classic false-memory experiment. We conclude with a comparison of the strengths and weaknesses of the packages. Our example code, data, and this text are available via https://osf.io/ucmaz/ .

Bayesian latent variable models for the analysis of experimental psychology data

Abstract: In this paper, we address the use of Bayesian factor analysis and structural equation models to draw inferences from experimental psychology data. While such application is non-standard, the models are generally useful for the unified analysis of multivariate data that stem from, e.g., subjects' responses to multiple experimental stimuli. We first review the models and the parameter identification issues inherent in the models. We then provide details on model estimation via JAGS and on Bayes factor estimation. Finally, we use the models to re-analyze experimental data on risky choice, comparing the approach to simpler, alternative methods.

A Bayesian Network model for predicting cooling load of commercial buildings

Abstract: Cooling load prediction is indispensable to many building energy saving strategies. In this paper, we proposed a new method for predicting the cooling load of commercial buildings. The proposed approach employs a Bayesian Network model to relate the cooling load to outdoor weather conditions and internal building activities. The proposed method is computationally efficient and implementable for use in real buildings, as it does not involve sophisticated mathematical theories. In this paper, we described the proposed method and demonstrated its use via a case study. In this case study, we considered three candidate models for cooling load prediction and they are the proposed Bayesian Network model, a Support Vector Machine model, and an Artificial Neural Network model. We trained the three models with fourteen different training data datasets, each of which had varying amounts and quality of data that were sampled on-site. The prediction results for a testing week shows that the Bayesian Network model achieves similar accuracy as the Support Vector Machine model but better accuracy than the Artificial Neural Network model. Notable in this comparison is that the training process of the Bayesian Network model is fifty-eight times faster than that of the Artificial Neural Network model. The results also suggest that all three models will have much larger prediction deviations if the testing data points are not covered by the training dataset for the studied case (The maximum absolute deviation of the predictions that are not covered by the training dataset can be up to seven times larger than that of the predictions covered by the training dataset). In addition, we also found the uncertainties in the weather forecast significantly affected the accuracy of the cooling load prediction for the studied case and the Support Vector Machine model was more sensitive to those uncertainties than the other two models.

Bayesian approaches to distribution regression

Abstract: Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level. Current approaches, however, do not propagate the uncertainty in observations due to sampling variability in the groups. This effectively assumes that small and large groups are estimated equally well, and should have equal weight in the final regression. We account for this uncertainty with a Bayesian distribution regression formalism, improving the robustness and performance of the model when group sizes vary. We frame our models in a neural network style, allowing for simple MAP inference using backpropagation to learn the parameters, as well as MCMC-based inference which can fully propagate uncertainty. We demonstrate our approach on illustrative toy datasets, as well as on a challenging problem of predicting age from images.

Using Bayesian regression to test hypotheses about relationships between parameters and covariates in cognitive models

Abstract: An important tool in the advancement of cognitive science are quantitative models that represent different cognitive variables in terms of model parameters. To evaluate such models, their parameters are typically tested for relationships with behavioral and physiological variables that are thought to reflect specific cognitive processes. However, many models do not come equipped with the statistical framework needed to relate model parameters to covariates. Instead, researchers often revert to classifying participants into groups depending on their values on the covariates, and subsequently comparing the estimated model parameters between these groups. Here we develop a comprehensive solution to the covariate problem in the form of a Bayesian regression framework. Our framework can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors. Moreover, we present a simulation study that demonstrates the superiority of the Bayesian regression framework to the conventional classification-based approach.

Model selection and parameter estimation in tumor growth models using approximate Bayesian computation-ABC

Abstract: Cancer is one of the most fatal diseases in the world. Governments and researchers from various areas have continuously concentrated efforts to better understand the disease and propose diagnostic and treatment techniques. The use of mathematical models of tumor growth is of great importance for the development of such techniques. Due to the variety of models nowadays available in the literature, the problems of model selection and parameter estimation come into picture, aiming at suitably predicting the patient’s status of the disease. As the available data on dependent variables of existing models might not justify the use of common likelihood functions, approximate Bayesian computation (ABC) becomes a very attractive tool for model selection and model calibration (parameter estimation) in tumor growth models. In the present study, a Monte Carlo approximate Bayesian computation (ABC) algorithm is applied to select among competing models of tumor growth, with and without chemotherapy treatment. Simulated measurements are used in this work. The results obtained show that the algorithm correctly selects the model and estimates the parameters used to generate the simulated measurements.

Bayesian Functional Optimization

Abstract: Bayesian optimization (BayesOpt) is a derivative-free approach for sequentially optimizing stochastic black-box functions. Standard BayesOpt, which has shown many successes in machine learning applications, assumes a finite dimensional domain which often is a parametric space. The parameter space is defined by the features used in the function approximations which are often selected manually. Therefore, the performance of BayesOpt inevitably depends on the quality of chosen features. This paper proposes a new Bayesian optimization framework that is able to optimize directly on the domain of function spaces. The resulting framework, Bayesian Functional Optimization (BFO), not only extends the application domains of BayesOpt to functional optimization problems but also relaxes the performance dependency on the chosen parameter space. We model the domain of functions as a reproducing kernel Hilbert space (RKHS), and use the notion of Gaussian processes on a real separable Hilbert space. As a result, we are able to define traditional improvement-based (PI and EI) and optimistic acquisition functions (UCB) as functionals. We propose to optimize the acquisition functionals using analytic functional gradients that are also proved to be functions in a RKHS. We evaluate BFO in three typical functional optimization tasks: i) a synthetic functional optimization problem, ii) optimizing activation functions for a multi-layer perceptron neural network, and iii) a reinforcement learning task whose policies are modeled in RKHS.

Energy performance evaluation of OECD countries using Bayesian stochastic frontier analysis and Bayesian network classifiers

Abstract: More recently a large amount of interest has been devoted to the use of Bayesian methods for deriving parameter estimates of the stochastic frontier analysis. Bayesian stochastic frontier analysis (BSFA) seems to be a useful method to assess the efficiency in energy sector. However, BSFA results do not expose the multiple relationships between input and output variables and energy efficiency. This study proposes a framework to make inferences about BSFA efficiencies, recognizing the underlying relationships between variables and efficiency, using Bayesian network (BN) approach. BN classifiers are proposed as a method to analyze the results obtained from BSFA.

Uncertainty Analysis by Bayesian Inference

Abstract: The scientific methodology of mathematical models and their credibility to form the basis of public policy decisions have been frequently challenged. The development of novel methods for rigorously assessing the uncertainty underlying model predictions is one of the priorities of the modeling community. Striving for novel uncertainty analysis tools, we present the Bayesian calibration of process-based models as a methodological advancement that warrants consideration in ecosystem analysis and biogeochemical research. This modeling framework combines the advantageous features of both process-based and statistical approaches; that is, mechanistic understanding that remains within the bounds of data-based parameter estimation. The incorporation of mechanisms improves the confidence in predictions made for a variety of conditions, whereas the statistical methods provide an empirical basis for parameter value selection and allow for realistic estimates of predictive uncertainty. Other advantages of the Bayesian approach include the ability to sequentially update beliefs as new knowledge is available, the rigorous assessment of the expected consequences of different management actions, the optimization of the sampling design of monitoring programs, and the consistency with the scientific process of progressive learning and the policy practice of adaptive management. We illustrate some of the anticipated benefits from the Bayesian calibration framework, well suited for stakeholders and policy makers when making environmental management decisions, using the Hamilton Harbour and the Bay of Quinte—two eutrophic systems in Ontario, Canada—as case studies.

Probabilistic Model Checking for Continuous-Time Markov Chains via Sequential Bayesian Inference

Abstract: Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while statistical approaches require a large number of samples to estimate the desired properties with high confidence. Here, we show how model checking of time-bounded path properties can be recast exactly as a Bayesian inference problem. In this novel formulation the problem can be efficiently approximated using techniques from machine learning. Our approach is inspired by a recent result in statistical physics which derived closed-form differential equations for the first-passage time distribution of stochastic processes. We show on a number of non-trivial case studies that our method achieves both high accuracy and significant computational gains compared to statistical model checking.

Combination of expert decision and learned based Bayesian Networks for multi-scale mechanical analysis of timber elements

Abstract: The use of Bayesian Networks allows to organize and correlate information gathered from different sources and its optimization may incorporate restrictions adjusting the network based on expert knowledge and network operativeness, in such a way that it may satisfactorily represent a given domain. The main goal of this paper is to study if an optimized learned Bayesian Network may be used as a prior structure for an expert based network of an engineering structural material analysis. The methodology is applied to a database of results from an experimental campaign that focused on the mechanical characterization of timber elements recovered from an early 20th century building. To that study case it is evidenced that through a suitable combination of model averaging and supervision steps it is possible to achieve robust and reliable models to underpin the causal structure of a typical multi-scale timber analysis.

Continuous time Markov chain approaches for analyzing transtheoretical models of health behavioral change: A case study and comparison of model estimations.

Abstract: Continuous time Markov chain models are frequently employed in medical research to study the disease progression but are rarely applied to the transtheoretical model, a psychosocial model widely used in the studies of health-related outcomes. The transtheoretical model often includes more than three states and conceptually allows for all possible instantaneous transitions (referred to as general continuous time Markov chain). This complicates the likelihood function because it involves calculating a matrix exponential that may not be simplified for general continuous time Markov chain models. We undertook a Bayesian approach wherein we numerically evaluated the likelihood using ordinary differential equation solvers available from the gnu scientific library. We compared our Bayesian approach with the maximum likelihood method implemented with the R package MSM. Our simulation study showed that the Bayesian approach provided more accurate point and interval estimates than the maximum likelihood method, especially in complex continuous time Markov chain models with five states. When applied to data from a four-state transtheoretical model collected from a nutrition intervention study in the next step trial, we observed results consistent with the results of the simulation study. Specifically, the two approaches provided comparable point estimates and standard errors for most parameters, but the maximum likelihood offered substantially smaller standard errors for some parameters. Comparable estimates of the standard errors are obtainable from package MSM, which works only when the model estimation algorithm converges.

Stochastic Efficiency of Bayesian Markov Chain Monte Carlo in Spatial Econometric Models: An Empirical Comparison of Exact Sampling Methods

Abstract: Spatial econometric specifications pose unique computational challenges to Bayesian analysis, making it difficult to estimate models efficiently. In the literature, the main focus has been on extending Bayesian analysis to increasingly complex spatial models. The stochastic efficiency of commonly used Markov Chain Monte Carlo (MCMC) samplers has received less attention by comparison. Specifically, Bayesian methods to analyze effective sample size and samplers that provide large effective size have not been thoroughly considered in the literature. Thus, we compare three MCMC techniques: the familiar Metropolis-within-Gibbs sampling, Slice-within-Gibbs sampling, and Hamiltonian Monte Carlo. The latter two methods, while common in other domains, are not as widely encountered in Bayesian spatial econometrics. We assess these methods across four different scenarios in which we estimate the spatial autoregressive parameter in a mixed regressive, spatial autoregressive specification (or, spatial lag model). We find that off-the-shelf implementations of the newer high-yield simulation techniques require significant adaptation to be viable. We further find that the effective sizes are often significantly smaller than nominal sizes. In addition, we find that stopping simulation early may understate posterior credible interval widths when effective sample size is small. More broadly, we suggest that sample information and stopping rules deserve more attention in both applied and basic Bayesian spatial econometric research.