scispace - formally typeset
Search or ask a question

Showing papers on "Bayesian inference published in 2008"


Book
25 Aug 2008
TL;DR: An overview of model-based geostatistics can be found in this paper, where a generalized linear model is proposed for estimating geometrical properties of geometrically constrained data.
Abstract: An overview of model-based geostatistics.- Gaussian models for geostatistical data.- Generalized linear models for geostatistical data.- Classical parameter estimation.- Spatial prediction.- Bayesian inference.- Geostatistical design.

2,397 citations


01 Jan 2008
TL;DR: Procedures of statistical inference are described which generalize Bayesian inference in specific ways and some comments are made on the general class of models which produce upper and lower probabilities, and on the specific models which underlie the suggested inference procedures.
Abstract: Procedures of statistical inference are described which generalize Bayesian inference in specific ways. Probability is used in such a way that in general only bounds may be placed on the probabilities of given events, and probability systems of this kind are suggested both for sample information and for prior information. These systems are then combined using a specified rule. Illustrations are given for inferences about trinomial probabilities, and for inferences about a monotone sequence of binomial pi. Finally, some comments are made on the general class of models which produce upper and lower probabilities, and on the specific models which underlie the suggested inference procedures.

1,478 citations


Journal ArticleDOI
TL;DR: This review is an introduction to Bayesian methods in cosmology and astrophysics and recent results in the field, and presents Bayesian probability theory and its conceptual underpinnings, Bayes' Theorem and the role of priors.
Abstract: The application of Bayesian methods in cosmology and astrophysics has flourished over the past decade, spurred by data sets of increasing size and complexity. In many respects, Bayesian methods have proven to be vastly superior to more traditional statistical tools, offering the advantage of higher efficiency and of a consistent conceptual basis for dealing with the problem of induction in the presence of uncertainty. This trend is likely to continue in the future, when the way we collect, manipulate and analyse observations and compare them with theoretical models will assume an even more central role in cosmology. This review is an introduction to Bayesian methods in cosmology and astrophysics and recent results in the field. I first present Bayesian probability theory and its conceptual underpinnings, Bayes' Theorem and the role of priors. I discuss the problem of parameter inference and its general solution, along with numerical techniques such as Monte Carlo Markov Chain methods. I then review the th...

962 citations


Journal ArticleDOI
TL;DR: It was from here that "Bayesian" ideas first spread through the mathematical world, as Bayes's own article was ignored until 1780 and played no important role in scientific debate until the 20th century.
Abstract: The influence of this Thomas Bayes' work was immense. It was from here that "Bayesian" ideas first spread through the mathematical world, as Bayes's own article was ignored until 1780 and played no important role in scientific debate until the 20th century. It was also this article of Laplace's that introduced the mathematical techniques for the asymptotic analysis of posterior distributions that are still employed today. And it was here that the earliest example of optimum estimation can be found, the derivation and characterization of an estimator that minimized a particular measure of posterior expected loss. After more than two centuries, we mathematicians, statisticians cannot only recognize our roots in this masterpiece of our science, we can still learn from it.

774 citations


Journal ArticleDOI
TL;DR: A general model that subsumes many parametric models for continuous data that can be inverted using exactly the same scheme, namely, dynamic expectation maximization, and is formulated as a simple neural network that may provide a useful metaphor for inference and learning in the brain.
Abstract: This paper describes a general model that subsumes many parametric models for continuous data. The model comprises hidden layers of state-space or dynamic causal models, arranged so that the output of one provides input to another. The ensuing hierarchy furnishes a model for many types of data, of arbitrary complexity. Special cases range from the general linear model for static data to generalised convolution models, with system noise, for nonlinear time-series analysis. Crucially, all of these models can be inverted using exactly the same scheme, namely, dynamic expectation maximization. This means that a single model and optimisation scheme can be used to invert a wide range of models. We present the model and a brief review of its inversion to disclose the relationships among, apparently, diverse generative models of empirical data. We then show that this inversion can be formulated as a simple neural network and may provide a useful metaphor for inference and learning in the brain.

771 citations


Book
30 Jun 2008
TL;DR: Approaches for statistical inference Motivating Vignettes Defining the Approaches The Bayes-Frequentist Controversy Some Basic Bayesian Models The Bayesian approach
Abstract: Approaches for statistical inference Introduction Motivating Vignettes Defining the Approaches The Bayes-Frequentist Controversy Some Basic Bayesian Models The Bayes approach Introduction Prior Distributions Bayesian Inference Hierarchical Modeling Model Assessment Nonparametric Methods Bayesian computation Introduction Asymptotic Methods Noniterative Monte Carlo Methods Markov Chain Monte Carlo Methods Model criticism and selection Bayesian Modeling Bayesian Robustness Model Assessment Bayes Factors via Marginal Density Estimation Bayes Factors via Sampling over the Model Space Other Model Selection Methods The empirical Bayes approach Introduction Parametric EB Point Estimation Nonparametric EB Point Estimation Interval Estimation Bayesian Processing and Performance Frequentist Performance Empirical Bayes Performance Bayesian design Principles of Design Bayesian Clinical Trial Design Applications in Drug and Medical Device Trials Special methods and models Estimating Histograms and Ranks Order Restricted Inference Longitudinal Data Models Continuous and Categorical Time Series Survival Analysis and Frailty Models Sequential Analysis Spatial and Spatio-Temporal Models Case studies Analysis of Longitudinal AIDS Data Robust Analysis of Clinical Trials Modeling of Infectious Diseases Appendices Distributional Catalog Decision Theory Answers to Selected Exercises References Author Index Subject Index Index Exercises appear at the end of each chapter.

756 citations


Journal ArticleDOI
TL;DR: The Extended Bayesian Skyline Plot is presented, a non-parametric Bayesian Markov chain Monte Carlo algorithm that extends a previous coalescent-based method in several ways, including the ability to analyze multiple loci, demonstrating the essential role of multiple loco in recovering population size dynamics.
Abstract: Effective population size (N e ) is related to genetic variability and is a basic parameter in many models of population genetics. A number of methods for inferring current and past population sizes from genetic data have been developed since JFC Kingman introduced the n-coalescent in 1982. Here we present the Extended Bayesian Skyline Plot, a non-parametric Bayesian Markov chain Monte Carlo algorithm that extends a previous coalescent-based method in several ways, including the ability to analyze multiple loci. Through extensive simulations we show the accuracy and limitations of inferring population size as a function of the amount of data, including recovering information about evolutionary bottlenecks. We also analyzed two real data sets to demonstrate the behavior of the new method; a single gene Hepatitis C virus data set sampled from Egypt and a 10 locus Drosophila ananassae data set representing 16 different populations. The results demonstrate the essential role of multiple loci in recovering population size dynamics. Multi-locus data from a small number of individuals can precisely recover past bottlenecks in population size which can not be characterized by analysis of a single locus. We also demonstrate that sequence data quality is important because even moderate levels of sequencing errors result in a considerable decrease in estimation accuracy for realistic levels of population genetic variability.

697 citations


Journal ArticleDOI
TL;DR: Five uncertainty analysis procedures for watershed models are compared and if computationally feasible, Bayesian-based approaches are most recommendable because of their solid conceptual basis, but construction and test of the likelihood function requires critical attention.

684 citations


Journal ArticleDOI
TL;DR: Simulations are presented showing that the model can account for key findings: data on the segmentation of continuous speech, word frequency effects, the effects of mispronunciations on word recognition, and evidence on lexical involvement in phonemic decision making.
Abstract: A Bayesian model of continuous speech recognition is presented. It is based on Shortlist (D. Norris, 1994; D. Norris, J. M. McQueen, A. Cutler, & S. Butterfield, 1997) and shares many of its key assumptions: parallel competitive evaluation of multiple lexical hypotheses, phonologically abstract prelexical and lexical representations, a feedforward architecture with no online feedback, and a lexical segmentation algorithm based on the viability of chunks of the input as possible words. Shortlist B is radically different from its predecessor in two respects. First, whereas Shortlist was a connectionist model based on interactive-activation principles, Shortlist B is based on Bayesian principles. Second, the input to Shortlist B is no longer a sequence of discrete phonemes; it is a sequence of multiple phoneme probabilities over 3 time slices per segment, derived from the performance of listeners in a large-scale gating study. Simulations are presented showing that the model can account for key findings: data on the segmentation of continuous speech, word frequency effects, the effects of mispronunciations on word recognition, and evidence on lexical involvement in phonemic decision making. The success of Shortlist B suggests that listeners make optimal Bayesian decisions during spoken-word recognition.

568 citations


Journal ArticleDOI
TL;DR: This work proposes a group inference approach which includes inference of outliers using a robust general linear model (GLM) approach which combines the benefits of outlier inference with thebenefits of using variance information from lower levels in the hierarchy.

489 citations


Book
05 Aug 2008
TL;DR: Bayesian Inference and Modeling Likelihood Models Prior Distributions Posterior Distributions Predictive Distributions Bayesian Hierarchical Models Posterior Inference Exercises Computational Issues Posterior Sampling Markov Chain Monte Carlo Methods Metropolis and Metropolis-Hastings Algorithms Gibbs Sampling Perfect Sampling Posterior and Likelihood Approximations Exercise Residuals and Goodness of Fit Model Goodness-of-Fit Measures General ResidUALs Bayesian Residualitys Predictive Residuallys and the Bootstrap
Abstract: BACKGROUND Introduction Data Sets Bayesian Inference and Modeling Likelihood Models Prior Distributions Posterior Distributions Predictive Distributions Bayesian Hierarchical Modeling Hierarchical Models Posterior Inference Exercises Computational Issues Posterior Sampling Markov Chain Monte Carlo Methods Metropolis and Metropolis-Hastings Algorithms Gibbs Sampling Perfect Sampling Posterior and Likelihood Approximations Exercises Residuals and Goodness-of-Fit Model Goodness-of-Fit Measures General Residuals Bayesian Residuals Predictive Residuals and the Bootstrap Interpretation of Residuals in a Bayesian Setting Exceedence Probabilities Exercises THEMES Disease Map Reconstruction and Relative Risk Estimation An Introduction to Case Event and Count Likelihoods Specification of the Predictor in Case Event and Count Models Simple Case and Count Data Models with Uncorrelated Random Effects Correlated Heterogeneity Models Convolution Models Model Comparison and Goodness-of-Fit Diagnostics Alternative Risk Models Edge Effects Exercises Disease Cluster Detection Cluster Definitions Cluster Detection using Residuals Cluster Detection using Posterior Measures Cluster Models Edge Detection and Wombling Ecological Analysis General Case of Regression Biases and Misclassification Error Putative Hazard Models Multiple Scale Analysis Modifiable Areal Unit Problem (MAUP) Misaligned Data Problem (MIDP) Multivariate Disease Analysis Notation for Multivariate Analysis Two Diseases Multiple Diseases Spatial Survival and Longitudinal Analyses General Issues Spatial Survival Analysis Spatial Longitudinal Analysis Extensions to Repeated Events Spatiotemporal Disease Mapping Case Event Data Count Data Alternative Models Infectious Diseases Appendix A: Basic R and WinBUGS Appendix B: Selected WinBUGS Code Appendix C: R Code for Thematic Mapping References Index

Posted Content
TL;DR: This work performs asymptotically optimal Bayesian inference using belief propagation (BP) decoding, which represents the CS encoding matrix as a graphical model, and focuses on a two-state mixture Gaussian model that is easily adapted to other signal models.
Abstract: Compressive sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable, sub-Nyquist signal acquisition. When a statistical characterization of the signal is available, Bayesian inference can complement conventional CS methods based on linear programming or greedy algorithms. We perform approximate Bayesian inference using belief propagation (BP) decoding, which represents the CS encoding matrix as a graphical model. Fast computation is obtained by reducing the size of the graphical model with sparse encoding matrices. To decode a length-N signal containing K large coefficients, our CS-BP decoding algorithm uses O(Klog(N)) measurements and O(Nlog^2(N)) computation. Finally, although we focus on a two-state mixture Gaussian model, CS-BP is easily adapted to other signal models.

Journal ArticleDOI
TL;DR: The proposed multimode process monitoring approach based on finite Gaussian mixture model (FGMM) and Bayesian inference strategy is superior to the conventional PCA method and can achieve accurate and early detection of various types of faults in multimode processes.
Abstract: For complex industrial processes with multiple operating conditions, the traditional multivariate process monitoring techniques such as principal component analysis (PCA) and partial least squares (PLS) are ill-suited because the fundamental assumption that the operating data follow a unimodal Gaussian distribution usually becomes invalid. In this article, a novel multimode process monitoring approach based on finite Gaussian mixture model (FGMM) and Bayesian inference strategy is proposed. First, the process data are assumed to be from a number of different clusters, each of which corresponds to an operating mode and can be characterized by a Gaussian component. In the absence of a priori process knowledge, the Figueiredo–Jain (F–J) algorithm is then adopted to automatically optimize the number of Gaussian components and estimate their statistical distribution parameters. With the obtained FGMM, a Bayesian inference strategy is further utilized to compute the posterior probabilities of each monitored sample belonging to the multiple components and derive an integrated global probabilistic index for fault detection of multimode processes. The validity and effectiveness of the proposed monitoring approach are illustrated through three examples: (1) a simple multivariate linear system, (2) a simulated continuous stirred tank heater (CSTH) process, and (3) the Tennessee Eastman challenge problem. The comparison of monitoring results demonstrates that the proposed approach is superior to the conventional PCA method and can achieve accurate and early detection of various types of faults in multimode processes. © 2008 American Institute of Chemical Engineers AIChE J, 2008

Journal ArticleDOI
TL;DR: A method for modeling the spatial genetic structure using a combination of analytical and stochastic methods is developed by extending a novel theory of Bayesian predictive classification with the spatial information available, described here in terms of a colored Voronoi tessellation over the sample domain.
Abstract: Natural populations of living organisms often have complex histories consisting of phases of expansion and decline, and the migratory patterns within them may fluctuate over space and time. When parts of a population become relatively isolated, e.g., due to geographical barriers, stochastic forces reshape certain DNA characteristics of the individuals over generations such that they reflect the restricted migration and mating/reproduction patterns. Such populations are typically termed as genetically structured and they may be statistically represented in terms of several clusters between which DNA variations differ clearly from each other. When detailed knowledge of the ancestry of a natural population is lacking, the DNA characteristics of a sample of current generation individuals often provide a wealth of information in this respect. Several statistical approaches to model-based clustering of such data have been introduced, and in particular, the Bayesian approach to modeling the genetic structure of a population has attained a vivid interest among biologists. However, the possibility of utilizing spatial information from sampled individuals in the inference about genetic clusters has been incorporated into such analyses only very recently. While the standard Bayesian hierarchical modeling techniques through Markov chain Monte Carlo simulation provide flexible means for describing even subtle patterns in data, they may also result in computationally challenging procedures in practical data analysis. Here we develop a method for modeling the spatial genetic structure using a combination of analytical and stochastic methods. We achieve this by extending a novel theory of Bayesian predictive classification with the spatial information available, described here in terms of a colored Voronoi tessellation over the sample domain. Our results for real and simulated data sets illustrate well the benefits of incorporating spatial information to such an analysis.

Journal ArticleDOI
TL;DR: MultiNest as mentioned in this paper is a multimodal nested sampling algorithm that produces posterior samples from distributions that may contain multiple modes and pronounced (curving) degeneracies in high dimensions.
Abstract: We present further development and the first public release of our multimodal nested sampling algorithm, called MultiNest. This Bayesian inference tool calculates the evidence, with an associated error estimate, and produces posterior samples from distributions that may contain multiple modes and pronounced (curving) degeneracies in high dimensions. The developments presented here lead to further substantial improvements in sampling efficiency and robustness, as compared to the original algorithm presented in Feroz & Hobson (2008), which itself significantly outperformed existing MCMC techniques in a wide range of astrophysical inference problems. The accuracy and economy of the MultiNest algorithm is demonstrated by application to two toy problems and to a cosmological inference problem focussing on the extension of the vanilla $\Lambda$CDM model to include spatial curvature and a varying equation of state for dark energy. The MultiNest software, which is fully parallelized using MPI and includes an interface to CosmoMC, is available at this http URL It will also be released as part of the SuperBayeS package, for the analysis of supersymmetric theories of particle physics, at this http URL

Journal ArticleDOI
TL;DR: DIC is shown to be an approximation to a penalized loss function based on the deviance, with a penalty derived from a cross-validation argument, which under-penalizes more complex models.
Abstract: The deviance information criterion (DIC) is widely used for Bayesian model comparison, despite the lack of a clear theoretical foundation. DIC is shown to be an approximation to a penalized loss function based on the deviance, with a penalty derived from a cross-validation argument. This approximation is valid only when the effective number of parameters in the model is much smaller than the number of independent observations. In disease mapping, a typical application of DIC, this assumption does not hold and DIC under-penalizes more complex models. Another deviance-based loss function, derived from the same decision-theoretic framework, is applied to mixture models, which have previously been considered an unsuitable application for DIC.

Book
11 Mar 2008
TL;DR: In this article, the authors presented a model for handling nonignorable missingness in a clinical trial of Recombinant Human Growth Hormone (rhGH) for increasing muscle strength in the elderly.
Abstract: PREFACE Description of Motivating Examples Overview Dose-Finding Trial of an Experimental Treatment for Schizophrenia Clinical Trial of Recombinant Human Growth Hormone (rhGH) for Increasing Muscle Strength in the Elderly Clinical Trials of Exercise as an Aid to Smoking Cessation in Women: The Commit to Quit Studies Natural History of HIV Infection in Women: HIV Epidemiology Research Study (HERS) Cohort Clinical Trial of Smoking Cessation among Substance Abusers: OASIS Study Equivalence Trial of Competing Doses of AZT in HIV-Infected Children: Protocol 128 of the AIDS Clinical Trials Group Regression Models Overview Preliminaries Generalized Linear Models Conditionally Specified Models Directly Specified (Marginal) Models Semiparametric Regression Interpreting Covariate Effects Further Reading Methods of Bayesian Inference Overview Likelihood and Posterior Distribution Prior Distributions Computation of the Posterior Distribution Model Comparisons and Assessing Model Fit Nonparametric Bayes Further Reading Bayesian Analysis using Data on Completers Overview Model Selection and Inference with a Multivariate Normal Model: Analysis of the Growth Hormone Clinical Study Inference with a Normal Random Effects Model: Analysis of the Schizophrenia Clinical Trial Model Selection and Inference for Binary Longitudinal Data: Analysis of CTQ I Summary Missing Data Mechanisms and Longitudinal Data Introduction Full vs. Observed Data Full-Data Models and Missing Data Mechanisms Assumptions about Missing Data Mechanism Missing at Random Applied to Dropout Processes Observed-Data Posterior of Full-Data Parameters The Ignorability Assumption Examples of Full-Data Models under MAR Full-Data Models under MNAR Summary Further Reading Inference about Full-Data Parameters under Ignorability Overview General Issues in Model Specification Posterior Sampling Using Data Augmentation Covariance Structures for Univariate Longitudinal Processes Covariate-Dependent Covariance Structures Multivariate Processes Model Comparisons and Assessing Model Fit with Incomplete Data under Ignorability Further Reading Case Studies: Ignorable Missingness Overview Analysis of the Growth Hormone Study under MAR Analysis of the Schizophrenia Clinical Trial under MAR Using Random Effects Models Analysis of CTQ I Using Marginalized Transition Models under MAR Analysis of Weekly Smoking Outcomes in CTQ II Using Auxiliary Variable MAR Analysis of HERS CD4 Data under Ignorability Using Bayesian p-Spline Models Summary Models for handling Nonignorable Missingness Overview Extrapolation Factorization Selection Models Mixture Models Shared Parameter Models Model Comparisons and Assessing Model Fit in Nonignorable Models Further Reading Informative Priors and Sensitivity Analysis Overview Some Principles Parameterizing the Full-Data Model Pattern-Mixture Models Selection Models Elicitation of Expert Opinion, Construction of Informative Priors, and Formulation of Sensitivity Analyses A Note on Sensitivity Analysis in Fully Parametric Models Literature on Local Sensitivity Further Reading Case Studies: Model Specification and Data Analysis under Missing Not at Random Overview Analysis of Growth Hormone Study Using Pattern-Mixture Models Analysis of OASIS Study Using Selection and Pattern-Mixture Models Analysis of Pediatric AIDS Trial Using Mixture of Varying Coefficient Models Appendix: distributions Bibliography Index

Journal ArticleDOI
01 Apr 2008-Genetics
TL;DR: It is shown that the statistical specification admits a standard mixed-effects linear model representation, with smoothing parameters treated as variance components, in reproducing kernel Hilbert spaces regression.
Abstract: Reproducing kernel Hilbert spaces regression procedures for prediction of total genetic value for quantitative traits, which make use of phenotypic and genomic data simultaneously, are discussed from a theoretical perspective. It is argued that a nonparametric treatment may be needed for capturing the multiple and complex interactions potentially arising in whole-genome models, i.e., those based on thousands of single-nucleotide polymorphism (SNP) markers. After a review of reproducing kernel Hilbert spaces regression, it is shown that the statistical specification admits a standard mixed-effects linear model representation, with smoothing parameters treated as variance components. Models for capturing different forms of interaction, e.g., chromosome-specific, are presented. Implementations can be carried out using software for likelihood-based or Bayesian inference.

Posted Content
TL;DR: In this paper, the authors compare and evaluate Bayesian predictive distributions from alternative models, using as an illustration five alternative models of asset returns applied to daily S&P 500 returns from 1976 through 2005.
Abstract: Bayesian inference in a time series model provides exact, out-of-sample predictive distributions that fully and coherently incorporate parameter uncertainty. This study compares and evaluates Bayesian predictive distributions from alternative models, using as an illustration five alternative models of asset returns applied to daily S&P 500 returns from 1976 through 2005. The comparison exercise uses predictive likelihoods and is inherently Bayesian. The evaluation exercise uses the probability integral transform and is inherently frequentist. The illustration shows that the two approaches can be complementary, each identifying strengths and weaknesses in models that are not evident using the other.

Journal ArticleDOI
TL;DR: The dynamics of spiking neurons can be interpreted as a form of Bayesian inference in time, and firing statistics are close to Poisson, albeit providing a deterministic representation of probabilities.
Abstract: We show that the dynamics of spiking neurons can be interpreted as a form of Bayesian inference in time. Neurons that optimally integrate evidence about events in the external world exhibit properties similar to leaky integrate-and-fire neurons with spike-dependent adaptation and maximally respond to fluctuations of their input. Spikes signal the occurrence of new information---what cannot be predicted from the past activity. As a result, firing statistics are close to Poisson, albeit providing a deterministic representation of probabilities.

Book
01 Apr 2008
TL;DR: In this article, the authors present a general framework to the claims-reserving problem, based on the Chain-Ladder method (Distribution-Free) and the Bornhuetter-Ferguson method.
Abstract: Preface. Acknowledgement. 1 Introduction and Notation. 1.1 Claims Process. 1.2 Structural Framework to the Claims-Reserving Problem. 1.3 Outstanding Loss Liabilities, Classical Notation. 1.4 General Remarks. 2 Basic Methods. 2.1 Chain-Ladder Method (Distribution-Free). 2.2 Bornhuetter-Ferguson Method. 2.3 Number of IBNyR Claims, Poisson Model. 2.4 Poisson Derivation of the CL Algorithm. 3 Chain-Ladder Models. 3.1 Mean Square Error of Prediction. 3.2 Chain-Ladder Method. 3.3 Bounds in the Unconditional Approach. 3.4 Analysis of Error Terms in the CL Method. 4 Bayesian Models. 4.1 Benktander-Hovinen Method and Cape-Cod Model. 4.2 Credible Claims Reserving Methods. 4.3 Exact Bayesian Models. 4.4 Markov Chain Monte Carlo Methods. 4.5 Buhlmann-Straub Credibility Model. 4.6 Multidimensional Credibility Models. 4.7 Kalman Filter. 5 Distributional Models. 5.1 Log-Normal Model for Cumulative Claims. 5.2 Incremental Claims. 6 Generalized Linear Models. 6.1 Maximum Likelihood Estimators. 6.2 Generalized Linear Models Framework. 6.3 Exponential Dispersion Family. 6.4 Parameter Estimation in the EDF. 6.5 Other GLM Models. 6.6 Bornhuetter-Ferguson Method, Revisited. 7 Bootstrap Methods. 7.1 Introduction. 7.2 Log-Normal Model for Cumulative Sizes. 7.3 Generalized Linear Models. 7.4 Chain-Ladder Method. 7.5 Mathematical Thoughts about Bootstrapping Methods. 7.6 Synchronous Bootstrapping of Seemingly Unrelated Regressions. 8 Multivariate Reserving Methods. 8.1 General Multivariate Framework. 8.2 Multivariate Chain-Ladder Method. 8.3 Multivariate Additive Loss Reserving Method. 8.4 Combined Multivariate CL and ALR Method. 9 Selected Topics I: Chain-Ladder Methods. 9.1 Munich Chain-Ladder. 9.2 CL Reserving: A Bayesian Inference Model. 10 Selected Topics II: Individual Claims Development Processes. 10.1 Modelling Claims Development Processes for Individual Claims. 10.2 Separating IBNeR and IBNyR Claims. 11 Statistical Diagnostics. 11.1 Testing Age-to-Age Factors. 11.2 Non-Parametric Smoothing. Appendix A: Distributions. A.1 Discrete Distributions. A.2 Continuous Distributions. Bibliography. Index.

Proceedings ArticleDOI
01 Jun 2008
TL;DR: This work shows how to obtain a good approximation to Bayesian analysis efficiently, using the Expectation Propagation method, and addresses the problems of optimal design and hyperparameter estimation.
Abstract: The linear model with sparsity-favouring prior on the coefficients has important applications in many different domains. In machine learning, most methods to date search for maximum a posteriori sparse solutions and neglect to represent posterior uncertainties. In this paper, we address problems of Bayesian optimal design (or experiment planning), for which accurate estimates of uncertainty are essential. To this end, we employ expectation propagation approximate inference for the linear model with Laplace prior, giving new insight into numerical stability properties and proposing a robust algorithm. We also show how to estimate model hyperparameters by empirical Bayesian maximisation of the marginal likelihood, and propose ideas in order to scale up the method to very large underdetermined problems. We demonstrate the versatility of our framework on the application of gene regulatory network identification from micro-array expression data, where both the Laplace prior and the active experimental design approach are shown to result in significant improvements. We also address the problem of sparse coding of natural images, and show how our framework can be used for compressive sensing tasks. Part of this work appeared in Seeger et al. (2007b). The gene network identification application appears in Steinke et al. (2007).

Journal ArticleDOI
TL;DR: This paper explores how to make Bayesian inference for the kinetic rate constants of regulatory networks, using the stochastic kinetic Lotka-Volterra system as a model.
Abstract: The ability to infer parameters of gene regulatory networks is emerging as a key problem in systems biology. The biochemical data are intrinsically stochastic and tend to be observed by means of discrete-time sampling systems, which are often limited in their completeness. In this paper we explore how to make Bayesian inference for the kinetic rate constants of regulatory networks, using the stochastic kinetic Lotka-Volterra system as a model. This simple model describes behaviour typical of many biochemical networks which exhibit auto-regulatory behaviour. Various MCMC algorithms are described and their performance evaluated in several data-poor scenarios. An algorithm based on an approximating process is shown to be particularly efficient.

Journal ArticleDOI
TL;DR: The authors proposed a new model of human concept learning based on Bayesian inference for a grammatically structured hypothesis space and compared the model predictions to human generalization judgments in several well-known category learning experiments, and found good agreement for both average and individual participant generalizations.

01 Dec 2008
TL;DR: In this paper, a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior, is proposed, where truncated Karhunen-Loeve expansions are introduced to efficiently parameterize the unknown field and specify a stochastic forward problem whose solution captures that of the deterministic forward model over the support of the prior.
Abstract: We consider a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior. Computational challenges in this construction arise from the need for repeated evaluations of the forward model (e.g., in the context of Markov chain Monte Carlo) and are compounded by high dimensionality of the posterior. We address these challenges by introducing truncated Karhunen-Loeve expansions, based on the prior distribution, to efficiently parameterize the unknown field and to specify a stochastic forward problem whose solution captures that of the deterministic forward model over the support of the prior. We seek a solution of this problem using Galerkin projection on a polynomial chaos basis, and use the solution to construct a reduced-dimensionality surrogate posterior density that is inexpensive to evaluate. We demonstrate the formulation on a transient diffusion equation with prescribed source terms, inferring the spatially-varying diffusivity of the medium from limited and noisy data.

Journal ArticleDOI
TL;DR: Multi-Entity Bayesian Networks is presented, a first-order language for specifying probabilistic knowledge bases as parameterized fragments of Bayesian networks, and a proof is given that MEBN can represent a probability distribution on interpretations of any finitely axiomatizable first- order theory.

Journal ArticleDOI
TL;DR: A variational treatment of dynamic models that furnishes time-dependent conditional densities on the path or trajectory of a system's states and the time-independent densities of its parameters using exactly the same principles is presented.

Journal ArticleDOI
TL;DR: In this paper, the authors consider Bayesian analysis for continuous-time Markov chain models based on a conditional reference prior and show that the use of a proper prior for the rate matrix parameters together with the conditional reference priors for the elapsed time yields a proper posterior distribution.
Abstract: The authors consider Bayesian analysis for continuous-time Markov chain models based on a conditional reference prior. For such models, inference of the elapsed time between chain observations depends heavily on the rate of decay of the prior as the elapsed time increases. Moreover, improper priors on the elapsed time may lead to improper posterior distributions. In addition, an infinitesimal rate matrix also characterizes this class of models. Experts often have good prior knowledge about the parameters of this matrix. The authors show that the use of a proper prior for the rate matrix parameters together with the conditional reference prior for the elapsed time yields a proper posterior distribution. The authors also demonstrate that, when compared to analyses based on priors previously proposed in the literature, a Bayesian analysis on the elapsed time based on the conditional reference prior possesses better frequentist properties. The type of prior thus represents a better default prior choice for estimation software.

Journal ArticleDOI
TL;DR: It is shown here that Bayesian updating and model class selection provide a powerful and rigorous approach to tackle the problem of hysteretic system identification when implemented using a recently developed stochastic simulation algorithm called Transitional Markov Chain Monte Carlo.
Abstract: System identification of structures using their measured earthquake response can play a key role in structural health monitoring, structural control and improving performance-based design. Implementation using data from strong seismic shaking is complicated by the nonlinear hysteretic response of structures. Furthermore, this inverse problem is ill-conditioned for example, even if some components in the structure show substantial yielding, others will exhibit nearly elastic response, producing no information about their yielding behavior. Classical least-squares or maximum likelihood estimation will not work with a realistic class of hysteretic models because it will be unidentifiable based on the data. It is shown here that Bayesian updating and model class selection provide a powerful and rigorous approach to tackle this problem when implemented using a recently developed stochastic simulation algorithm called Transitional Markov Chain Monte Carlo. The updating and model class selection is performed on a previously-developed class of Masing hysteretic structural models that are relatively simple yet can give realistic responses to seismic loading. The theory for the Masing hysteretic models, and the theory used to perform the updating and model class selection, are presented and discussed. An illustrative example is given that uses simulated dynamic response data and shows the ability of the algorithm to identify hysteretic systems even when the class of models is unidentifiable based on the data.

Journal ArticleDOI
TL;DR: A global MCMC scheme that can be applied to a large class of diffusions and whose performance is not adversely affected by the number of latent values is explored and illustrated by estimating parameters governing an auto-regulatory gene network, using partial and discrete data that are subject to measurement error.