Showing papers on "Likelihood principle published in 2011"

Bayesian inference based only on simulated likelihood: particle filter analysis of dynamic economic models

Abstract: Suppose we wish to carry out likelihood based inference but we solely have an unbiased simulation based estimator of the likelihood. We note that unbiasedness is enough when the estimated likelihood is used inside a Metropolis-Hastings algorithm. This result has recently been introduced in statistics literature by Andrieu, Doucet, and Holenstein (2007) and is perhaps surprising given the celebrated results on maximum simulated likelihood estimation. Bayesian inference based on simulated likelihood can be widely applied in microeconomics, macroeconomics and flnancial econometrics. One way of generating unbiased estimates of the likelihood is by the use of a particle fllter. We illustrate these methods on four problems in econometrics, producing rather generic methods. Taken together, these methods imply that if we can simulate from an economic model we can carry out likelihood based inference using its simulations.

On the likelihood function of Gaussian max-stable processes

Abstract: SUMMARY WederiveaclosedformexpressionforthelikelihoodfunctionofaGaussianmax-stableprocessindexed by R d at p d + 1 sites, d 1. We demonstrate the gain in efficiency in the maximum composite likelihood estimators of the covariance matrix from p = 2t op =3 sites in R 2 by means of a Monte Carlo

Likelihood-based scoring rules for comparing density forecasts in tails

Abstract: We propose new scoring rules based on conditional and censored likelihood for assessing the predictive accuracy of competing density forecasts over a specific region of interest, such as the left tail in financial risk management. These scoring rules can be interpreted in terms of Kullback-Leibler divergence between weighted versions of the density forecast and the true density. Existing scoring rules based on weighted likelihood favor density forecasts with more probability mass in the given region, rendering predictive accuracy tests biased towards such densities. Using our novel likelihood-based scoring rules avoids this problem.

The Likelihood Principle

Abstract: Publisher Summary The likelihood principle (LP) is a normative principle for evaluating statistical inference procedures. The LP can be proved from arguably self-evident premises; indeed, it can be proved to be logically equivalent to these premises. This chapter attempts to prove a precise version of the LP, with a number of caveats; and briefly mentions some alternative versions. The importance of the likelihood principle is that it discusses if the comparison is not relevant. LP does rule out many specific inferences. It allows categorizing methods of statistical inference in a very natural and powerful way: a way, which is more abstract and more general than the usual ways of classifying statistical theories. The LP also captures some of the most attractive features of Bayesianism, while leaving open the question of whether a subjective prior should be. Since it provides a lot of common ground between factions of Bayesians, the LP is a good, irenic starting point for agreement between factions of philosophers of statistics.

Maximum likelihood estimation in log-linear models

Abstract: We study maximum likelihood estimation in log-linear models under conditional Poisson sampling schemes. We derive necessary and sufficient conditions for existence of the maximum likelihood estimator (MLE) of the model parameters and investigate estimability of the natural and mean-value parameters under a nonexistent MLE. Our conditions focus on the role of sampling zeros in the observed table. We situate our results within the framework of extended exponential families, and we exploit the geometric properties of log-linear models. We propose algorithms for extended maximum likelihood estimation that improve and correct the existing algorithms for log-linear model analysis.

Comments on pairwise likelihood in time series models

Abstract: This note is concerned with the asymptotic properties of pairwise like- lihood estimation procedures for linear time series models. The latter includes ARMA as well as fractionally integrated ARMA processes, where the fractional integration parameter d 0.25, the pairwise likelihood estimator is not even asymptotically normal. A comparison between using all pairs and consecutive pairs of observations in defining the likelihood is given. We also explore the application of pairwise likelihood to a popular nonlinear model for time series of counts. In this case, the likelihood based on the entire data set cannot be computed without resorting to simulation-based procedures. On the other hand, it is possible to numerically compute the pairwise likelihood precisely. We illustrate the good performance of pairwise likelihood in this case.

Improved likelihood inference in beta regression

Abstract: We consider the issue of performing accurate small-sample likelihood-based inference in beta regression models, which are useful for modelling continuous proportions that are affected by independent variables. We derive small-sample adjustments to the likelihood ratio statistic in this class of models. The adjusted statistics can be easily implemented from standard statistical software. We present Monte Carlo simulations showing that inference based on the adjusted statistics we propose is much more reliable than that based on the usual likelihood ratio statistic. A real data example is presented.

Detection with the scan and the average likelihood ratio

Abstract: We investigate the performance of the scan (maximum likelihood ratio statistic) and of the average likelihood ratio statistic in the problem of detecting a deterministic signal with unknown spatial extent in the prototypical univariate sampled data model with white Gaussian noise. Our results show that the scan statistic, a popular tool for detection problems, is optimal only for the detection of signals with the smallest spatial extent. For signals with larger spatial extent the scan is suboptimal, and the power loss can be considerable. In contrast, the average likelihood ratio statistic is optimal for the detection of signals on all scales except the smallest ones, where its performance is only slightly suboptimal. We give rigorous mathematical statements of these results as well as heuristic explanations which suggest that the essence of these findings applies to detection problems quite generally, such as the detection of clusters in models involving densities or intensities or the detection of multivariate signals. We present a modification of the average likelihood ratio that yields optimal detection of signals with arbitrary spatial extent and which has the additional benefit of allowing for a fast computation of the statistic. In contrast, optimal detection with the scan seems to require the use of scale-dependent critical values.

Copula cosmology: Constructing a likelihood function

Abstract: To estimate cosmological parameters from a given data set, we need to construct a likelihood function, which sometimes has a complicated functional form. We introduce the copula, a mathematical tool to construct an arbitrary multivariate distribution function from one-dimensional marginal distribution functions with any given dependence structure. It is shown that a likelihood function constructed by the so-called Gaussian copula can reproduce very well the $n$-dimensional probability distribution of the cosmic shear power spectrum obtained from a large number of ray-tracing simulations. This suggests that the Copula likelihood will be a powerful tool for future weak lensing analyses, instead of the conventional multivariate Gaussian likelihood.

An Introduction to Maximum Likelihood Estimation and Information Geometry

Abstract: In this paper, we review the maximum likelihood method for estimating the statistical parameters which specify a probabilistic model and show that it generally gives an optimal estimator with minimum mean square error asymptotically. Thus, for most applications in information sciences, the maximum likelihood estimation suffices. Fisher information matrix, which defines the orthogonality between parameters in a probabilistic model, naturally arises from the maximum likelihood estimation. As the inverse of the Fisher information matrix gives the covariance matrix for the estimation errors of the parameters, the orthogonalization of the parameters guarantees that the estimates of the parameters distribute independently from each other. The theory of information geometry provides procedures to diagonalize parameters globally or at all parameter values at least for the exponential and mixture families of distributions. The global orthogonalization gives a simplified and better view for statistical inference and, for example, makes it possible to perform a statistical test for each unknown parameter separately. Therefore, for practical applications, a good start is to examine if the probabilistic model under study belongs to these families.

Statistical Information and Likelihood

Abstract: In part one of this essay the notion of ‘statistical information generated by a data’ is formulated in terms of some intuitively appealing principles of data analysis. The author comes out very strongly in favour of the unrestricted likelihood principle after demonstrating (to his own satisfaction) the reasonableness of the Bayes-Fisher postulate that, within the framework of a particular statistical model, the ‘whole of the relevant information in the data’ must be supposed to be summarised in the likelihood function generated by the data.

The Empirical Likelihood for First-Order Random Coefficient Integer-Valued Autoregressive Processes

Abstract: This article studies the empirical likelihood method for the first-order random coefficient integer-valued autoregressive process. The limiting distribution of the log empirical likelihood ratio statistic is established. Confidence region for the parameter of interest and its coverage probabilities are given, and hypothesis testing is considered. The maximum empirical likelihood estimator for the parameter is derived and its asymptotic properties are established. The performances of the estimator are compared with the conditional least squares estimator via simulation.

A two-sample empirical likelihood ratio test based on samples entropy

Abstract: Powerful entropy-based tests for normality, uniformity and exponentiality have been well addressed in the statistical literature. The density-based empirical likelihood approach improves the performance of these tests for goodness-of-fit, forming them into approximate likelihood ratios. This method is extended to develop two-sample empirical likelihood approximations to optimal parametric likelihood ratios, resulting in an efficient test based on samples entropy. The proposed and examined distribution-free two-sample test is shown to be very competitive with well-known nonparametric tests. For example, the new test has high and stable power detecting a nonconstant shift in the two-sample problem, when Wilcoxon's test may break down completely. This is partly due to the inherent structure developed within Neyman-Pearson type lemmas. The outputs of an extensive Monte Carlo analysis and real data example support our theoretical results. The Monte Carlo simulation study indicates that the proposed test compares favorably with the standard procedures, for a wide range of null and alternative distributions.

Parameter Estimation for Hidden Markov Models with Intractable Likelihoods

Abstract: Approximate Bayesian computation (ABC) is a popular technique for approximating likelihoods and is often used in parameter estimation when the likelihood functions are analytically intractable. Although the use of ABC is widespread in many fields, there has been little investigation of the theoretical properties of the resulting estimators. In this paper we give a theoretical analysis of the asymptotic properties of ABC based maximum likelihood parameter estimation for hidden Markov models. In particular, we derive results analogous to those of consistency and asymptotic normality for standard maximum likelihood estimation. We also discuss how Sequential Monte Carlo methods provide a natural method for implementing likelihood based ABC procedures.

Full likelihood inferences in the Cox model: an empirical likelihood approach

Abstract: For the regression parameter β 0 in the Cox model, there have been several estimators constructed based on various types of approximated likelihood, but none of them has demonstrated small-sample advantage over Cox’s partial likelihood estimator In this article, we derive the full likelihood function for (β 0, F 0), where F 0 is the baseline distribution in the Cox model Using the empirical likelihood parameterization, we explicitly profile out nuisance parameter F 0 to obtain the full-profile likelihood function for β 0 and the maximum likelihood estimator (MLE) for (β 0, F 0) The relation between the MLE and Cox’s partial likelihood estimator for β 0 is made clear by showing that Taylor’s expansion gives Cox’s partial likelihood estimating function as the leading term of the full-profile likelihood estimating function We show that the log full-likelihood ratio has an asymptotic chi-squared distribution, while the simulation studies indicate that for small or moderate sample sizes, the MLE performs favorably over Cox’s partial likelihood estimator In a real dataset example, our full likelihood ratio test and Cox’s partial likelihood ratio test lead to statistically different conclusions

Likelihood and its Evidential Framework

Abstract: Publisher Summary Statistics is the discipline responsible for the interpretation of data as scientific evidence. There is a broad statistical literature dealing with the interpretation of data as statistical evidence, the foundations of statistical inference, and the various statistical paradigms for measuring statistical inference. This chapter offers a general framework that enables the comparison and the evaluation of statistical paradigms claiming to measure the strength of statistical evidence in data. The framework is simple and general, consisting of only three key components. Three essential quantities for assessing and interpreting the strength of statistical evidence in data are: the measure of the strength of evidence, the probability that a particular study design will generate misleading evidence, and the probability that observed evidence is misleading. The advantages of a well-defined evidential framework are illustrated by examining the Likelihood paradigm in the context of multiple examinations of data and multiple comparisons. The Likelihood principle sets forth the conditions under which two experiments yield equivalent statistical evidence for two hypotheses of interest.

Functional Uniform Priors for Nonlinear Modelling

Abstract: This paper considers the topic of finding prior distributions when a major component of the statistical model depends on a nonlinear function. Using results on how to construct uniform distributions in general metric spaces, we propose a prior distribution that is uniform in the space of functional shapes of the underlying nonlinear function and then back-transform to obtain a prior distribution for the original model parameters. The primary application considered in this article is nonlinear regression, but the idea might be of interest beyond this case. For nonlinear regression the so constructed priors have the advantage that they are parametrization invariant and do not violate the likelihood principle, as opposed to uniform distributions on the parameters or the Jeffrey's prior, respectively. The utility of the proposed priors is demonstrated in the context of nonlinear regression modelling in clinical dose-finding trials, through a real data example and simulation. In addition the proposed priors are used for calculation of an optimal Bayesian design.

Conditional Density Estimation by Penalized Likelihood Model Selection and Applications

Abstract: In this technical report, we consider conditional density estimation with a maximum likelihood approach. Under weak assumptions, we obtain a theoretical bound for a Kullback-Leibler type loss for a single model maximum likelihood estimate. We use a penalized model selection technique to select a best model within a collection. We give a general condition on penalty choice that leads to oracle type inequality for the resulting estimate. This construction is applied to two examples of partition-based conditional density models, models in which the conditional density depends only in a piecewise manner from the covariate. The first example relies on classical piecewise polynomial densities while the second uses Gaussian mixtures with varying mixing proportion but same mixture components. We show how this last case is related to an unsupervised segmentation application that has been the source of our motivation to this study.

Asymptotic properties of maximum likelihood estimators in models with multiple change points

Abstract: Models with multiple change points are used in many fields; however, the theoretical properties of maximum likelihood estimators of such models have received relatively little attention. The goal of this paper is to establish the asymptotic properties of maximum likelihood estimators of the parameters of a multiple change-point model for a general class of models in which the form of the distribution can change from segment to segment and in which, possibly, there are parameters that are common to all segments. Consistency of the maximum likelihood estimators of the change points is established and the rate of convergence is determined; the asymptotic distribution of the maximum likelihood estimators of the parameters of the within-segment distributions is also derived. Since the approach used in single change-point models is not easily extended to multiple change-point models, these results require the introduction of those tools for analyzing the likelihood function in a multiple change-point model.

Empirical likelihood method for linear transformation models

Abstract: Empirical likelihood inferential procedure is proposed for right censored survival data under linear transformation models, which include the commonly used proportional hazards model as a special case. A log-empirical likelihood ratio test statistic for the regression coefficients is developed. We show that the proposed log-empirical likelihood ratio test statistic converges to a standard chi-squared distribution. The result can be used to make inference about the entire regression coefficients vector as well as any subset of it. The method is illustrated by extensive simulation studies and a real example.

Unit Roots: Bayesian Significance Test

Abstract: The unit root problem plays a central role in empirical applications in the time series econometric literature. However, significance tests developed under the frequentist tradition present various conceptual problems that jeopardize the power of these tests, especially for small samples. Bayesian alternatives, although having interesting interpretations and being precisely defined, experience problems due to the fact that that the hypothesis of interest in this case is sharp or precise. The Bayesian significance test used in this article, for the unit root hypothesis, is based solely on the posterior density function, without the need of imposing positive probabilities to sets of zero Lebesgue measure. Furthermore, it is conducted under strict observance of the likelihood principle. It was designed mainly for testing sharp null hypotheses and it is called FBST for Full Bayesian Significance Test.

Statistics, probability, significance, likelihood: words mean what we define them to mean

Abstract: • ‘Student’ was a statistician who worked in quality control. • The t-test asks ‘how probable are these samples, if they have been sampled from the same source?’ • Considering what would be found when repeated samples are taken is the frequentist approach to statistical testing. • Student’s t-test does not indicate the probability that the null hypothesis is true. • Other methods of testing experimental results can be more appropriate. • We may need to ask ‘how different?’ or ‘is there no effect?’

Restricted likelihood ratio testing in linear mixed models with general error covariance structure

Abstract: We consider the problem of testing for zero variance components in linear mixed models with correlated or heteroscedastic errors. In the case of independent and identically distributed errors, a valid test exists, which is based on the exact finite sample distribution of the restricted likelihood ratio test statistic under the null hypothesis. We propose to make use of a transformation to derive the (approximate) test distribution for the restricted likelihood ratio test statistic in the case of a general error covariance structure. The proposed test proves its value in simulations and is finally applied to an interesting question in the field of well-being economics.

Composite likelihood-based inferences on genetic data from dependent loci

Abstract: The structure of dependence between neighboring genetic loci is intractable under some models that treat each locus as a single data-point Composite likelihood-based methods present a simple approach under such models by treating the data as if they are independent A maximum composite likelihood estimator (MCLE) is not easy to find numerically, as in most cases we do not have a way of knowing if a maximum is global We study the local maxima of the composite likelihood (ECLE, the efficient composite likelihood estimators), which is straightforward to compute We establish desirable properties of the ECLE and provide an estimator of the variance of MCLE and ECLE We also modify two proper likelihood-based tests to be used with composite likelihood We modify our methods to make them applicable to datasets where some loci are excluded

Statistics, probability, significance, likelihood: words mean what we define them to mean.

Abstract: • ‘Student’ was a statistician who worked in quality control. • The t-test asks ‘how probable are these samples, if they have been sampled from the same source?’ • Considering what would be found when repeated samples are taken is the frequentist approach to statistical testing. • Student’s t-test does not indicate the probability that the null hypothesis is true. • Other methods of testing experimental results can be more appropriate. • We may need to ask ‘how different?’ or ‘is there no effect?’

Empirical Likelihood for an Autoregressive Model with Explanatory Variables

Abstract: In this article, we use the empirical likelihood method to construct the confidence region for parameters in autoregressive model with martingale difference error. It is shown that the empirical log-likelihood ratio at the true parameter converges to the standard chi-square distribution. The simulation results suggest that the empirical likelihood method outperforms the normal approximation based method in terms of coverage probability.