Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

Big Data applications are typically associated with systems involving large numbers of users, massive complex software systems, and large-scale heterogeneous computing and storage architectures. The construction of such systems involves many distributed design choices. The end products (e.g., recommendation systems, medical analysis tools, real-time game engines, speech recognizers) thus involve many tunable configuration parameters. These parameters are often specified and hard-coded into the software by various developers or teams. If optimized jointly, these parameters can result in significant improvements. Bayesian optimization is a powerful tool for the joint optimization of design choices that is gaining great popularity in recent years. It promises greater automation so as to increase both product quality and human productivity. This review paper introduces Bayesian optimization, highlights some of its methodological aspects, and showcases a wide range of applications.

/pdf/taking-the-human-out-of-the-loop-a-review-of-bayesian-4olcznoudw.pdf

Taking the Human Out of the Loop: A Review of Bayesian Optimization

Dose-response analysis can be carried out using multi-purpose commercial statistical software, but except for a few special cases the analysis easily becomes cumbersome as relevant, non-standard output requires manual programming. The extension package drc for the statistical environment R provides a flexible and versatile infrastructure for dose-response analyses in general. The present version of the package, reflecting extensions and modifications over the last decade, provides a user-friendly interface to specify the model assumptions about the dose-response relationship and comes with a number of extractors for summarizing fitted models and carrying out inference on derived parameters. The aim of the present paper is to provide an overview of state-of-the-art dose-response analysis, both in terms of general concepts that have evolved and matured over the years and by means of concrete examples.

/pdf/dose-response-analysis-using-r-1rjy8qez6o.pdf

Dose-Response Analysis Using R

Monetary policy and the private sector behavior of the US economy are modeled as a time varying structural vector autoregression, where the sources of time variation are both the coefficients and the variance covariance matrix of the innovations. The paper develops a new, simple modeling strategy for the law of motion of the variance covariance matrix and proposes an efficient Markov chain Monte Carlo algorithm for the model likelihood/posterior numerical evaluation. The main empirical conclusions are: 1) both systematic and non-systematic monetary policy have changed during the last forty years. In particular, systematic responses of the interest rate to inflation and unemployment exhibit a trend toward a more aggressive behavior, despite remarkable oscillations; 2) this has had a negligible effect on the rest of the economy. The role played by exogenous non-policy shocks seems more important than interest rate policy in explaining the high inflation and unemployment episodes in recent US economic history.

/pdf/time-varying-structural-vector-autoregressions-and-monetary-3ch9uubw9g.pdf

Time Varying Structural Vector Autoregressions and Monetary Policy

Tom A. B. Snijders & Roel J. Bosker. Los Angeles, CA: Sage, 2012, 354 pp. $45.00 (softcover). In the second edition of Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling...

Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling (2nd Edition)

Douglas M. Bates Department of Statistics University of Wisconsin Madison Jose C. Pinheiro Bell Laboratories Lucent Technologies 1 Recent developments in computational methods for maximum likelihood (ML) or restricted maximum likelihood (REML) estimation of parameters in general linear mixed-effects models have made the analysis of data in typical agricultural settings much easier. With software such as SAS PROC MIXED we are able to handle da~ from random-effects one-way classifications, from blocked designs including incomplete blocked designs, from hierarchical designs such as splitplot designs, and other types of data that may be described as repeated measures or longitudinal data or growth-curve data. It is especially helpful that the new computational methods do not depend on balance in the data so we are able to deal more easily with observational studies or with randomly missing data in a designed experiment. We describe some of the new computational approaches and how they are implemented in the nlme3.0 library for the S-PLUS language. One of the most powerful features of this language is the graphics capabilities, especially the trellis graphics facilities developed by Bill Cleveland and his coworkers at Bell Labs. Although most participants in this conference may be more familiar with SAS, and most of the models described here can be fit with PROC MIXED or the NLiNMIX macro or new P ROC N LM IXED in SAS version 7, some exposure to the combination of graphical display and model-fitting approaches from S-PLUS may be informative. 1 Annual Conference on Applied Statistics in Agriculture Kansas State University New Prairie Press http://newprairiepress.org/agstatconference/1998/proceedings/2 2 Kansas State University We show how data exploration with trellis graphics, followed by fitting and comparing mixedeffects models, followed by graphical assessment of the fitted model can be used in a variety of situations. On some occasions, such as modeling growth curves, a linear trend or polynomial trend or other types of linear statistical models for the within-subject time dependence are just not going to do an adequate job of representing the data. In those cases, a nonlinear model is more appropriate. We show how the concept of a random coefficient model can be extended to nonlinear models so as to fit nonlinear mixed-effects models.

/pdf/linear-and-nonlinear-mixed-effects-models-4rdzc6h0s3.pdf

Linear and nonlinear mixed-effects models

Nonlinear mixed-effects models have received a great deal of attention in the statistical literature in recent years because of the flexibility they offer in handling the unbalanced repeated-measures data that arise in different areas of investigation, such as pharmacokinetics and economics. Several different methods for estimating the parameters in nonlinear mixed-effects model have been proposed. We concentrate here on two of them—maximum likelihood and restricted maximum likelihood. A rather complex numerical issue for (restricted) maximum likelihood estimation in nonlinear mixed-effects models is the evaluation of the log-likelihood function of the data, because it involves the evaluation of a multiple integral that, in most cases, does not have a closed-form expression. We consider here four different approximations to the log-likelihood, comparing their computational and statistical properties. We conclude that the linear mixed-effects (LME) approximation suggested by Lindstrom and Bates, t...

/pdf/approximations-to-the-log-likelihood-function-in-the-30ub6xsa3z.pdf

Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model

The estimation of variance-covariance matrices through optimization of an objective function, such as a log-likelihood function, is usually a difficult numerical problem. Since the estimates should be positive semi-definite matrices, we must use constrained optimization, or employ a parametrization that enforces this condition. We describe here five different parametrizations for variance-covariance matrices that ensure positive definiteness, thus leaving the estimation problem unconstrained. We compare the parametrizations based on their computational efficiency and statistical interpretability. The results described here are particularly useful in maximum likelihood and restricted maximum likelihood estimation in linear and non-linear mixed-effects models, but are also applicable to other areas of statistics.

Unconstrained parametrizations for variance-covariance matrices

Linear mixed-effects models are frequently used to analyze repeated measures data, because they model flexibly the within-subject correlation often present in this type of data. The most popular linear mixed-effects model for a continuous response assumes normal distributions for the random effects and the within-subject errors, making it sensitive to outliers. Such outliers are more problematic for mixed-effects models than for fixed-effects models, because they may occur in the random effects, in the within-subject errors, or in both, making them harder to be detected in practice. Motivated by a real dataset from an orthodontic study, we propose a robust hierarchical linear mixed-effects model in which the random effects and the within-subject errors have multivariate t-distributions, with known or unknown degrees-of-freedom, which are allowed to vary with groups of subjects. By using a gamma-normal hierarchical structure, our model allows the identification and classification of both types of outliers,...

José C. Pinheiro

Papers

Linear and nonlinear mixed-effects models

Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model

Unconstrained parametrizations for variance-covariance matrices

Efficient Algorithms for Robust Estimation in Linear Mixed-Effects Models Using the Multivariate t Distribution

Linear and Nonlinear Mixed Effects Models [R package nlme version 3.1-149]