Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1977.tb01600.x

Maximum likelihood from incomplete data via the EM algorithm

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Meta-Analysis in Clinical Trials*

We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.

/pdf/latent-dirichlet-allocation-4gmscb06en.pdf

Latent dirichlet allocation

Multiple-bias modelling for analysis of observational data - Discussion

The estimation of causal effects has been the subject of extensive research. In unconfounded studies with a dichotomous outcome, Y, Cangul, Chretien, Gutman and Rubin (2009) demonstrated that logistic regression for a scalar continuous covariate X is generally statistically invalid for testing null treatment effects when the distributions of X in the treated and control populations differ and the logistic model for Y given X is misspecified. In addition, they showed that an approximately valid statistical test can be generally obtained by discretizing X followed by regression adjustment within each interval defined by the discretized X. This paper extends the work of Cangul et al. 2009 in three major directions. First, we consider additional estimation procedures, including a new one that is based on two independent splines and multiple imputation; second, we consider additional distributional factors; and third, we examine the performance of the procedures when the treatment effect is non-null. Of all the methods considered and in most of the experimental conditions that were examined, our proposed new methodology appears to work best in terms of point and interval estimation. Copyright © 2012 John Wiley & Sons, Ltd.

Robust estimation of causal effects of binary treatments in unconfounded studies with dichotomous outcomes

Much research in the social sciences attempts to estimate the effect of some intervention or “treatment” such as a school dropout prevention program or television watching. However, particularly in the social sciences, it is generally not possible to randomly assign units to receive the treatment condition or the control condition, and thus the resulting data are observational, where we simply observe that some units received the treatment and others did not. In such cases, there is a need to control for differences in the covariate distributions between the treatment and control groups. Matching methods, such as propensity score matching, effect this control by selecting subsets of the treatment and control groups with similar covariate distributions. The overall theme is of replicating a randomized experiment in two ways: first, by comparing treated and control units who look as if they could have been randomly assigned to treatment or control status; and second, by forming the comparison groups without the use of the outcome, thus preventing intentional or unintentional bias in selecting a particular sample to achieve a desired result. This chapter focuses on how to design observational studies using matching methods and the related ideas of subclassification and weighting. We present practical guidance regarding the use of matching methods, as well as examples of their use and evidence of their improved performance relative to other methods of controlling for bias due to observed covariates.

/pdf/matching-methods-for-causal-inference-designing-1259wag3nf.pdf

Matching methods for causal inference: Designing observational studies

This study reports evidence that schizophrenia patients are significantly impaired in both spatial and object (shape) working memory. A 3-s delay between exposure and recall of targets was used and Bayesian item-response theory was applied to compensate for the tasks' differential difficulty while simultaneously taking account of missing data from participant attrition. Weaker evidence was found that in schizophrenia both domains are equally impaired on average, that spatial and object working memory appear to be more highly correlated with each other in the schizophrenia population than in the normal population, and that schizophrenia patients show greater variability in spatial than object working memory performance.

Spatial and object working memory impairments in schizophrenia patients: A Bayesian item-response theory analysis

Factor analysis is a standard tool in educational testing contexts, which can be fit using the EM algorithm (Dempster, Laird and Rubin (1977)) An ex- tension of EM, called the ECME algorithm (Liu and Rubin (1994)), can be used to obtain ML estimates more efficiently in factor analysis models ECME has an E-step, identical to the E-step of EM, but instead of EM's M-step, it has a sequence of CM (conditional maximization) steps, each of which maximizes either the con- strained expected complete-data log-likelihood, as with the ECM algorithm (Meng and Rubin (1993)), or the constrained actual log-likelihood For factor analysis, we use two CM steps: the first maximizes the expected complete-data log-likelihood over the factor loadings given fixed uniquenesses, and the second maximizes the actual likelihood over the uniquenesses given fixed factor loadings We also de- scribe EM and ECME for ML estimation of factor analysis from incomplete data, which arise in applications of factor analysis in educational testing contexts ECME shares with EM its monotone increase in likelihood and stable convergence to an ML estimate, but converges more quickly than EM This more rapid convergence not only can shorten CPU time, but at least as important, it allows for a substan- tially easier assessment of convergence, as shown by examples We believe that the application of ECME to factor analysis illustrates the role that extended EM-type algorithms, such as the even more general AECM algorithm (Meng and van Dyk (1997)) and the PX-EM algorithm (Liu, Rubin and Wu (1997)), can play in fitting complex models that can arise in educational testing contexts

/pdf/maximum-likelihood-estimation-of-factor-analysis-using-the-4mm28dq6ai.pdf

Donald B. Rubin

Papers

Multiple-bias modelling for analysis of observational data - Discussion

Robust estimation of causal effects of binary treatments in unconfounded studies with dichotomous outcomes

Matching methods for causal inference: Designing observational studies

Spatial and object working memory impairments in schizophrenia patients: A Bayesian item-response theory analysis

Maximum likelihood estimation of factor analysis using the ecme algorithm with complete and incomplete data