Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1977.tb01600.x

Maximum likelihood from incomplete data via the EM algorithm

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Meta-Analysis in Clinical Trials*

We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.

/pdf/latent-dirichlet-allocation-4gmscb06en.pdf

Latent dirichlet allocation

Maximum Likelihood for General Patterns of Missing Data: Introduction and Theory with Ignorable Nonresponse

Consider a statistical analysis that draws causal inferences from an observational dataset, inferences that are presented as being valid in the frequentist senses; i.e., the analysis produces: 1) c...

/pdf/bridging-observational-studies-and-randomized-experiments-by-3md7ir1w49.pdf

Bridging Observational Studies and Randomized Experiments by Embedding the Former in the Latter

Models for analyzing multivariate data sets with missing values require strong, often unassessable, assumptions. The most common of these is that the mechanism that created the missing data is ignorable - a twofold assumption dependent on the mode of inference. The first part, which is the focus here, under the Bayesian and direct-likelihood paradigms, requires that the missing data are missing at random; in contrast, the frequentist-likelihood paradigm demands that the missing data mechanism always produces missing at random data, a condition known as missing always at random. Under certain regularity conditions, assuming missing always at random leads to an assumption that can be tested using the observed data alone namely, the missing data indicators only depend on fully observed variables. Here, we propose three different diagnostic tests that not only indicate when this assumption is incorrect but also suggest which variables are the most likely culprits. Although missing always at random is not a necessary condition to ensure validity under the Bayesian and direct-likelihood paradigms, it is sufficient, and evidence for its violation should encourage the careful statistician to conduct targeted sensitivity analyses.

Diagnosing missing always at random in multivariate data

Estimation of Imputation Uncertainty

The control of confounding is essential in many statistical problems, especially in those that attempt to estimate exposure effects. In some cases, in addition to the 'primary' sample, there is another 'secondary' sample which, though having no direct information about the exposure effect, contains information about the confounding factors. The purpose of this article is to study the influence of the secondary sample on likelihood inference for the exposure effect. In particular, we investigate the interplay between the efficiency improvement and the possible bias introduced by the secondary sample as a function of the degree of confounding in the primary sample and the sizes of the primary and secondary samples. In the case of weak confounding, the secondary sample can only little improve estimation of the exposure effect, whereas with strong confounding the secondary sample can be much more useful. On the other hand, it will be more important to consider possible biasing effects in the latter case. For illustration, we use a formal example of a generalized linear model and a real example with sparse data from a case-control study of the association between gastric cancer and HM-CAP/Band 120.

Donald B. Rubin

Papers

Maximum Likelihood for General Patterns of Missing Data: Introduction and Theory with Ignorable Nonresponse

Bridging Observational Studies and Randomized Experiments by Embedding the Former in the Latter

Diagnosing missing always at random in multivariate data

Estimation of Imputation Uncertainty

Control of confounding through secondary samples.