Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1977.tb01600.x

Maximum likelihood from incomplete data via the EM algorithm

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Meta-Analysis in Clinical Trials*

We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.

/pdf/latent-dirichlet-allocation-4gmscb06en.pdf

Latent dirichlet allocation

Some patterned covariance matrices used to model multivariate normal data that do not have explicit maximum likelihood estimates can be viewed as submatrices of larger patterned covariance matrices that do have explicit maximum likelihood estimates. In such cases, the smaller covariance matrix can be viewed as the covariance matrix for observed variables and the larger covariance matrix can be viewed as the covariance matrix for both observed and missing variables. The advantage of this perspective is that the EM algorithm can be used to calculate the desired maximum likelihood estimates for the original problem. Two examples are presented.

Finding maximum likelihood estimates of patterned covariance matrices by the EM algorithm

Although complete randomization ensures covariate balance on average, the chance for observing significant differences between treatment and control covariate distributions increases with many covariates. Rerandomization discards randomizations that do not satisfy a predetermined covariate balance criterion, generally resulting in better covariate balance and more precise estimates of causal effects. Previous theory has derived finite sample theory for rerandomization under the assumptions of equal treatment group sizes, Gaussian covariate and outcome distributions, or additive causal effects, but not for the general sampling distribution of the difference-in-means estimator for the average causal effect. To supplement existing results, we develop asymptotic theory for rerandomization without these assumptions, which reveals a non-Gaussian asymptotic distribution for this estimator, specifically a linear combination of a Gaussian random variable and a truncated Gaussian random variable. This distribution follows because rerandomization affects only the projection of potential outcomes onto the covariate space but does not affect the corresponding orthogonal residuals. We also demonstrate that, compared to complete randomization, rerandomization reduces the asymptotic sampling variances and quantile ranges of the difference-in-means estimator. Moreover, our work allows the construction of accurate large-sample confidence intervals for the average causal effect, thereby revealing further advantages of rerandomization over complete randomization.

Asymptotic Theory of Rerandomization in Treatment-Control Experiments

We review advances toward credible causal inference that have wide application for empirical legal studies. Our chief point is simple: Research design trumps methods of analysis. We explain matching and regression discontinuity approaches in intuitive (nontechnical) terms. To illustrate, we apply these to existing data on the impact of prison facilities on inmate misconduct, which we compare to experimental evidence. What unifies modern approaches to causal inference is the prioritization of research design to create—without reference to any outcome data—subsets of comparable units. Within those subsets, outcome differences may then be plausibly attributed to exposure to the treatment rather than control condition. Traditional methods of analysis play a small role in this venture. Credible causal inference in law turns on substantive legal, not mathematical, knowledge.

Credible Causal Inference for Empirical Legal Studies

Using new robust matching methods for making causal inferences from survey data, I demonstrate that there are profound differences between how voters behave in advanced democracies versus how they behave in new electoral democracies. The problems of voter ignorance and inattentiveness are not as serious in advanced democracies as many analysts have suggested but are of grave concern in new democracies. Citizens in advanced democracies are able to accomplish something that citizens in fledgling democracies are not: inattentive and poorly informed citizens are able to vote like their better informed compatriots and hence need to pay little attention to political events such as election campaigns in order to vote as if they were attentive. The results from the U.S. (which rely on various National Election Studies) and Mexico (2000 Panel Study) are reported in detail. Results from other countries are briefly reported. “The people should have as little to do as may be about the government. They lack information and are constantly liable to be misled.” —Roger Sherman, June 7, 1787 at the Federal Constitutional Convention (Collier 1971) “In a crowd men always tend to the same level, and, on general questions, a vote, recorded by forty academicians is no better than that of forty water-carriers.” —Gustave Le Bon, The Crowd (1896, 200)

/pdf/the-varying-role-of-voter-information-across-democratic-1jivfsgyhc.pdf

The Varying Role of Voter Information Across Democratic Societies

Randomized experiments suffering from missing data and noncompliance are a recurring problem for experimenters whose subjects are human. Until recently, analysts of such broken randomized experiments were largely forced to squeeze the data into the idealized template of the randomized experiment with neither noncompliance nor missing data. Such practices necessitate throwing away information and making strong, and often unwarranted, assumptions. The Milwaukee Parental Choice Program, a natural experiment, is used to illustrate the flexibility of a new template, which allows for missing data and certain forms of simple noncompliance. The generality of this new template, which is based on a formulation of causal effects called the Rubin causal model, is contrasted with existing alternatives. The multiple imputation technology needed to proceed with analyses from the framework of this template is briefly described, and technical aspects will be presented in depth in subsequent work.

Donald B. Rubin

Papers

Finding maximum likelihood estimates of patterned covariance matrices by the EM algorithm

Asymptotic Theory of Rerandomization in Treatment-Control Experiments

Credible Causal Inference for Empirical Legal Studies

The Varying Role of Voter Information Across Democratic Societies

A Broader Template for Analyzing Broken Randomized Experiments