Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1977.tb01600.x

Maximum likelihood from incomplete data via the EM algorithm

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Meta-Analysis in Clinical Trials*

We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.

/pdf/latent-dirichlet-allocation-4gmscb06en.pdf

Latent dirichlet allocation

The general approach to nonresponse (missingness) in sur veys that I will take here will be to impute values for missing data (really, several values for each missing datum). The ap proach that imputes one value for each missing datum is stan dard in practice, although often criticized by more mathematical statisticians who prefer to think about estimating parameters un der some model. I am very sympathetic with the imputation position. There do not exist parameters except under hypothetical models; there do, however, exist actual observed values and values that would have been observed. Focusing on the estimation of parameters is often not what we want to do since a hypothetical model is simply a structure that guides us to do sensible things with observed values. Of course (1) imputing one value for missing datum can't be correct in general, and (2) in order to insert sensible values for a missing datum we must rely more or less on some model relating unobserved values to observed values. Hence, I see the best approach to be one where we can (1) insert more than one value for a missing datum, and (2) the inserted values reflect a variety of models for the dataset. This position focusing on values to impute rather than param eters to be estimated is actually very Bayesian, and the Bayesian perspective guides us in our design of a general system for nonre sponse. What we really want to impute is the "predictive distribu tion" of the missing values given then observed values (having integrated?averaged?over all model parameters). The theo retical Bayesian position tells us that (1) the missing data has a distribution given the observed data (the predictive distribution) and (2) this distribution depends on assumptions that have been made about the model. Notice that the (l)'s and (2)'s in the above paragraphs are meant to refer to the same two points. The related practical questions are (1) how do we represent in a dataset a distribution of values to impute for each missing datum? And (2) what models should we use to tie observed and unobserved values to each other in order to produce the predictive distribution needed in (1). Section 2 addresses the first question and Section 3 addresses the second question.

https://www.jstor.org/stable/pdfplus/27643587.pdf

The Design of a General and Flexible System for Handling Nonresponse in Sample Surveys

: We consider three simple approaches to rounding error in least squares regression. The first treats the rounded data as if they were unrounded, the second adds an adjustment to the diagonal of the covariance matrix of the variables, and the third subtracts an adjustment from the diagonal. The third, Sheppard's corrections, can be motivated as maximum likelihood with small rounding error and either (1) joint normal data or (2) normal residuals, regular independent variables, and large samples. Although an example and theory suggest that the third approach is usually preferable to the first two, a generally satisfactory attack on rounding error in regression requires the specification of the full distribution of variables, and convenient computational methods for this problem are not currently available. (Author)

Rounding Error in Regression: The Appropriateness of Sheppard's Corrections,

Reaction times for schizophrenic individuals in a simple visual tracking experiment can be substantially more variable than for non-schizophrenic individuals Current psychological theory suggests that at least some of this extra variability arises from an attentional lapse that delays some, but not all, of each schizophrenic's reaction times Based on this theory, we pursue models in which measurements from non-schizophrenics arise from a normal linear model with a separate mean for each individual, whereas measurements from schizophrenics arise from a mixture of (i) a component analogous to the distribution of response times for non-schizophrenics and (ii) a mean-shifted component We fit four mixture models within this framework, where the distinctions between models arise from assumptions about the variance of the shifted observations and the exchangeability of schizophrenic individuals Some of these models can be fit by maximum likelihood using the EM algorithm, and all can be fit using the ECM algorithm, where the covariance matrices associated with the parameters are calculated by the SEM and SECM algorithms, respectively Bayesian model monitoring using posterior predictive checks is invoked to discard models that fail to reproduce certain observed features of the data and to stimulate the development of better models

The analysis of repeated‐measures data on schizophrenic reaction times using mixture models

In the process of collecting Post-Enumeration Survey (PES) data to evaluate census coverage, it is inevitable that there will be some individuals whose enumeration status (outcome in the census-PES match) remains unresolved even after extensive field follow-up operations. Earlier work developed a logistic regression framework for imputing the probability that unresolved individuals were enumerated in the census, so that the probability of having been enumerated is allowed to depend on covariates. The covariates may include demographic characteristics, geographic information, and census codes that summarize information on the characteristics of the match (e.g., the before-follow-up match code assigned by clerks to describe the type of match between PES and census records). In the production of 1990 undercount estimates, the basic logistic regression model was expanded into a mixed hierarchical model to allow for the presence of group-specific effects, where groups are characterized by common befor...

Hierarchical logistic regression models for imputation of unresolved enumeration status in undercount estimation.

Although recent guidelines for dealing with missing data emphasize the need for sensitivity analyses, and such analyses have a long history in statistics, universal recommendations for conducting and displaying these analyses are scarce. We propose graphical displays that help formalize and visualize the results of sensitivity analyses, building upon the idea of 'tipping-point' analysis for randomized experiments with a binary outcome and a dichotomous treatment. The resulting 'enhanced tipping-point displays' are convenient summaries of conclusions obtained from making different modeling assumptions about missingness mechanisms. The primary goal of the displays is to make formal sensitivity analysesmore comprehensible to practitioners, thereby helping them assess the robustness of the experiment's conclusions to plausible missingness mechanisms. We also present a recent example of these enhanced displays in amedical device clinical trial that helped lead to FDA approval.

/pdf/sensitivity-analysis-for-a-partially-missing-binary-outcome-9seqmynha4.pdf

Donald B. Rubin

Papers

The Design of a General and Flexible System for Handling Nonresponse in Sample Surveys

Rounding Error in Regression: The Appropriateness of Sheppard's Corrections,

The analysis of repeated‐measures data on schizophrenic reaction times using mixture models

Hierarchical logistic regression models for imputation of unresolved enumeration status in undercount estimation.

Sensitivity analysis for a partially missing binary outcome in a two-arm randomized clinical trial