Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

/pdf/fitting-linear-mixed-effects-models-using-lme4-4zzrpoy9lh.pdf

Fitting Linear Mixed-Effects Models Using lme4

The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.

/pdf/a-tutorial-on-support-vector-machines-for-pattern-5cn6x20psd.pdf

A Tutorial on Support Vector Machines for Pattern Recognition

Many areas of science depend on exploratory data analysis and visualization. The need to analyze large amounts of multivariate data raises the fundamental problem of dimensionality reduction: how to discover compact representations of high-dimensional data. Here, we introduce locally linear embedding (LLE), an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs. Unlike clustering methods for local dimensionality reduction, LLE maps its inputs into a single global coordinate system of lower dimensionality, and its optimizations do not involve local minima. By exploiting the local symmetries of linear reconstructions, LLE is able to learn the global structure of nonlinear manifolds, such as those generated by images of faces or documents of text.

/pdf/nonlinear-dimensionality-reduction-by-locally-linear-vmiazg4oee.pdf

Nonlinear dimensionality reduction by locally linear embedding.

In this paper, we discuss consensus problems for networks of dynamic agents with fixed and switching topologies. We analyze three cases: 1) directed networks with fixed topology; 2) directed networks with switching topology; and 3) undirected networks with communication time-delays and fixed topology. We introduce two consensus protocols for networks with and without time-delays and provide a convergence analysis in all three cases. We establish a direct connection between the algebraic connectivity (or Fiedler eigenvalue) of the network and the performance (or negotiation speed) of a linear consensus protocol. This required the generalization of the notion of algebraic connectivity of undirected graphs to digraphs. It turns out that balanced digraphs play a key role in addressing average-consensus problems. We introduce disagreement functions for convergence analysis of consensus protocols. A disagreement function is a Lyapunov function for the disagreement network dynamics. We proposed a simple disagreement function that is a common Lyapunov function for the disagreement dynamics of a directed network with switching topology. A distinctive feature of this work is to address consensus problems for networks with directed information flow. We provide analytical tools that rely on algebraic graph theory, matrix theory, and control theory. Simulations are provided that demonstrate the effectiveness of our theoretical results.

/pdf/consensus-problems-in-networks-of-agents-with-switching-25c3hqvtd8.pdf

Consensus problems in networks of agents with switching topology and time-delays

Preface 1. Introduction Overview A Brief History of LMIs in Control Theory Notes on the Style of the Book Origin of the Book 2. Some Standard Problems Involving LMIs. Linear Matrix Inequalities Some Standard Problems Ellipsoid Algorithm Interior-Point Methods Strict and Nonstrict LMIs Miscellaneous Results on Matrix Inequalities Some LMI Problems with Analytic Solutions 3. Some Matrix Problems. Minimizing Condition Number by Scaling Minimizing Condition Number of a Positive-Definite Matrix Minimizing Norm by Scaling Rescaling a Matrix Positive-Definite Matrix Completion Problems Quadratic Approximation of a Polytopic Norm Ellipsoidal Approximation 4. Linear Differential Inclusions. Differential Inclusions Some Specific LDIs Nonlinear System Analysis via LDIs 5. Analysis of LDIs: State Properties. Quadratic Stability Invariant Ellipsoids 6. Analysis of LDIs: Input/Output Properties. Input-to-State Properties State-to-Output Properties Input-to-Output Properties 7. State-Feedback Synthesis for LDIs. Static State-Feedback Controllers State Properties Input-to-State Properties State-to-Output Properties Input-to-Output Properties Observer-Based Controllers for Nonlinear Systems 8. Lure and Multiplier Methods. Analysis of Lure Systems Integral Quadratic Constraints Multipliers for Systems with Unknown Parameters 9. Systems with Multiplicative Noise. Analysis of Systems with Multiplicative Noise State-Feedback Synthesis 10. Miscellaneous Problems. Optimization over an Affine Family of Linear Systems Analysis of Systems with LTI Perturbations Positive Orthant Stabilizability Linear Systems with Delays Interpolation Problems The Inverse Problem of Optimal Control System Realization Problems Multi-Criterion LQG Nonconvex Multi-Criterion Quadratic Problems Notation List of Acronyms Bibliography Index.

/pdf/linear-matrix-inequalities-in-system-and-control-theory-2q5cf0pqm2.pdf

Linear Matrix Inequalities in System and Control Theory

We describe an estimator of heteroscedastic variances in the Gauss-Markov linear model where E(e) = 0 and with σ i 2 and unknown. It may be thought of as an approximation to the MINQUE method which results in computational economy, positive estimates, and decreased mean square error. Properties of this almost unbiased estimator are stated. It is compared with other estimators, and extensions to more general models are discussed.

https://www.jstor.org/stable/pdfplus/2285827.pdf

Estimating Heteroscedastic Variances in Linear Models

To address the question of quantification of severity of illness on a wide scale, the Computerized Severity Index (CSI) was developed by a research team at the Johns Hopkins University. This article describes an initial assessment of some aspects of the validity and reliability of the CSI on a sample of 2,378 patients within 27 high-volume DRGs from five teaching hospitals. The 27 DRGs predicted 27% of the variation in LOS, while DRGs adjusted for Admission CSI scores predicted 38% and DRGs adjusted for Maximum CSI scores throughout the hospital stay predicted 54% of this variation. Thus, the Maximum CSI score increased the predictability of DRGs by 100%. We explored the impact of including a 7-day cutoff criterion along with the Maximum CSI score similar to a criterion used in an alternative severity of illness measure. The DRG/Maximum CSI score's predictive power increased to 63% when the 7-day cutoff was added to the CSI definition. The Admission CSI score was used to predict in-hospital mortality and correlated R = 0.603 with mortality. The reliability of Admission and Maximum CSI data collection was high, with agreement of 95% and kappa statistics of 0.88 and 0.90, respectively.

The relationship between severity of illness and hospital length of stay and mortality.

We evaluated the ability of the diagnosis-related-group (DRG) classification system to account adequately for severity of illness and, by implication, for the costs of medical care. Hospital inpatients on medicine, surgery, obstetrics/gynecology, and pediatrics services in six hospitals were evaluated to provide a spectrum of patient and hospital characteristics. This evaluation was based on data from a generic index of severity of illness obtained by trained personnel from a review of hospital charts after patient discharge. Within each DRG, substantial differences were found in the distribution of severity of illness in different hospitals. Some hospitals treated larger proportions of severely ill patients and had a wide range of severity within each DRG, but these differences did not always agree with the teaching classification or the Health Care Financing Administration's case-mix index. These findings suggest that patient classification by means of unadjusted DRGs does not adequately reflect severity of illness, and they indicate that prospective payment programs based on DRGs alone may unfairly and adversely discriminate against certain hospitals.

Interhospital differences in severity of illness. Problems for prospective payment based on diagnosis-related groups (DRGs).

The authors discuss the objectives and definition of the Severity of Illness Index, which has been developed and refined at The Johns Hopkins University over the past 5 years. In addition, the training program for raters, the method used to ascertain reliability, and data from reliability testing in

Roger A. Horn

Papers

Estimating Heteroscedastic Variances in Linear Models

The relationship between severity of illness and hospital length of stay and mortality.

Interhospital differences in severity of illness. Problems for prospective payment based on diagnosis-related groups (DRGs).

Reliability and validity of the Severity of Illness Index.

Canonical forms for complex matrix congruence and ∗congruence