This paper proposes entropy balancing, a data preprocessing method to achieve covariate balance in observational studies with binary treatments. Entropy balancing relies on a maximum entropy reweighting scheme that calibrates unit weights so that the reweighted treatment and control group satisfy a potentially large set of prespecified balance conditions that incorporate information about known sample moments. Entropy balancing thereby exactly adjusts inequalities in representation with respect to the first, second, and possibly higher moments of the covariate distributions. These balance improvements can reduce model dependence for the subsequent estimation of treatment effects. The method assures that balance improves on all covariate moments included in the reweighting. It also obviates the need for continual balance checking and iterative searching over propensity score models that may stochastically balance the covariate moments. We demonstrate the use of entropy balancing with Monte Carlo simulations and empirical applications.

/pdf/entropy-balancing-for-causal-effects-a-multivariate-3wwvsjcqyj.pdf

Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF) This includes NMFs various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD) NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors own recently developed techniques in the subject area Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms Provides a comparative analysis of the different methods in order to identify approximation error and complexity Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia

Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

Improving predictive inference under covariate shift by weighting the log-likelihood function

This introductory book on the most useful parts of large-sample theory is designed to be accessible to scientists outside statistics and certainly to master’s-level statistics students who ignore most of measure theory. According to the author, “the subject of this book, first-order large- sample theory, constitutes a coherent body of concepts and results that are central to both theoretical and applied statistics.” All of the other existing books published on the subject over the last 20 years, from Ibragimov and Has’minskii in 1979 to the most recent by Van der Waart in 1998 have a common prerequisite in mathematical sophistication (measure theory in particular) that do not make the concepts available to a wide audience.

/pdf/elements-of-large-sample-theory-2ij6yyaggv.pdf

Elements of large-sample theory

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMFs various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models. Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors own recently developed techniques in the subject area. Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms. Provides a comparative analysis of the different methods in order to identify approximation error and complexity. Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book. The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.

Nonnegative Matrix and Tensor Factorizations

Multinomial goodness-of-fit tests

1 Introduction to the Power-Divergence Statistic.- 1.1 A Unified Approach to Model Testing.- 1.2 The Power-Divergence Statistic.- 1.3 Outline of the Chapters.- 2 Defining and Testing Models: Concepts and Examples.- 2.1 Modeling Discrete Multivariate Data.- 2.2 Testing the Fit of a Model.- 2.3 An Example: Time Passage and Memory Recall.- 2.4 Applying the Power-Divergence Statistic.- 2.5 Power-Divergence Measures in Visual Perception.- 3 Modeling Cross-Classified Categorical Data.- 3.1 Association Models and Contingency Tables.- 3.2 Two-Dimensional Tables: Independence and Homogeneity.- 3.3 Loglinear Models for Two and Three Dimensions.- 3.4 Parameter Estimation Methods: Minimum Distance Estimation.- 3.5 Model Generation: A Characterization of the Loglinear, Linear, and Other Models through Minimum Distance Estimation.- 3.6 Model Selection and Testing Strategy for Loglinear Models.- 4 Testing the Models: Large-Sample Results.- 4.1 Significance Levels under the Classical (Fixed-Cells) Assumptions.- 4.2 Efficiency under the Classical (Fixed-Cells) Assumptions.- 4.3 Significance Levels and Efficiency under Sparseness Assumptions.- 4.4 A Summary Comparison of the Power-Divergence Family Members.- 4.5 Which Test Statistic?.- 5 Improving the Accuracy of Tests with Small Sample Size.- 5.1 Improved Accuracy through More Accurate Moments.- 5.2 A Second-Order Correction Term Applied Directly to the Asymptotic Distribution.- 5.3 Four Approximations to the Exact Significance Level: How Do They Compare?.- 5.4 Exact Power Comparisons.- 5.5 Which Test Statistic?.- 6 Comparing the Sensitivity of the Test Statistics.- 6.1 Relative Deviations between Observed and Expected Cell Frequencies.- 6.2 Minimum Magnitude of the Power-Divergence Test Statistic.- 6.3 Further Insights into the Accuracy of Large-Sample Approximations.- 6.4 Three Illustrations.- 6.5 Transforming for Closer Asymptotic Approximations in Contingency Tables with Some Small Expected Cell Frequencies.- 6.6 A Geometric Interpretation of the Power-Divergence Statistic.- 6.7 Which Test Statistic?.- 7 Links with Other Test Statistics and Measures of Divergence.- 7.1 Test Statistics Based on Quantiles and Spacings.- 7.2 A Continuous Analogue to the Discrete Test Statistic.- 7.3 Comparisons of Discrete and Continuous Test Statistics.- 7.4 Diversity and Divergence Measures from Information Theory.- 8 Future Directions.- 8.1 Hypothesis Testing and Parameter Estimation under Sparseness Assumptions.- 8.2 The Parameter ? as a Transformation.- 8.3 A Generalization of Akaike's Information Criterion.- 8.4 The Power-Divergence Statistic as a Measure of Loss and a Criterion for General Parameter Estimation.- 8.5 Generalizing the Multinomial Distribution.- Historical Perspective: Pearson's X2 and the Loglikelihood Ratio Statistic G2.- 1. Small-Sample Comparisons of X2 and G2 under the Classical (Fixed-Cells) Assumptions.- 2. Comparing X2 and G2 under Sparseness Assumptions.- 3. Efficiency Comparisons.- 4. Modified Assumptions and Their Impact.- Appendix: Proofs of Important Results.- A1. Some Results on Rao Second-Order Efficiency and Hodges-Lehmann Deficiency (Section 3.4).- A2. Characterization of the Generalized Minimum Power-Divergence Estimate (Section 3.5).- A3. Characterization of the Lancaster-Additive Model (Section 3.5).- A4. Proof of Results (i), (ii), and (iii) (Section 4.1).- A5. Statement of Birch's Regularity Conditions and Proof that the Minimum Power-Divergence Estimator Is BAN (Section 4.1).- A6. Proof of Results (i*), (ii*), and (iii*) (Section 4.1).- A7. The Power-Divergence Generalization of the Chernoff-Lehmann Statistic: An Outline (Section 4.1).- A8. Derivation of the Asymptotic Noncentral Chi-Squared Distribution for the Power-Divergence Statistic under Local Alternative Models (Section 4.2).- A9. Derivation of the Mean and Variance of the Power-Divergence Statistic for ? > -1 under a Nonlocal Alternative Model (Section 4.2).- A10. Proof of the Asymptotic Normality of the Power-Divergence Statistic under Sparseness Assumptions (Section 4.3).- A12. Derivation of the Second-Order Terms for the Distribution Function of the Power-Divergence Statistic under the Classical (Fixed-Cells) Assumptions (Section 5.2).- A13. Derivation of the Minimum Asymptotic Value of the Power-Divergence Statistic (Section 6.2).- A14. Limiting Form of the Power-Divergence Statistic as the Parameter ? ? +- ? (Section 6.2).- Author Index.

Goodness-of-Fit Statistics for Discrete Multivariate Data

Summary The importance of developing useful and appropriate statistical methods for analyzing discrete multivariate data is apparent from the enormous amount of attention this subject has commanded in the literature over the last thirty years. Central to these discussions has been Pearson's X2 statistic and the loglikelihood ratio statistic G2. Our review seeks to consolidate this fragmented literature and develop a unifying theme for much of this research. The traditional X2 and G2 statistics are viewed as members of the power-divergence family of statistics, and are linked through a single real-valued parameter. The principal areas covered in this comparative survey are small-sample comparisons of X2 and G2 under both classical (fixed-cells) assumptions and sparseness assumptions, efficiency comparisons, and various modifications to the test statistics (including parameter estimation for ungrouped data, data-dependent and overlapping cell boundaries, serially dependent data, and smoothing). Finally some future areas for research are discussed.

Timothy R. C. Read

Papers

Multinomial goodness-of-fit tests

Goodness-of-Fit Statistics for Discrete Multivariate Data

Pearsons-X2 and the loglikelihood ratio statistic-G2: a comparative review

Cressie‐Read Statistic