scispace - formally typeset
Search or ask a question
Author

Ehsan S. Soofi

Bio: Ehsan S. Soofi is an academic researcher from University of Wisconsin–Milwaukee. The author has contributed to research in topics: Entropy (information theory) & Principle of maximum entropy. The author has an hindex of 24, co-authored 75 publications receiving 2163 citations.


Papers
More filters
Journal ArticleDOI
TL;DR: The purpose of this article is to develop a general appreciation for the meanings of information functions rather than their mathematical use and to discuss the intricacies of quantifying information in some statistical problems.
Abstract: The purpose of this article is to discuss the intricacies of quantifying information in some statistical problems. The aim is to develop a general appreciation for the meanings of information functions rather than their mathematical use. This theme integrates fundamental aspects of the contributions of Kullback, Lindley, and Jaynes and bridges chaos to probability modeling. A synopsis of information-theoretic statistics is presented in the form of a pyramid with Shannon at the vertex and a triangular base that signifies three distinct variants of quantifying information: discrimination information (Kullback), mutual information (Lindley), and maximum entropy information (Jaynes). Examples of capturing information by the maximum entropy (ME) method are discussed. It is shown that the ME approach produces a general class of logit models capable of capturing various forms of sample and nonsample information. Diagnostics for quantifying information captured by the ME logit models are given, and decom...

176 citations

Journal ArticleDOI
TL;DR: A test of fit for exponentiality based on the estimated Kullback-Leibler information and a procedure for choosing m for various sample sizes is proposed and corresponding critical values are computed by Monte Carlo simulations.
Abstract: In this paper a test of fit for exponentiality based on the estimated Kullback-Leibler information is proposed. The procedure is applicable when the exponential parameter is or is not specified under the null hypothesis. The test uses the Vasicek entropy estimate, so to compute it a window size m must first be fixed. A procedure for choosing m for various sample sizes is proposed and corresponding critical values are computed by Monte Carlo simulations

168 citations

Journal ArticleDOI
TL;DR: Based on the innovation adoption, organizations, and information systems (IS) implementation literature, a set of variables that are related to adoption, use, and benefits of information technology (IT), with particular emphasis on small businesses are identified and explored.
Abstract: Small businesses play an important role in the U.S. economy and there is anecdotal evidence that use of the Web is beneficial to such businesses. There is, however, little systematic analysis of the conditions that lead to successful use of and thereby benefits from the Web for small businesses. Based on the innovation adoption, organizations, and information systems (IS) implementation literature, we identify a set of variables that are related to adoption, use, and benefits of information technology (IT), with particular emphasis on small businesses. These variables are reflective of an organization's contextual characteristics, its IT infrastructure, Web use, and Web benefits. Since the extant research does not suggest a single theoretical model for Web use and benefits in the context of small businesses, we adopt a modeling approach and explore the relationships between “context-IT-use-benefit” (CIUB) through three models—partial-mediator, reduced partial-mediator, and mediator. These models posit that the extent of Web use by small businesses and the associated benefits are driven by organizations’ contextual characteristics and their IT infrastructure. They differ in the endogeneity/exogeneity of the extent of IT sophistication, and in the direct/mediated effects of organizational context. We examine whether the relationships between variables identified in the literature hold within the context of these models using two samples of small businesses with national coverage, including various sizes, and representing several industry sectors. The results show that the evidence for patterns of relationships is similar across the two independent samples for two of these models. We highlight the relationships within the reduced partial-mediator and mediator models for which conclusive evidence are given by both samples. Implications for small business managers and providers of Web-based technologies are discussed.

147 citations

Journal ArticleDOI
TL;DR: In this article, the role of variance and entropy in ordering distributions and random prospects is examined and the results are conveniently tabulated in terms of distribution parameters, which do not disturb the agreement between these rankings.

143 citations

Journal ArticleDOI
TL;DR: The Principal Information Theoretic Approaches (PITA) approach as mentioned in this paper is a principal information theoretic approach to principal information theory that has been proposed for statistical data analysis.
Abstract: (2000). Principal Information Theoretic Approaches. Journal of the American Statistical Association: Vol. 95, No. 452, pp. 1349-1353.

131 citations


Cited by
More filters
Journal ArticleDOI

6,278 citations

Journal ArticleDOI
TL;DR: Two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y), based on entropy estimates from k -nearest neighbor distances are presented.
Abstract: We present two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y). In contrast to conventional estimators based on binnings, they are based on entropy estimates from k -nearest neighbor distances. This means that they are data efficient (with k=1 we resolve structures down to the smallest possible scales), adaptive (the resolution is higher where data are more numerous), and have minimal bias. Indeed, the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale, giving typically systematic errors which scale as functions of k/N for N points. Numerically, we find that both families become exact for independent distributions, i.e. the estimator M(X,Y) vanishes (up to statistical fluctuations) if mu(x,y)=mu(x)mu(y). This holds for all tested marginal distributions and for all dimensions of x and y. In addition, we give estimators for redundancies between more than two random variables. We compare our algorithms in detail with existing algorithms. Finally, we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis (ICA), for improving ICA, and for estimating the reliability of blind source separation.

3,224 citations

Journal ArticleDOI
TL;DR: The information-theoretic (I-T) approaches to valid inference are outlined including a review of some simple methods for making formal inference from all the hypotheses in the model set (multimodel inference).
Abstract: We briefly outline the information-theoretic (I-T) approaches to valid inference including a review of some simple methods for making formal inference from all the hypotheses in the model set (multimodel inference). The I-T approaches can replace the usual t tests and ANOVA tables that are so inferentially limited, but still commonly used. The I-T methods are easy to compute and understand and provide formal measures of the strength of evidence for both the null and alternative hypotheses, given the data. We give an example to highlight the importance of deriving alternative hypotheses and representing these as probability models. Fifteen technical issues are addressed to clarify various points that have appeared incorrectly in the recent literature. We offer several remarks regarding the future of empirical science and data analysis under an I-T framework.

3,105 citations

Journal ArticleDOI
TL;DR: In this paper, the Gibbs sampler is used to indirectly sample from the multinomial posterior distribution on the set of possible subset choices to identify the promising subsets by their more frequent appearance in the Gibbs sample.
Abstract: A crucial problem in building a multiple regression model is the selection of predictors to include. The main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets. This procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices. In this framework the promising subsets of predictors can be identified as those with higher posterior probability. The computational burden is then alleviated by using the Gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices. Those subsets with higher probability—the promising ones—can then be identified by their more frequent appearance in the Gibbs sample.

2,780 citations

Journal ArticleDOI
01 May 1970

1,935 citations