fn is shown to have smaller mean integrated squared error compared with kernel-based methods, even when we allow the use of a theoret- ical, optimal fixed bandwidth for the kernel estimator that would not be available in practice. We also present a real data clustering example, which shows that our methodology can be used in conjunction with the expectation-maximization algorithm to fit finite mixtures of log-concave densities.

/pdf/maximum-likelihood-estimation-of-a-multi-dimensional-log-4r9wx98jdy.pdf

Maximum likelihood estimation of a multi‐dimensional log‐concave density

We study nonparametric maximum likelihood estimation of a log-concave probability density and its distribution and hazard function. Some general properties of these estimators are derived from two characterizations. It is shown that the rate of convergence with respect to supremum norm on a compact interval for the density and hazard rate estimator is at least $(\log(n)/n)^{1/3}$ and typically $(\log(n)/n)^{2/5}$, whereas the difference between the empirical and estimated distribution function vanishes with rate $o_{\mathrm{p}}(n^{-1/2})$ under certain regularity assumptions.

Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency

We present theoretical properties of the log-concave maximum likelihood estimator of a density based on an independent and identically distributed sample in R d . Our study covers both the case where the true underlying density is log-concave, and where this model is misspecified. We begin by showing that for a sequence of log-concave densities, con- vergence in distribution implies much stronger types of convergence - in particular, it implies convergence in Hellinger distance and even in cer- tain exponentially weighted total variation norms. In our main result, we prove the existence and uniqueness of a log-concave density that minimises the Kullback-Leibler divergence from the true density over the class of all log-concave densities, and also show that the log-concave maximum like- lihood estimator converges almost surely in these exponentially weighted total variation norms to this minimiser. In the case of a correctly specified model, this demonstrates a strong type of consistency for the estimator; in a misspecified model, it shows that the estimator converges to the log- concave density that is closest in the Kullback-Leibler sense to the true density.

/pdf/theoretical-properties-of-the-log-concave-maximum-likelihood-51bgiwio1m.pdf

Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density

The estimation of a log-concave density on $\mathbb{R}^{d}$ represents a central problem in the area of nonparametric inference under shape constraints. In this paper, we study the performance of log-concave density estimators with respect to global loss functions, and adopt a minimax approach. We first show that no statistical procedure based on a sample of size $n$ can estimate a log-concave density with respect to the squared Hellinger loss function with supremum risk smaller than order $n^{-4/5}$, when $d=1$, and order $n^{-2/(d+1)}$ when $d\geq2$. In particular, this reveals a sense in which, when $d\geq3$, log-concave density estimation is fundamentally more challenging than the estimation of a density with two bounded derivatives (a problem to which it has been compared). Second, we show that for $d\leq3$, the Hellinger $\varepsilon$-bracketing entropy of a class of log-concave densities with small mean and covariance matrix close to the identity grows like $\max\{\varepsilon^{-d/2},\varepsilon^{-(d-1)}\}$ (up to a logarithmic factor when $d=2$). This enables us to prove that when $d\leq3$ the log-concave maximum likelihood estimator achieves the minimax optimal rate (up to logarithmic factors when $d=2,3$) with respect to squared Hellinger loss.

/pdf/global-rates-of-convergence-in-log-concave-density-3ts7e7e536.pdf

Global rates of convergence in log-concave density estimation

Log-concave distributions are an attractive choice for modeling and inference, for several reasons: The class of log-concave distributions contains most of the commonly used parametric distributions and thus is a rich and flexible nonparametric class of distributions. Further, the MLE exists and can be computed with readily available algorithms. Thus, no tuning parameter, such as a bandwidth, is necessary for estimation. Due to these attractive properties, there has been considerable recent research activity concerning the theory and applications of log-concave distributions. This article gives a review of these results.

/pdf/inference-and-modeling-with-log-concave-distributions-4lgmc56ufo.pdf

Inference and Modeling with Log-concave Distributions

We develop an active set algorithm for the maximum likelihood estimation of a log-concave density based on complete data. Building on this fast algorithm, we indidate an EM algorithm to treat arbitrarily censored or binned data.

Active Set and EM Algorithms for Log-Concave Densities Based on Complete and Censored Data

In this paper we show that the family P_d of probability distributions on R^d with log-concave densities satisfies a strong continuity condition. In particular, it turns out that weak convergence within this family entails (i) convergence in total variation distance, (ii) convergence of arbitrary moments, and (iii) pointwise convergence of Laplace transforms. Hence the nonparametric model P_d has similar properties as parametric models such as, for instance, the family of all d-variate Gaussian distributions.

/pdf/multivariate-log-concave-distributions-as-a-nearly-21gu2kko0e.pdf

Andre Huesler

Papers

Active Set and EM Algorithms for Log-Concave Densities Based on Complete and Censored Data

Multivariate Log-Concave Distributions as a Nearly Parametric Model