Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using costly Monte-Carlo stochastic integration, approximated, or bounded using various techniques. We present a fast and generic method that builds algorithmically closed-form lower and upper bounds on the entropy, the cross-entropy and the Kullback-Leibler divergence of mixtures. We illustrate the versatile method by reporting on our experiments for approximating the Kullback-Leibler divergence between univariate exponential mixtures, Gaussian mixtures, Rayleigh mixtures, and Gamma mixtures.

Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalities.

Information-theoretic measures, such as the entropy, the cross-entropy and the Kullback–Leibler divergence between two mixture models, are core primitives in many signal processing tasks. Since the Kullback–Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using costly Monte Carlo stochastic integration, approximated or bounded using various techniques. We present a fast and generic method that builds algorithmically closed-form lower and upper bounds on the entropy, the cross-entropy, the Kullback–Leibler and the α-divergences of mixtures. We illustrate the versatile method by reporting our experiments for approximating the Kullback–Leibler and the α-divergences between univariate exponential mixtures, Gaussian mixtures, Rayleigh mixtures and Gamma mixtures.

https://www.mdpi.com/1099-4300/18/12/442/pdf

Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities

Elements of the Theory of Functions and Functional Analysis, Volume 1. Me and Normed Spaces.

https://www.preprints.org/manuscript/201610.0086/v1/download

We consider a bilateral birth-death process having sigmoidal-type rates. A thorough discussion on its transient behaviour is given, which includes studying symmetry properties of the transition probabilities, finding conditions leading to their bimodality, determining mean and variance of the process, and analyzing absorption problems in the presence of 1 or 2 boundaries. In particular, thanks to the symmetry properties we obtain the avoiding transition probabilities in the presence of a pair of absorbing boundaries, expressed as a series.

https://www.mdpi.com/2073-8994/1/2/201/pdf

On a Symmetric, Nonlinear Birth-Death Process with Bimodal Transition Probabilities

Common necessary and sufficient conditions of unimodality and bimodality of a two-component Gaussian mixture with equal variances of components are obtained. An exact equation for the boundary of unimodal and bimodal domains is presented. The boundary is proved to be a set of degenerated critical inflection points of the probability density.

Exact equation of the boundary of unimodal and bimodal domains of a two-component Gaussian mixture

Several sufficient conditions are formulated for the uni- and bimodality of a mixture of two Gaussian distributions with equal variances σ2 and different expectation values μi, i = 1, 2. An equation governing all the degenerate critical inflection points for the probability density f(x) of the mixture is derived by a statistical method. This equation describes the boundary of the uni- and bimodality domains of f(x).

http://www.ccas.ru/personal/apraush/05.pdf

On the uni- and bimodality of a two-component Gaussian mixture

Several theorems on sufficient unimodality conditions are formulated for a sum of k normal distributions with the same variance and with different mean values μi, i = 1, ..., k, 2 ≤ k < ∞, taken with their a priori probabilities πi. On the basis of these theorems, estimates for the lower and upper bounds for the mode numbers m are obtained for k ≥ 3 in the case when the mixture contains k* components, 2 ≤ k* < k, satisfying the unimodality conditions.

/pdf/bounds-for-the-number-of-modes-of-the-simplest-gaussian-pi6kd6pcnl.pdf

Bounds for the number of modes of the simplest Gaussian mixture

A two-component Gaussian mixture with equal variances and different mathematical expectations is studied. It is ascertained that the set of degenerate critical points of its probability density is the boundary of the uni- and bimodality. An approximate equation of this boundary is expressed by the distribution parameters. The domain of bifurcations of the investigated function is given in the probabilistic form. An experiment on the extrapolation of the equation of this boundary to the small segment is presented.

Interpolation and extrapolation of the boundary of uni- and bimodality of a two-component Gaussian mixture

For the two-component Gaussian mixture with different variances, several sufficient unimodality and bimodality conditions are obtained and a set of bifurcations is found that includes the equation of the boundary of unimodality and bimodality domains.

S. V. Sorokin

Papers

Exact equation of the boundary of unimodal and bimodal domains of a two-component Gaussian mixture

On the uni- and bimodality of a two-component Gaussian mixture

Bounds for the number of modes of the simplest Gaussian mixture

Interpolation and extrapolation of the boundary of uni- and bimodality of a two-component Gaussian mixture

Unimodality and bimodality of a two-component Gaussian mixture with different variances