Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

Two convergence aspects of the EM algorithm are studied: (i) does the EM algorithm find a local maximum or a stationary value of the (incomplete-data) likelihood function? (ii) does the sequence of parameter estimates generated by EM converge? Several convergence results are obtained under conditions that are applicable to many practical situations Two useful special cases are: (a) if the unobserved complete-data specification can be described by a curved exponential family with compact parameter space, all the limit points of any EM sequence are stationary points of the likelihood function; (b) if the likelihood function is unimodal and a certain differentiability condition is satisfied, then any EM sequence converges to the unique maximum likelihood estimate A list of key properties of the algorithm is included

/pdf/on-the-convergence-properties-of-the-em-algorithm-zxjq0mqgzw.pdf

On the convergence properties of the em algorithm

The problem of estimating the parameters which determine a mixture density has been the subject of a large, diverse body of literature spanning nearly ninety years. During the last two decades, the...

Mixture densities, maximum likelihood, and the EM algorithm

We present an integral equation which generalizes a variety of known rendering algorithms. In the course of discussing a monte carlo solution we also present a new form of variance reduction, called Hierarchical sampling and give a number of elaborations shows that it may be an efficient new technique for a wide variety of monte carlo procedures. The resulting rendering algorithm extends the range of optical phenomena which can be effectively simulated.

/pdf/the-rendering-equation-123av1m206.pdf

The rendering equation

Surrogate-based Analysis and Optimization

Let $\{(X_i,Y_i)\}$ be a stationary ergodic time series with $(X,Y)$ values in the product space $\R^d\bigotimes \R .$ This study offers what is believed to be the first strongly consistent (with respect to pointwise, least-squares, and uniform distance) algorithm for inferring $m(x)=E[Y_0|X_0=x]$ under the presumption that $m(x)$ is uniformly Lipschitz continuous. Auto-regression, or forecasting, is an important special case, and as such our work extends the literature of nonparametric, nonlinear forecasting by circumventing customary mixing assumptions. The work is motivated by a time series model in stochastic finance and by perspectives of its contribution to the issues of universal time series estimation.

Strongly consistent nonparametric forecasting and regression for stationary ergodic sequences

The setting is a stationary, ergodic time series. The challenge is to construct a sequence of functions, each based on only finite segments of the past, which together provide a strongly consistent estimator for the conditional probability of the next observation, given the infinite past. Ornstein gave such a construction for the case that the values are from a finite set, and recently Algoet extended the scheme to time series with coordinates in a Polish space. 
The present study relates a different solution to the challenge. The algorithm is simple and its verification is fairly transparent. Some extensions to regression, pattern recognition, and on-line forecasting are mentioned.

Nonparametric inference for ergodic, stationary time series

It is widely recognized that Bayes decision analysis with gamma variables leads almost invariably to fairly troublesome computational difficulties. The purpose of this paper is to show how these difficulties can be circumvented by the use of certain results of decision theory in conjunction with some specialized developments on series expansions. Specifically, the posterior law will be approximated by the computationally and analytically convenient bivariate normal distribution. The approximation technique derived here is compared to exact formulas through statistical measures as well as comparative performance in water resource management contexts.

Large‐sample methods for decision analysis of gamma variates

A fast technique for efficient online signal detection and tracking is extended by analyzing the asymptotic distribution of the detector. A version of the technique is seen to have potentially greater precision than other fast estimators, the upper bound being O(1/n/sup 2/3/). Coupled with another device, there is potential for achieving the convergence precision of the maximum likelihood estimate, which, by conventional methods, entails considerably computational expense. >

Asymptotic theory for a fast frequency detector

Let A be a square matrix of order n. Consider the equation 

$$Ax = \,\lambda x,$$

(7.1)

where λ is a real or complex number and x is an n-tuple. For x = 0, equation (7.1) is satisfied by an arbitrary scalar λ. The values of λ for which there exists some nonzero vector x satisfying (7.1) are called eigenvalues and the corresponding vectors x are called eigenvectors. Equation (7.1) can be written as 

$$(\left. {A\, - \,\lambda } \right|)x\, = 0,$$

(7.2)

which for fixed λ is a homogeneous system of linear equations. This observation implies that if somehow we can obtain the eigenvalue λ of a given matrix A, we can find a corresponding eigenvector by solving (7.2) by methods from Chapter 6. Equation (7.2) has a nontrivial solution if and only if 

$$\det (\left. {A - \,\lambda } \right|)\, = \,\left| {\begin{array}{*{20}{c}} {a{}_{11} - \lambda }&{a{}_{21}}& \cdots &{a{}_{1n}} \\ {a{}_{21}}&{a{}_{22} - \lambda }& \cdots &{a{}_{2n}} \\ \vdots &{}&{}& \vdots \\ {a{}_{n1}}&{a{}_{n2}}& \cdots &{a{}_{nn} - \lambda } \end{array}} \right| = 0.$$

Sidney Yakowitz

Papers

Strongly consistent nonparametric forecasting and regression for stationary ergodic sequences

Nonparametric inference for ergodic, stationary time series

Large‐sample methods for decision analysis of gamma variates

Asymptotic theory for a fast frequency detector

The Solution of Matrix Eigenvalue Problems