The EM algorithm is a numerical technique for the evaluation of maximum likelihood estimates for parameters describing incomplete data models. It is easy to apply in many problems and is stable but slow. The algorithm fails to provide a consistent estimator of the standard errors of the maximum likelihood estimates unless the additional analysis required by the Louis method is performed. Newton-type or other gradient methods are faster and provide error estimates but tend to be unstable and require the analytical evaluation of likelihoods to derive expressions for the score function and (at least) approximations to the Fisher information matrix. The purpose of this paper is to expand on a result by Fisher that permits a unification of EM methodology and Newton methods. The evaluation of the individual observation-by-observation score functions of the incomplete data is a byproduct of the application of the E step of the EM algorithm. Once these become available, the Fisher information matrix may be consistently estimated, and the M step may be replaced by a fast Newton-type step.

A fast improvement to the EM algorithm on its own terms

A fundamental issue in the use of optimization models is the tradeoff between the level of detail and the ease of using and solving the model. Aggregation and disaggregation techniques have proven to be valuable tools for manipulating data and determining the appropriate policies to employ for this tradeoff. Furthermore, aggregation and disaggregation techniques offer promise for solving large-scale optimization models, supply a set of promising methodologies for studying the underlying structure of both univariate and multivariate data sets, and provide a set of tools for manipulating data for different levels of decision makers. In this paper, we develop a general framework for aggregation and disaggregation methodology, survey previous work regarding aggregation and disaggregation techniques for optimization problems, illuminate the appropriate role of aggregation and disaggregation methodology for optimization applications, and propose future research directions.

Aggregation and Disaggregation Techniques and Methodology in Optimization

The fast adaptive composite grid (FAG) method is a systematic process for solving differential boundary value problems. FAC uses global and local uniform grids both to define the composite grid problem and to interact for its fast solution. It can with little added cost substantially improve accuracy of the coarse grid solution and is very suitable for vector and parallel computation. This paper develops both the theoretical and practical aspects of FAC as it applies to elliptic problems, 1. Introduction. The need for local resolution in physical models occurs frequently in practice. Special local features of the forcing function, operator coefficients, boundary, and boundary conditions can demand resolution in restricted regions of the domain that is much finer than the required global resolution. It is important that the discretization and solution processes account for this locally, that is, that the local phenomena do not precipitate a dramatic increase in the overall computation. Unfortunately, this objective of efficiently adapting to local features is often in conflict with the solution process: equation solvers can degrade or even fail to apply in the presence of varying discretization scales; data structures that account for irregular grids can be cumbersome; the computer architecture may not be able to effectively account for grid irregularity (e.g., " vectorizability" may be inhibited); etc. In fact, even the discretization process itself may find difficulty with this objective: for finite differences, it is problematic to develop accurate difference formulae for irregular grids; for finite elements, this objective is reflected in the substantial overhead costs needed to automate the discretization. The fast adaptive composite grid method (FAC (11)) is a discretization and solution method designed to achieve efficient local resolution by constructing the

/pdf/the-fast-adaptive-composite-grid-fac-method-for-elliptic-3crfevfxv3.pdf

The fast adaptive composite grid (FAC) method for elliptic equation

A class of iterative aggregation algorithms for solving infinite horizon dynamic programming problems is proposed. The idea is to interject aggregation iterations in the course of the usual successive approximation method. An important feature that sets this method apart from earlier ones is that the aggregate groups of states change adaptively from one aggregation iteration to the next, depending on the progress of the computation. This allows acceleration of convergence in difficult problems involving multiple-ergodic classes for which methods using fixed groups of aggregate states are ineffective. No knowledge of special problem structure is utilized by the algorithms. >

/pdf/adaptive-aggregation-methods-for-infinite-horizon-dynamic-57pjt7qptv.pdf

Adaptive aggregation methods for infinite horizon dynamic programming

Many large optimization problems represent a family of models of varying size, corresponding to different discretizations. An example is optimal control problems where the solution is a function that is approximated by its values at finitely many points. We discuss optimization techniques suitable for nonlinear programs of this type, with an emphasis on algorithms that guarantee global convergence. The goal is to exploit the similar structure among the subproblems, using the solutions of smaller subproblems to accelerate the solution of larger, more refined subproblems.

A multigrid approach to discretized optimization problems

Acceleration by aggregation of successive approximation methods

http://i.stanford.edu/pub/cstr/reports/cs/tr/75/479/CS-TR-75-479.pdf

Error bounds in the approximation of eigenvalues of differential and integral operators

The theoretical framework of this study is presented in Sect. 1, with a review of practical numerical methods. The linear operatorT and its approximationTn are defined in the same Banach space, which is a very common situation. The notion of strong stability forTn is essential and cannot be weakened without introducing a numerical instability [2]. IfT (or its inverse) is compact, most numerical methods are strongly stable. Without compactness forT(T−1) they may not be strongly stable [20].

A posteriori error bounds for eigenelements of linear operators

The theoretical framework of this study is presented in Sect. 1, with a review of practical numerical methods. The linear operatorT and its approximationT n are defined in the same Banach space, which is a very common situation. The notion of strong stability forT n is essential and cannot be weakened without introducing a numerical instability [2]. IfT (or its inverse) is compact, most numerical methods are strongly stable. Without compactness forT(T ?1) they may not be strongly stable [20].

In Sect. 2 we establish error bounds valid in the general setting of a strongly stable approximation of a closedT. This is a generalization of Vainikko [24, 25] (compact approximation). Osborn [19] (uniform and collectivity compact approximation) and Chatelin and Lemordant [6] (strong approximation), based on the equivalence between the eigenvalues convergence with preservation of multiplicities and the collectively compact convergence of spectral projections. It can be summarized in the following way: ?, eigenvalue ofT of multiplicitym is approximated bym numbers,? n is their arithmetic mean.?-? n and the gap between invariant subspaces are of order? n =?(T-T n)?P. IfT n * converges toT *, pointwise inX *, the principal term in the error on ??-? n ? is $$\frac{1}{m}|tr (T - T_n )P|$$ . And for projection methods, withT n=? n T, we get the bound $$|tr (T - T_n )P| \leqq C ||(1 - \pi _n )P|| ||(1 - \pi _n^* )P*||$$ . It applies to the finite element method for a differential operator with a noncompact resolvent. Aposteriori error bounds are given, and thegeneralized Rayleigh quotient $$\frac{1}{m}tr TP_n $$ TP n appears to be an approximation of ? of the second order, as in the selfadjoint case [12].

In Sect. 3, these results are applied to the Galerkin method and its Sloan variant [22], and to approximate quadrature methods. The error bounds and the generalized Rayleigh quotient are numerically tested in Sect. 4.

Françoise Chatelin

Papers

Acceleration by aggregation of successive approximation methods

Error bounds in the approximation of eigenvalues of differential and integral operators

A posteriori error bounds for eigenelements of linear operators

Sur les bornes d'erreur a posteriori pour les éléments propres d'opérateurs linéaires