scispace - formally typeset
Search or ask a question
Journal Article•DOI•

Computational Efficiency in Continuous (and Discrete!) Time Models – Comment on Hecht and Zitzmann

Charles C. Driver1•
Max Planck Society1
04 Feb 2021-Structural Equation Modeling (Routledge)-Vol. 28, Iss: 5, pp 791-793
TL;DR: It has been claimed that a SEM style continuous-time model can reduce run times for Bayesian estimations of continuous- time models from hours to minutes, but this claim is not true in the general case, but requires that individuals are characterized by the same covariance and means structure, and that the number of time points is not large.
Abstract: Continuous-time models generally imply a stochastic differential equation for latent processes, coupled to a measurement model. Various computational issues can arise, and there are different estim...
References
More filters
Journal Article•DOI•
OpenMx 2.0: Extended Structural Equation and Statistical Modeling

[...]

Michael C. Neale1, Michael D. Hunter2, Joshua N. Pritikin3, Mahsa Zahery1, Timothy R. Brick4, Robert M. Kirkpatrick1, Ryne Estabrook5, Timothy C. Bates6, Hermine H. Maes1, Steven M. Boker3 •
Virginia Commonwealth University1, University of Oklahoma2, University of Virginia3, Pennsylvania State University4, Northwestern University5, University of Edinburgh6
01 Jun 2016-Psychometrika
TL;DR: Ease-of-use improvements include helper functions to standardize model parameters and compute their Jacobian-based standard errors, access to model components through standard R $ mechanisms, and improved tab completion from within the R Graphical User Interface.
Abstract: The new software package OpenMx 2.0 for structural equation and other statistical modeling is introduced and its features are described. OpenMx is evolving in a modular direction and now allows a mix-and-match computational approach that separates model expectations from fit functions and optimizers. Major backend architectural improvements include a move to swappable open-source optimizers such as the newly written CSOLNP. Entire new methodologies such as item factor analysis and state space modeling have been implemented. New model expectation functions including support for the expression of models in LISREL syntax and a simplified multigroup expectation function are available. Ease-of-use improvements include helper functions to standardize model parameters and compute their Jacobian-based standard errors, access to model components through standard R $ mechanisms, and improved tab completion from within the R Graphical User Interface.

756 citations

"Computational Efficiency in Continu..." refers methods in this paper

  • ...Such computational differences are, as mentioned, already leveraged in software such as OpenMx (Neale et al., 2016) to maximize performance....

    [...]

  • ...In the case that individuals all share the same observation timing pattern, then OpenMx detects that there is no need to re-compute the model expectations, and performance can be very fast, as seen in the results from Hecht and Zitzmann....

    [...]

  • ...This approach they favor is actually what is used in the original mixed effects implementation of ctsem (Driver et al., 2017), which used OpenMx to implement CT models using a RAM (McArdle, 2005) formulation of SEM....

    [...]

Journal Article•DOI•
Exponential Integrators for Large Systems of Differential Equations

[...]

Marlis Hochbruck, Christian Lubich, Hubert Selhofer
01 Sep 1998-SIAM Journal on Scientific Computing
TL;DR: This work studies the numerical integration of large stiff systems of differential equations by methods that use matrix--vector products with the exponential or a related function of the Jacobian, and derives methods up to order 4 which are exact for linear constant-coefficient equations.
Abstract: We study the numerical integration of large stiff systems of differential equations by methods that use matrix--vector products with the exponential or a related function of the Jacobian. For large problems, these can be approximated by Krylov subspace methods, which typically converge faster than those for the solution of the linear systems arising in standard stiff integrators. The exponential methods also offer favorable properties in the integration of differential equations whose Jacobian has large imaginary eigenvalues. We derive methods up to order 4 which are exact for linear constant-coefficient equations. The implementation of the methods is discussed. Numerical experiments with reaction-diffusion problems and a time-dependent Schrodinger equation are included.

533 citations

"Computational Efficiency in Continu..." refers methods in this paper

  • ...The primary approach ctsem uses for this is based on matrix exponentiation, as this is near perfect (up to accuracy of the exponential algorithm) for linear systems and can have higher accuracy for moderately non-linear systems (Hochbruck et al., 1998)....

    [...]

Journal Article•DOI•
Adaptive importance sampling in monte carlo integration

[...]

Man Suk Oh1, James O. Berger2•
University of California, Berkeley1, Purdue University2
01 Jul 1992-Journal of Statistical Computation and Simulation
TL;DR: In this paper, an adaptive importance sampling (AIS) scheme is introduced to compute integrals of the form as a mechanical, yet flexible, way of dealing with the selection of parameters of the importance function.
Abstract: An Adaptive Importance Sampling (AIS) scheme is introduced to compute integrals of the form as a mechanical, yet flexible, way of dealing with the selection of parameters of the importance function. AIS starts with a rough estimate for the parameters λ of the importance function g , and runs importance sampling in an iterative way to continually update λ using only linear accumulation. Consistency of AIS is established. The efficiency of the algorithm is studied in three examples and found to be substantially superior to ordinary importance sampling.

180 citations

"Computational Efficiency in Continu..." refers methods in this paper

  • ...For improved uncertainty quantification, adaptive importance sampling (Oh & Berger, 1992) can be requested, which uses the maximum likelihood (or posterior mode when priors are used) and estimated asymptotic covariance matrix as an initial proposal distribution....

    [...]

Journal Article•DOI•
Continuous time structural equation modeling with R package ctsem

[...]

Charles C. Driver, Johan H. L. Oud, Manuel C. Voelkle1•
Max Planck Society1
07 Apr 2017-Journal of Statistical Software
TL;DR: In this article, an R package for continuous time structural equation modeling of panel (N > 1) and time series (N = 1) data, using full information maximum likelihood, is presented.
Abstract: We introduce ctsem, an R package for continuous time structural equation modeling of panel (N > 1) and time series (N = 1) data, using full information maximum likelihood. Most dynamic models (e.g., cross-lagged panel models) in the social and behavioural sciences are discrete time models. An assumption of discrete time models is that time intervals between measurements are equal, and that all subjects were assessed at the same intervals. Violations of this assumption are often ignored due to the difficulty of accounting for varying time intervals, therefore parameter estimates can be biased and the time course of effects becomes ambiguous. By using stochastic differential equations to estimate an underlying continuous process, continuous time models allow for any pattern of measurement occasions. By interfacing to OpenMx, ctsem combines the flexible specification of structural equation models with the enhanced data gathering opportunities and improved estimation of continuous time models. ctsem can estimate relationships over time for multiple latent processes, measured by multiple noisy indicators with varying time intervals between observations. Within and between effects are estimated simultaneously by modeling both observed covariates and unobserved heterogeneity. Exogenous shocks with different shapes, group differences, higher order diffusion effects and oscillating processes can all be simply modeled. We first introduce and define continuous time models, then show how to specify and estimate a range of continuous time models using ctsem.

152 citations

"Computational Efficiency in Continu..." refers methods in this paper

  • ...This approach they favor is actually what is used in the original mixed effects implementation of ctsem (Driver et al., 2017), which used OpenMx to implement CT models using a RAM (McArdle, 2005) formulation of SEM....

    [...]

  • ...…a Bayesian SEM style approach to the estimation of continuous time dynamic systems models, and claim dramatic computational improvements over the hierarchical Bayesian instantiation (Driver & Voelkle, 2018) of the ctsem (Continuous Time Structural Equation Modeling) software (Driver et al., 2017)....

    [...]

Journal Article•DOI•
Computational aspects of continuous---discrete extended Kalman-filtering

[...]

Thomas Mazzoni1•
Rolf C. Hagen Group1
01 Oct 2008-Computational Statistics
TL;DR: The specific structure of the EKF-moment differential equations leads to a hybrid integration algorithm, featuring a new Taylor–Heun-approximation of the nonlinear vector field and a modified Gauss–Legendre-scheme, generating positive semidefinite solutions for the state error covariance.
Abstract: This paper elaborates how the time update of the continuous---discrete extended Kalman-filter (EKF) can be computed in the most efficient way. The specific structure of the EKF-moment differential equations leads to a hybrid integration algorithm, featuring a new Taylor---Heun-approximation of the nonlinear vector field and a modified Gauss---Legendre-scheme, generating positive semidefinite solutions for the state error covariance. Furthermore, the order of consistency and stability behavior of the outlined procedure is investigated. The results are incorporated into an algorithm with adaptive controlled step size, assuring a fixed numerical precision with minimal computational effort.

83 citations

"Computational Efficiency in Continu..." refers methods in this paper

  • ...At present one alternative is available within ctsem, using a Taylor–Heun-approximation of the vector field and a modified Gauss–Legendre-scheme for the covariance (Mazzoni, 2008), but testing of this is very limited and it is unclear under which circumstances `good enough’ answers are obtained....

    [...]

  • ...The default is for maximum likelihood estimation using a form of hybrid (discrete observations, continuous time) extended Kalman filter (Mazzoni, 2008), in which individuals system and measurement parameters may all vary as correlated random effects, fixed effects via covariates, or some…...

    [...]

Related Papers (5)
Some Discrete Approximations to Continuous Time Stochastic Models

[...]

01 Sep 1974-Journal of the royal statistical society series b-methodological
J. D. Sargan
The exact discrete model of a third-order system of linear stochastic differential equations with observable stochastic trends

[...]

01 Nov 2009-Macroeconomic Dynamics
Theodore Simos
Deriving the Exact Discrete Analog of a Continuous Time System

[...]

01 Jan 1997
J R McCrorie
Dynamical Systems and Models of Continuous Time

[...]

21 Mar 2013
Pascal R. Deboeck
Simultaneous modeling of nonlinear deterministic and stochastic dynamics

[...]

15 Dec 1996-Physica D: Nonlinear Phenomena
Lisa Borland