Joint Learning of Topology and Invertible Nonlinearities from Multiple Time Series
22 Dec 2022-pp 483-488
TL;DR: In this paper , a nonlinear modeling technique for multiple time series that has a complexity similar to that of linear vector autoregressive (VAR), but it can account for nonlinear interactions for each sensor variable is proposed.
Abstract: Discovery of causal dependencies among time series has been tackled in the past either by using linear models, or using kernel- or deep learning-based nonlinear models, the latter ones entailing great complexity. This paper proposes a nonlinear modelling technique for multiple time series that has a complexity similar to that of linear vector autoregressive (VAR), but it can account for nonlinear interactions for each sensor variable. The modelling assumption is that the time series are generated in two steps: i) a VAR process in a latent space, and ii) a set of invertible nonlinear mappings applied component-wise, mapping each sensor variable into a latent space. Successful identification of the support of the VAR coefficients reveals the topology of the interconnected system. The proposed method enforces sparsity on the VAR coefficients and models the component-wise nonlinearities using invertible neural networks. To solve the estimation problem, a technique combining proximal gradient descent (PGD) and projected gradient descent is designed. Experiments conducted on real and synthetic data sets show that the proposed algorithm provides an improved identification of the support of the VAR coefficients, while improving also the prediction capabilities.
23 Dec 2011
TL;DR: This monograph covers proximal methods, block-coordinate descent, reweighted l2-penalized techniques, working-set and homotopy methods, as well as non-convex formulations and extensions, and provides an extensive set of experiments to compare various algorithms from a computational point of view.
Abstract: Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. They were first dedicated to linear variable selection but numerous extensions have now emerged such as structured sparsity or kernel selection. It turns out that many of the related estimation problems can be cast as convex optimization problems by regularizing the empirical risk with appropriate nonsmooth norms. The goal of this monograph is to present from a general perspective optimization tools and techniques dedicated to such sparsity-inducing penalties. We cover proximal methods, block-coordinate descent, reweighted l2-penalized techniques, working-set and homotopy methods, as well as non-convex formulations and extensions, and provide an extensive set of experiments to compare various algorithms from a computational point of view.
TL;DR: A nonlinear extension of DCM that models such processes (to second order) at the neuronal population level is presented and it is found that attention-induced increases in V5 responses could be best explained as a gating of the V1-->V5 connection by activity in posterior parietal cortex.
TL;DR: In this paper, the authors survey solutions to the problem of graph learning, including classical viewpoints from statistics and physics, and more recent approaches that adopt a graph signal processing (GSP) perspective.
Abstract: The construction of a meaningful graph topology plays a crucial role in the effective representation, processing, analysis and visualization of structured data. When a natural choice of the graph is not readily available from the data sets, it is thus desirable to infer or learn a graph topology from the data. In this tutorial overview, we survey solutions to the problem of graph learning, including classical viewpoints from statistics and physics, and more recent approaches that adopt a graph signal processing (GSP) perspective. We further emphasize the conceptual similarities and differences between classical and GSP-based graph inference methods, and highlight the potential advantage of the latter in a number of theoretical and practical scenarios. We conclude with several open issues and challenges that are keys to the design of future signal processing and machine learning algorithms for learning graphs from data.
••25 Apr 2018
TL;DR: The main goal of this paper is to outline overarching advances, and develop a principled framework to capture nonlinearities through kernels, which are judiciously chosen from a preselected dictionary to optimally fit the data.
Abstract: Identifying graph topologies as well as processes evolving over graphs emerge in various applications involving gene-regulatory, brain, power, and social networks, to name a few. Key graph-aware learning tasks include regression, classification, subspace clustering, anomaly identification, interpolation, extrapolation, and dimensionality reduction. Scalable approaches to deal with such high-dimensional tasks experience a paradigm shift to address the unique modeling and computational challenges associated with data-driven sciences. Albeit simple and tractable, linear time-invariant models are limited since they are incapable of handling generally evolving topologies, as well as nonlinear and dynamic dependencies between nodal processes. To this end, the main goal of this paper is to outline overarching advances, and develop a principled framework to capture nonlinearities through kernels, which are judiciously chosen from a preselected dictionary to optimally fit the data. The framework encompasses and leverages (non) linear counterparts of partial correlation and partial Granger causality, as well as (non)linear structural equations and vector autoregressions, along with attributes such as low rank, sparsity, and smoothness to capture even directional dependencies with abrupt change points, as well as time-evolving processes over possibly time-evolving topologies. The overarching approach inherits the versatility and generality of kernel-based methods, and lends itself to batch and computationally affordable online learning algorithms, which include novel Kalman filters over graphs. Real data experiments highlight the impact of the nonlinear and dynamic models on consumer and financial networks, as well as gene-regulatory and functional connectivity brain networks, where connectivity patterns revealed exhibit discernible differences relative to existing approaches.
TL;DR: A recently proposed flexible approach has been recently proposed, consisting in the kernel version of Granger causality, to capture nonlinear interactions between even short and noisy time series.