Parallel tensor decomposition with distributed memory based on hierarchical singular value decomposition

This paper considers the distributed “nonsmooth+nonsmooth” composite optimization problems for which n agents collaboratively minimize the sum of their local objective functions over the directed networks. In particular, we focus on the scenarios where the sought solutions are desired to possess some structural properties, e.g., sparsity. However, to ensure the convergence, most existing methods produce an ergodic solution via the averaging schemes as the output, which causes the desired structural properties of the output to be destroyed. To address this issue, we develop a new decentralized stochastic proximal gradient method, termed DSPG, in which the nonergodic (last) iteration acts as the output. We also show that the DSPG method achieves the nonergodic convergence rate \(O(\log (T)/\sqrt{T})\) for generally convex objective functions and \(O(\log (T)/T)\) for strongly convex objective functions. When the structure-enhancing regularization is absent and the simple and suffix averaging schemes are used, the convergence rates of DSPG reach \(O(1/\sqrt{T})\) for generally convex objective functions and O(1/T) for strongly convex objective functions, showing improvement relative to the rates \(O(\log (T)/\sqrt{T})\) and \(O(\log (T)/T)\) provided by the existing methods. Simulation examples further illustrate the effectiveness of the proposed method.

On the Non-ergodic Convergence Rate of the Directed Nonsmooth Composite Optimization.

As an important tool of multi-way/tensor data analysis tool, Tucker decomposition has been applied widely in various fields But traditional sequential Tucker algorithms have been outdated because tensor data is growing rapidly in term of size To address this problem, in this paper, we focus on parallel Tucker decomposition of dense tensors on distributed-memory systems The proposed method uses Hierarchical SVD to accelerate the SVD step in traditional sequential algorithms, which usually takes up most computation time The data distribution strategy is designed to follow the implementation of Hierarchical SVD We also find that compared with the state-of-the-art method, the proposed method has lower communication cost in large-scale parallel cases under the assumption of the α–β model

Yong Zhang

Papers

Parallel tensor decomposition with distributed memory based on hierarchical singular value decomposition

On the Non-ergodic Convergence Rate of the Directed Nonsmooth Composite Optimization.

Distributed Dense Tucker Decomposition Based on Hierarchical SVD