A framework of constraint preserving update schemes for optimization on Stiefel manifold

TLDR: The new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn–Sham total energy minimization.

Abstract: This paper considers optimization problems on the Stiefel manifold $$X^{\mathsf{T}}X=I_p$$XTX=Ip, where $$X\in \mathbb {R}^{n \times p}$$X?Rn×p is the variable and $$I_p$$Ip is the $$p$$p-by-$$p$$p identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of $$X$$X and the null space of $$X^{\mathsf{T}}$$XT. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai---Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn---Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn---Sham total energy minimization.