Junyu Zhang

TL;DR: A new Variational Policy Gradient Theorem for RL with general utilities is derived, which establishes that the parametrized policy gradient may be obtained as the solution of a stochastic saddle point problem involving the Fenchel dual of the utility function.

TL;DR: In this paper, a variational Monte Carlo gradient estimation algorithm is proposed to compute the policy gradient based on sample paths, and the algorithm converges globally to the optimal policy for the general objective, though the optimization problem is nonconvex.

TL;DR: This work proposes a new definition of risk, which is called caution, as a penalty function added to the dual of the linear programming (LP) formulation of tabular RL, and proposes a block-coordinate augmentation of the aforementioned approach, which improves the reliability of reward accumulation without additional computation as compared to risk-neutral LP solvers.

TL;DR: In this article, a cubic regularized Newton (CRN) method was proposed for solving convex-concave saddle point problems, where at each iteration, a saddle point subproblem is constructed and solved, which provides a search direction for the iterate.

TL;DR: In this paper, a lower bound for the complexity of finding the saddle point of a strongly convex and strongly concave saddle point problem with gradient Lipschitz constants was derived.