Jonas Degrave

TL;DR: In this paper , a novel architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils is presented. But this approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations.

TL;DR: Scheduled auxiliary control (SAC-X) as discussed by the authors enables learning of complex behaviors from scratch in the presence of multiple sparse reward signals, where the agent is equipped with a set of general auxiliary tasks, that it attempts to learn simultaneously via off-policy RL.

TL;DR: The key idea behind the method is that active (learned) scheduling and execution of auxiliary policies allows the agent to efficiently explore its environment - enabling it to excel at sparse reward RL.

TL;DR: In this article, the authors propose a physics engine that can differentiate control parameters, which is implemented for both CPU and GPU and shows how such an engine can speed up the optimization process, even for small problems.