Single crystals for radiation detectors

Deep reinforcement learning has led to many recent-and groundbreaking-advancements. However, these advances have often come at the cost of both the scale and complexity of the underlying RL algorithms. Increases in complexity have in turn made it more difficult for researchers to reproduce published RL algorithms or rapidly prototype ideas. To address this, we introduce Acme, a tool to simplify the development of novel RL algorithms that is specifically designed to enable simple agent implementations that can be run at various scales of execution. Our aim is also to make the results of various RL algorithms developed in academia and industrial labs easier to reproduce and extend. To this end we are releasing baseline implementations of various algorithms, created using our framework. In this work we introduce the major design decisions behind Acme and show how these are used to construct these baselines. We also experiment with these agents at different scales of both complexity and computation-including distributed versions. Ultimately, we show that the design decisions behind Acme lead to agents that can be scaled both up and down and that, for the most part, greater levels of parallelization result in agents with equivalent performance, just faster.

Acme: A Research Framework for Distributed Reinforcement Learning

The combination of Reinforcement Learning (RL) with deep learning has led to a series of impressive feats, with many believing (deep) RL provides a path towards generally capable agents. However, the success of RL agents is often highly sensitive to design choices in the training process, which may require tedious and error-prone manual tuning. This makes it challenging to use RL for new problems and also limits its full potential. In many other areas of machine learning, AutoML has shown that it is possible to automate such design choices, and AutoML has also yielded promising initial results when applied to RL. However, Automated Reinforcement Learning (AutoRL) involves not only standard applications of AutoML but also includes additional challenges unique to RL, that naturally produce a different set of methods. As such, AutoRL has been emerging as an important area of research in RL, providing promise in a variety of applications from RNA design to playing games, such as Go. Given the diversity of methods and environments considered in RL, much of the research has been conducted in distinct subfields, ranging from meta-learning to evolution. In this survey, we seek to unify the field of AutoRL, provide a common taxonomy, discuss each area in detail and pose open problems of interest to researchers going forward.

/pdf/automated-reinforcement-learning-autorl-a-survey-and-open-2t39z0z5.pdf

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

The authors review the techniques used to measure the energy of elementary particles. The methods discussed span more than 20 orders of magnitude in energy, ranging from a small fraction of an electronvolt, where relic remainders of the Big Bang are being looked for, to the 1020 eV domain, where the highest-energy cosmic rays can be found. The emphasis is, however, on techniques employed in particle physics for studying collision processes at accelerators in the GeV-TeV range.

/pdf/energy-measurement-of-elementary-particles-1pz0ir41ph.pdf

Energy measurement of elementary particles

Experimental particle physicists study the fundamental structure of matter with a variety of approaches, which may be subdivided in two classes: accelerator and non-accelerator experiments. Accelerator experiments have the advantage of well-controlled experimental circumstances, non-accelerator experiments offer the possibility of studying processes that are not accessible to the available accelerator technology.

/pdf/calorimetry-in-high-energy-physics-1trl7at44h.pdf

Calorimetry in High Energy Physics

Abstract Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable 1,2 , including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. 

/pdf/magnetic-control-of-tokamak-plasmas-through-deep-2xsf5s2a.pdf

Magnetic control of tokamak plasmas through deep reinforcement learning

A prototye NaI(Tl) detector (the Cluster of 54) of spherical geometry subtending a solid angle of 7.5% of 4? at its center, has recently been assembled and tested. This detector consisted of 54 close-packed but optically isolated NaI(Tl) modules and the associated electronic circuitry. The Cluster of 54 is the predecessor of an almost complete spherical detector, the Crystal Ball, which will cover 94% of 4?. The latter detector is now under construction and is especially designed for the study of ?-rays produced in electron-positron collisions at colliding beam facilities. This article will outline the mechanical, optical, and electronic assembly of the prototype system. Cluster of 54 test data will be presented.

Design and Performance of a Modularized NaI(Tl) Detector (The Crystal Ball Prototype)

A fast and accurate turbulence transport model based on quasilinear gyrokinetics is developed. The model consists of a set of neural networks trained on a bespoke quasilinear GENE dataset, with a saturation rule calibrated to dedicated nonlinear simulations. The resultant neural network is approximately eight orders of magnitude faster than the original GENE quasilinear calculations. ITER predictions with the new model project a fusion gain in line with ITER targets. While the dataset is currently limited to the ITER baseline regime, this approach illustrates a pathway to develop reduced-order turbulence models both faster and more accurate than the current state-of-the-art.

https://pubs.aip.org/aip/pop/article-pdf/doi/10.1063/5.0136752/17925119/062501_1_5.0136752.pdf

D. Pfau

Papers

Magnetic control of tokamak plasmas through deep reinforcement learning

Design and Performance of a Modularized NaI(Tl) Detector (The Crystal Ball Prototype)

Fast transport simulations with higher-fidelity surrogate models for ITER