Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning
Reads0
Chats0
TLDR
An artificial intelligence agent known as an asynchronous advantage actor-critic is utilized to explore type IIA compactifications with intersecting D6-branes to solve various string theory consistency conditions simultaneously, phrased in terms of non-linear, coupled Diophantine equations.Abstract:
We propose deep reinforcement learning as a model-free method for exploring the landscape of string vacua. As a concrete application, we utilize an artificial intelligence agent known as an asynchronous advantage actor-critic to explore type IIA compactifications with intersecting D6-branes. As different string background configurations are explored by changing D6-brane configurations, the agent receives rewards and punishments related to string consistency conditions and proximity to Standard Model vacua. These are in turn utilized to update the agent’s policy and value neural networks to improve its behavior. By reinforcement learning, the agent’s performance in both tasks is significantly improved, and for some tasks it finds a factor of $$ \mathcal{O}(200) $$
more solutions than a random walker. In one case, we demonstrate that the agent learns a human-derived strategy for finding consistent string models. In another case, where no human-derived strategy exists, the agent learns a genuinely new strategy that achieves the same goal twice as efficiently per unit time. Our results demonstrate that the agent learns to solve various string theory consistency conditions simultaneously, which are phrased in terms of non-linear, coupled Diophantine equations.read more
Citations
More filters
Journal ArticleDOI
Data science applications to string theory
Fabian Ruehle,Fabian Ruehle +1 more
TL;DR: While there is a strong focus on neural network applications in unsupervised, supervised and reinforcement learning, other machine learning techniques are discussed as well, including various clustering and anomaly detection algorithms, support vector machines, and decision trees.
Journal ArticleDOI
When does reinforcement learning stand out in quantum control? A comparative study on state preparation
TL;DR: A comparative study on the efficacy of three reinforcement learning algorithms: tabular Q- learning, deep Q-learning, and policy gradient, as well as two non-machine-learning methods: stochastic gradient descent and Krotov algorithms, in the problem of preparing a desired quantum state is performed.
Journal ArticleDOI
Machine Learning Line Bundle Cohomology
TL;DR: In this article, the authors investigate different approaches to machine learning of line bundle cohomology on complex surfaces as well as on Calabi-Yau three-folds, and set up a network capable of identifying the regions and their associated polynomials.
Journal ArticleDOI
Searching the landscape of flux vacua with genetic algorithms
TL;DR: It is shown that genetic algorithms can efficiently scan the landscape for viable solutions satisfying various criteria and it is argued that in both cases genetic algorithms are powerful tools for finding flux vacua with interesting phenomenological properties.
Book
The Calabi–Yau Landscape: From Geometry, to Physics, to Machine Learning
TL;DR: In this paper, the authors present a pedagogical introduction to the recent advances in computational geometry, physical implications, and data science of Calabi-Yau manifolds aimed at the beginning research student.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Journal ArticleDOI
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Journal ArticleDOI
Mastering the game of Go with deep neural networks and tree search
David Silver,Aja Huang,Chris J. Maddison,Arthur Guez,Laurent Sifre,George van den Driessche,Julian Schrittwieser,Ioannis Antonoglou,Veda Panneershelvam,Marc Lanctot,Sander Dieleman,Dominik Grewe,John Nham,Nal Kalchbrenner,Ilya Sutskever,Timothy P. Lillicrap,Madeleine Leach,Koray Kavukcuoglu,Thore Graepel,Demis Hassabis +19 more
TL;DR: Using this search algorithm, the program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0.5, the first time that a computer program has defeated a human professional player in the full-sized game of Go.
Journal ArticleDOI
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Journal ArticleDOI
Mastering the game of Go without human knowledge
David Silver,Julian Schrittwieser,Karen Simonyan,Ioannis Antonoglou,Aja Huang,Arthur Guez,Thomas Hubert,Lucas Baker,Matthew Lai,Adrian Bolton,Yutian Chen,Timothy P. Lillicrap,Fan Hui,Laurent Sifre,George van den Driessche,Thore Graepel,Demis Hassabis +16 more
TL;DR: An algorithm based solely on reinforcement learning is introduced, without human data, guidance or domain knowledge beyond game rules, that achieves superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo.