On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central to more capable AI systems. OpenAI Five leveraged existing reinforcement learning techniques, scaled to learn from batches of approximately 2 million frames every 2 seconds. We developed a distributed training system and tools for continual training which allowed us to train OpenAI Five for 10 months. By defeating the Dota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.

Dota 2 with Large Scale Deep Reinforcement Learning

Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today’s machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this Perspective we seek to distil how many of deep learning’s failures can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in comparative psychology, education and linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications. Deep learning has resulted in impressive achievements, but under what circumstances does it fail, and why? The authors propose that its failures are a consequence of shortcut learning, a common characteristic across biological and artificial systems in which strategies that appear to have solved a problem fail unexpectedly under different circumstances.

Shortcut learning in deep neural networks

Quantum cryptography is arguably the fastest growing area in quantum
information science. Novel theoretical protocols are designed on a regular
basis, security proofs are constantly improving, and experiments are
gradually moving from proof-of-principle lab demonstrations to in-field
implementations and technological prototypes. In this paper, we provide
both a general introduction and a state-of-the-art description of the
recent advances in the field, both theoretical and experimental. We start
by reviewing protocols of quantum key distribution based on discrete
variable systems. Next we consider aspects of device independence,
satellite challenges, and protocols based on continuous-variable systems.
We will then discuss the ultimate limits of point-to-point private
communications and how quantum repeaters and networks may overcome these
restrictions. Finally, we will discuss some aspects of quantum
cryptography beyond standard quantum key distribution, including quantum
random number generators and quantum digital signatures.

/pdf/advances-in-quantum-cryptography-47uxe1y6u2.pdf

Advances in quantum cryptography

A new coded modulation scheme is proposed. At the transmitter, the concatenation of a distribution matcher and a systematic binary encoder performs probabilistic signal shaping and channel coding. At the receiver, the output of a bitwise demapper is fed to a binary decoder. No iterative demapping is performed. Rate adaption is achieved by adjusting the input distribution and the transmission power. The scheme is applied to bipolar amplitude-shift keying (ASK) constellations with equidistant signal points and it is directly applicable to two-dimensional quadrature amplitude modulation (QAM). The scheme is implemented by using the DVB-S2 low-density parity-check (LDPC) codes. At a frame error rate of $10^{-3}$ , the new scheme operates within less than 1.1 dB of the AWGN capacity $\frac{1}{2}\log_2(1+{\mathsf{SNR}})$ at any spectral efficiency between 1 and 5 bits/s/Hz by using only 5 modes, i.e., 4-ASK with code rate 2/3, 8-ASK with 3/4, 16-ASK and 32-ASK with 5/6, and 64-ASK with 9/10.

Bandwidth Efficient and Rate-Matched Low-Density Parity-Check Coded Modulation

The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.

/pdf/toward-causal-representation-learning-2ml0lp4x43.pdf

Toward Causal Representation Learning

We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot. This is made possible by two key components: a novel algorithm, which we call automatic domain randomization (ADR) and a robot platform built for machine learning. ADR automatically generates a distribution over randomized environments of ever-increasing difficulty. Control policies and vision state estimators trained with ADR exhibit vastly improved sim2real transfer. For control policies, memory-augmented models trained on an ADR-generated distribution of environments show clear signs of emergent meta-learning at test time. The combination of ADR with our custom robot platform allows us to solve a Rubik's cube with a humanoid robot hand, which involves both control and state estimation problems. Videos summarizing our results are available: this https URL

Solving Rubik's Cube with a Robot Hand.

We design staircase codes with overheads between 6.25% and 33.3% for high-speed optical transport networks. Using a reduced-complexity simulation of staircase coded transmission over the BSC, we select code candidates from within a limited parameter space. Software simulations of coded BSC transmission are performed with algebraic component code decoders. The net coding gain of the best code designs are competitive with the best known hard-decision decodable codes over the entire range of overheads. At 20% overhead, staircase codes are within 0.92 dB of BSC capacity at a bit error-rate of $10^{-15}$ . Decoding complexity and latency of the new staircase codes are also significantly reduced from existing hard-decision decodable schemes.

/pdf/staircase-codes-with-6-to-33-overhead-1kisca2txa.pdf

Staircase Codes With 6% to 33% Overhead

The speed at which two remote parties can exchange secret keys over a fixed-length fiber-optic cable in continuous-variable quantum key distribution (CV-QKD) is currently limited by the computational complexity of post-processing algorithms for key reconciliation. Multi-edge low-density parity-check (LDPC) codes with low code rates and long block lengths were proposed for CV-QKD, in order to extend the maximum reconciliation distance between the two remote parties. Key reconciliation over multiple dimensions has been shown to further improve the error-correction performance of multi-edge LDPC codes in CV-QKD, thereby increasing both the secret key rate and distance. However, the computational complexity of LDPC decoding for long block lengths on the order of 10^6 bits remains a challenge. This work introduces a quasi-cyclic (QC) code construction for multi-edge LDPC codes that is highly suitable for hardware-accelerated decoding on a modern graphics processing unit (GPU). When combined with an 8-dimensional reconciliation scheme, the LDPC decoder achieves a raw decoding throughput of 1.72Mbit/s and an information throughput of 7.16Kbit/s using an NVIDIA GeForce GTX 1080 GPU at a maximum distance of 160km with a secret key rate of 4.10x10^{-7} bits/pulse for a rate 0.02 multi-edge code with block length of 10^6 bits when finite-size effects are considered. This work extends the previous maximum CV-QKD distance of 100km to 160km, while delivering between 1.07x and 8.03x higher decoded information throughput over the upper bound on the secret key rate for a lossy channel. The GPU-based QC-LDPC decoder achieves a 1.29x improvement in throughput over the best existing GPU decoder implementation for a rate 1/10 multi-edge LDPC code with block length of 2^{20} bits. These results show that LDPC decoding is no longer the computational bottleneck in long-distance CV-QKD.

/pdf/key-reconciliation-with-low-density-parity-check-codes-for-4mdb7vvd1n.pdf

Key Reconciliation with Low-Density Parity-Check Codes for Long-Distance Quantum Cryptography

The speed at which two remote parties can exchange secret keys in continuous-variable quantum key distribution (CV-QKD) is currently limited by the computational complexity of key reconciliation. Multi-dimensional reconciliation using multi-edge low-density parity-check (LDPC) codes with low code rates and long block lengths has been shown to improve error-correction performance and extend the maximum reconciliation distance. We introduce a quasi-cyclic code construction for multi-edge codes that is highly suitable for hardware-accelerated decoding on a graphics processing unit (GPU). When combined with an 8-dimensional reconciliation scheme, our LDPC decoder achieves an information throughput of 7.16 Kbit/s on a single NVIDIA GeForce GTX 1080 GPU, at a maximum distance of 142 km with a secret key rate of 6.64 × 10−8 bits/pulse for a rate 0.02 code with block length of 106 bits. The LDPC codes presented in this work can be used to extend the previous maximum CV-QKD distance of 100 km to 142 km, while delivering up to 3.50× higher information throughput over the tight upper bound on secret key rate for a lossy channel. Improvements in the post-processing algorithms for quantum cryptography can extend the secure transmission distance by over 40%. Quantum key distribution protocols rely on the transmission of quantum states, but also on classical post-processing to eliminate errors introduced by imperfect equipment or the interference of an attacker. Over long distances, the requirements of this classical 'reconciliation' processing can become the bottleneck for key exchange. Mario Milicevic and colleagues from the University of Toronto and the University of British Columbia in Canada have developed a high-throughput error correction scheme that increases the potential operating range for quantum key distribution from 100 to 143 km. Their method is fast enough that the rate of key distribution is instead limited by the physical properties of the communication channel.

/pdf/quasi-cyclic-multi-edge-ldpc-codes-for-long-distance-quantum-2cy1b2xz1w.pdf

Quasi-cyclic multi-edge LDPC codes for long-distance quantum cryptography

A concatenated soft-decision forward error correction (FEC) scheme consisting of an inner low-density generator-matrix (LDGM) code and an outer staircase code is proposed. The soft-decision LDGM code is used for error reduction, while the majority of bit errors are corrected by the low-complexity hard-decision staircase code. Decoding complexity of the concatenated code is quantified by a score based on the number of edges in the LDGM code Tanner graph, the number of decoding iterations, and the number of staircase code decoding operations. The inner LDGM ensemble is designed by solving an optimization problem, which minimizes the product of the average node degree and an estimate of the required number of decoding iterations. A search procedure is used to find the inner and outer code pair with lowest complexity. The design procedure results in a Pareto-frontier characterization of the tradeoff between net coding gain and complexity for the concatenated code. Simulations of code designs at $\text{20}\%$ overhead showed that the proposed scheme achieves net coding gains equivalent to existing soft-decision FEC solutions, with up to $\text{57}\%$ reduction in complexity.

Lei M. Zhang

Papers

Solving Rubik's Cube with a Robot Hand.

Staircase Codes With 6% to 33% Overhead

Key Reconciliation with Low-Density Parity-Check Codes for Long-Distance Quantum Cryptography

Quasi-cyclic multi-edge LDPC codes for long-distance quantum cryptography

Low-Complexity Soft-Decision Concatenated LDGM-Staircase FEC for High-Bit-Rate Fiber-Optic Communication