Todd Massengill

TL;DR: A new cloud architecture that uses reconfigurable logic to accelerate both network plane functions and applications, and is much more scalable than prior work which used secondary rack-scale networks for inter-FPGA communication.

TL;DR: This paper describes the NPU architecture for Project Brainwave, a production-scale system for real-time AI, and achieves more than an order of magnitude improvement in latency and throughput over state-of-the-art GPUs on large RNNs at a batch size of 1.5 teraflops.

TL;DR: Project Brainwave, Microsofts principal infrastructure for AI serving in real time, accelerates deep neural network inferencing in major services such as Bings intelligent search features and Azure by exploiting distributed model parallelism and pinning over low-latency hardware microservices.

TL;DR: The authors deploy the Configurable Cloud architecture over a production server bed and show how it can be used to accelerate applications that were explicitly ported to FPGAs and support hardware-first services.

TL;DR: The Project Brainwave NPU is described, a parameterized microarchitecture specialized at synthesis time for convolutional and recurrent DNN workloads that achieves sustained performance of 35 teraflops at a batch size of 1 on a large recurrent neural network (RNN).