There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The organization of behavior: A neuropsychological theory

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

In-Datacenter Performance Analysis of a Tensor Processing Unit

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU) --- deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X -- 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X -- 80X higher. Moreover, using the CPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

/pdf/in-datacenter-performance-analysis-of-a-tensor-processing-4pmp83zqw4.pdf

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

This paper introduces a heterogeneous spiking neural network (H-SNN) as a novel, feedforward SNN structure capable of learning complex spatiotemporal patterns with spike-timing-dependent plasticity (STDP) based unsupervised training. Within H-SNN, hierarchical spatial and temporal patterns are constructed with convolution connections and memory pathways containing spiking neurons with different dynamics. We demonstrate analytically the formation of long and short term memory in H-SNN and distinct response functions of memory pathways. In simulation, the network is tested on visual input of moving objects to simultaneously predict for object class and motion dynamics. Results show that H-SNN achieves prediction accuracy on similar or higher level than supervised deep neural networks (DNN). Compared to SNN trained with back-propagation, H-SNN effectively utilizes STDP to learn spatiotemporal patterns that have better generalizability to unknown motion and/or object classes encountered during inference. In addition, the improved performance is achieved with 6x fewer parameters than complex DNNs, showing H-SNN as an efficient approach for applications with constrained computation resources.

/pdf/a-heterogeneous-spiking-neural-network-for-unsupervised-4nq3t0ehe8.pdf

A Heterogeneous Spiking Neural Network for Unsupervised Learning of Spatiotemporal Patterns.

Switching Energy in CMOS Logic: How far are we from physical limit?

Site-specific and on-demand cooling of hot spots in the microprocessors can provide efficient cooling solution, improve its performance and increase its life time by reducing peak temperature and achieving more uniform thermal profile on the chip. Thermoelectric coolers (TEC) have the potential to provide such efficient cooling of hot spots on a chip. We investigate pulse cooling behavior of ultra-thin multiple TEC devices integrated inside the electronic package on the active side of a chip below the heat spreader. Various pulse profiles have been studied to obtain optimal shape of the current pulse in order to efficiently operate TECs considering crucial parameters such as the total energy consumed in TECs, peak temperature on the chip, temperature overshoot at hot spot and settling time during pulsed cooling of hot spots. The square root pulse profile is found to be most effective with maximum cooling and half the energy expenditure in comparison to a constant current pulse. It has been observed that high thermal contact resistances can entirely negate the transient cooling effect of the TECs. We analyze the operation of multiple TECs for cooling spatiotemporally varying hot spots. The analysis shows that the temperature of the hot spots can be retained below a threshold using transient current pulses through the TECs. This underlines the benefits of using multiple TECs for hot spot cooling in order to obtain favorable thermal profile on the chip in an energy efficient way.Copyright © 2010 by ASME

Thermoelectric Coolers for Thermal Gradient Management on Chip

In this paper we model (numerically and analytically) and analyze sub-threshold, gate-to-channel tunneling, and edge direct tunneling leakage in double gate (DG) devices. We compare the leakage of different DG structures, namely, doped body symmetric device with polysilicon gates, intrinsic body symmetric device with metal gates and intrinsic body asymmetric device with different front and back gate material. It is observed that, use of (near-mid-gap) metal gate and intrinsic body devices significantly reduces both the total leakage and its sensitivity to parametric variations in DG circuits.

/pdf/modeling-and-analysis-of-total-leakage-currents-in-nanoscale-3coe9pg53v.pdf

Modeling and analysis of total leakage currents in nanoscale double gate devices and circuits

In this paper we propose a methodology for energy efficient Spin-Torque-Transfer Random Access Memory (STTRAM) array design at scaled technology nodes. We present a model to estimate and analyze the energy dissipation of an STTRAM array. The presented model shows the strong dependence of the array energy on the silicon transistor width, word line voltage and row/column organization. Using the array energy model we propose a design methodology for STTRAM arrays which minimizes the energy dissipation while maintaining the required robustness in read and write operations at scaled technologies.

Saibal Mukhopadhyay

Papers

A Heterogeneous Spiking Neural Network for Unsupervised Learning of Spatiotemporal Patterns.

Switching Energy in CMOS Logic: How far are we from physical limit?

Thermoelectric Coolers for Thermal Gradient Management on Chip

Modeling and analysis of total leakage currents in nanoscale double gate devices and circuits

A methodology for robust, energy efficient design of Spin-Torque-Transfer RAM arrays at scaled technologies