There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

The organization of behavior: A neuropsychological theory

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

In-Datacenter Performance Analysis of a Tensor Processing Unit

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU) --- deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X -- 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X -- 80X higher. Moreover, using the CPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

/pdf/in-datacenter-performance-analysis-of-a-tensor-processing-4pmp83zqw4.pdf

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

A sense amplifier circuit includes a first double-gate metal oxide semiconductor field effect transistor (DGMOSFET) having a first gate defining a first input to the circuit, a second gate and an output being coupled to a first output of the circuit and a second DGMOSFET having a first gate defining a second input of the circuit, a second gate connected to the output of the first DGMOSFET and an output connected to the second gate of the first DGMOSFET, the output of the second DGMOSFET being coupled to a second output of the circuit.

Sense amplifier circuit

Operating frequency of a pipelined circuit is determined by the delay of the slowest pipeline stage. However, under statistical delay variation in sub-100nm technology regime, the slowest stage is not readily identifiable and the estimation of the pipeline yield with respect to a target delay is a challenging problem. We have proposed analytical models to estimate yield for a pipelined design based on delay distributions of individual pipe stages. Using the proposed models, we have shown that change in logic depth and imbalance between the stage delays can improve the yield of a pipeline. A statistical methodology has been developed to optimally design a pipeline circuit for enhancing yield. Optimization results show that, proper imbalance among the stage delays in a pipeline improves design yield by 9% for the same area and performance (and area reduction by about 8.4% under a yield constraint) over a balanced design.

/pdf/statistical-modeling-of-pipeline-delay-and-design-of-1pkr9dt5wq.pdf

Statistical Modeling of Pipeline Delay and Design of Pipeline under Process Variation to Enhance Yield in sub-100nm Technologies

3-D integrated circuits (3-D ICs) are expected to have shorter wirelength, better performance, and less power consumption than 2-D ICs. These benefits come from die stacking and use of through-silicon vias (TSVs) fabricated for interconnections across dies. However, the use of TSVs has several negative impacts such as area and capacitance overhead. To predict the quality of 3-D ICs more accurately, TSV-aware 3-D wirelength distribution models considering the negative impacts were developed. In this paper, we apply an optimal buffer insertion algorithm to the TSV-aware 3-D wirelength distribution models and present various prediction results on wirelength, delay, and power consumption of 3-D ICs. We also apply the framework to 2-D and 3-D ICs built with various combinations of process and TSV technologies and predict the quality of today and future 3-D ICs.

TSV-Aware Interconnect Distribution Models for Prediction of Delay and Power Consumption of 3-D Stacked ICs

An inductive boost regulator is presented that can harvest energy over a wide input voltage range. The wide input range is enabled by a current-based analog oscillator capable of providing very high duty cycle. The analog control circuits are designed to operate at subthreshold voltage to enable low voltage all electronic autonomous startup. A test chip designed in 130-nm CMOS demonstrates energy harvesting from as low as 12-mV input while biased by an external 1-V battery and autonomous (without using any additional device or battery) startup above 305 mV at output. The design typically consumes 3.5-μA standby current, achieves peak efficiency of 82%, and can regulate output from 0.66 to 3.3 V. The booster is coupled with a thermoelectric generator and the integrated system delivers output power of 6 μW at ~2 °C temperature difference to a maximum of 12 mW at ~50 °C temperature difference.

A Wide Conversion Ratio, Extended Input 3.5-μA Boost Regulator With 82% Efficiency for Low-Voltage Energy Harvesting

In a wireless video sensor system, encoder and transmitter parameters should be jointly adjusted for the optimal tradeoff between video quality and energy consumption under variable channel conditions. The optimization of the system can be further enhanced by exploiting relative importance of moving objects in remote surveillance applications. This paper presents an energy-efficient wireless video sensor node for remote surveillance using content-aware pre-processing and an energy- and content-aware feedback control scheme. A low-overhead pre-processing engine detects the region of interest (ROI) in a frame defined as the macroblocks with moving objects, and an encoder selectively processes ROI macroblocks, resulting in reduction of both computation and transmission energy with minimum quality degradation of the ROI. We also propose an energy- and content-aware control scheme that jointly tunes the system parameters to maintain target transmission energy with minimum degradation of the ROI quality under varying wireless channel conditions. Compared to conventional encoding approaches, the proposed design consumes up to 56 percent less energy at the same quality.

Saibal Mukhopadhyay

Papers

Sense amplifier circuit

Statistical Modeling of Pipeline Delay and Design of Pipeline under Process Variation to Enhance Yield in sub-100nm Technologies

TSV-Aware Interconnect Distribution Models for Prediction of Delay and Power Consumption of 3-D Stacked ICs

A Wide Conversion Ratio, Extended Input 3.5-μA Boost Regulator With 82% Efficiency for Low-Voltage Energy Harvesting

An Energy-Efficient Wireless Video Sensor Node for Moving Object Surveillance