Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Low power dissipation and maximum battery runtime are crucial in portable electronics. With accurate and efficient circuit and battery models in hand, circuit designers can predict and optimize battery runtime and circuit performance. In this paper, an accurate, intuitive, and comprehensive electrical battery model is proposed and implemented in a Cadence environment. This model accounts for all dynamic characteristics of the battery, from nonlinear open-circuit voltage, current-, temperature-, cycle number-, and storage time-dependent capacity to transient response. A simplified model neglecting the effects of self-discharge, cycle number, and temperature, which are nonconsequential in low-power Li-ion-supplied applications, is validated with experimental data on NiMH and polymer Li-ion batteries. Less than 0.4% runtime error and 30-mV maximum error voltage show that the proposed model predicts both the battery runtime and I-V performance accurately. The model can also be easily extended to other battery and power sourcing technologies.

Accurate electrical battery model capable of predicting runtime and I-V performance

A geometric program (GP) is a type of mathematical optimization problem characterized by objective and constraint functions that have a special form. Recently developed solution methods can solve even large-scale GPs extremely efficiently and reliably; at the same time a number of practical problems, particularly in circuit design, have been found to be equivalent to (or well approximated by) GPs. Putting these two together, we get effective solutions for the practical problems. The basic approach in GP modeling is to attempt to express a practical problem, such as an engineering analysis or design problem, in GP format. In the best case, this formulation is exact; when this is not possible, we settle for an approximate formulation. This tutorial paper collects together in one place the basic background material needed to do GP modeling. We start with the basic definitions and facts, and some methods used to transform problems into GP format. We show how to recognize functions and problems compatible with GP, and how to approximate functions or data in a form compatible with GP (when this is possible). We give some simple and representative examples, and also describe some common extensions of GP, along with methods for solving (or approximately solving) them.

/pdf/a-tutorial-on-geometric-programming-47sdznbzxw.pdf

A tutorial on geometric programming

There are two kinds of tutorial articles: those that provide a primer on an established topic and those that let us in on the ground floor of something of emerging importance. The first type of tutorial can have a noted expert who has been gracious (and brave) enough to write a field guide about a particular topic. The other sort of tutorial typically involves researchers who have each been laboring on a topic for some years. Both sorts of tutorial articles are very much desired. But we, as an editorial board for both Systems and Transactions, know that there has been no logical place for them in the AESS until this series was started several years ago. With these tutorials, we hope to continue to give them a home, a welcome, and provide a service to our membership. We do not intend to publish tutorials on a regular basis, but we hope to deliver them once or twice per year. We need and welcome good, useful tutorial articles (both kinds) in relevant AESS areas. If you, the reader, can offer a topic of interest and an author to write about it, please contact us. Self-nominations are welcome, and even more ideal is a suggestion of an article that the editor(s) can solicit. All articles will be reviewed in detail. Criteria on which they will be judged include their clarity of presentation, relevance, and likely audience, and, of course, their correctness and scientific merit. As to the mathematical level, the articles in this issue are a good guide: in each case the author has striven to explain complicated topics in simple-well, tutorial-terms. There should be no (or very little) novel material: the home for archival science is the Transactions Magazine, and submissions that need to be properly peer reviewed would be rerouted there. Likewise, articles that are interesting and descriptive, but lack significant tutorial content, ought more properly be submitted to the Systems Magazine.

What is a tutorial

Using detailed traces from 255 users, we conduct a comprehensive study of smartphone use. We characterize intentional user activities -- interactions with the device and the applications used -- and the impact of those activities on network and energy usage. We find immense diversity among users. Along all aspects that we study, users differ by one or more orders of magnitude. For instance, the average number of interactions per day varies from 10 to 200, and the average amount of data received per day varies from 1 to 1000 MB. This level of diversity suggests that mechanisms to improve user experience or energy consumption will be more effective if they learn and adapt to user behavior. We find that qualitative similarities exist among users that facilitate the task of learning user behavior. For instance, the relative application popularity for can be modeled using an exponential distribution, with different distribution parameters for different users. We demonstrate the value of adapting to user behavior in the context of a mechanism to predict future energy drain. The 90th percentile error with adaptation is less than half compared to predictions based on average behavior across users.

/pdf/diversity-in-smartphone-usage-2ok5cjokfk.pdf

Diversity in smartphone usage

Convolutional Neural Networks (CNNs) have gained popularity in many computer vision applications such as image classification, face detection, and video analysis, because of their ability to train and classify with high accuracy. Due to multiple convolution and fully-connected layers that are compute-/memory-intensive, it is difficult to perform real-time classification with low power consumption on today?s computing systems. FPGAs have been widely explored as hardware accelerators for CNNs because of their reconfigurability and energy efficiency, as well as fast turn-around-time, especially with high-level synthesis methodologies. Previous FPGA-based CNN accelerators, however, typically implemented generic accelerators agnostic to the CNN configuration, where the reconfigurable capabilities of FPGAs are not fully leveraged to maximize the overall system throughput. In this work, we present a systematic design space exploration methodology to maximize the throughput of an OpenCL-based FPGA accelerator for a given CNN model, considering the FPGA resource constraints such as on-chip memory, registers, computational resources and external memory bandwidth. The proposed methodology is demonstrated by optimizing two representative large-scale CNNs, AlexNet and VGG, on two Altera Stratix-V FPGA platforms, DE5-Net and P395-D8 boards, which have different hardware resources. We achieve a peak performance of 136.5 GOPS for convolution operation, and 117.8 GOPS for the entire VGG network that performs ImageNet classification on P395-D8 board.

Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks

Advances in battery technology have not kept pace with rapidly growing energy demands. Most laptops, handheld PCs, and cell phones use batteries that take anywhere from 1.5 to 4 hours to fully charge but can run on this charge for only a few hours. The battery has thus become a key control parameter in the energy management of portables. To meet the stringent power budget of these devices, researchers have explored various architectural, hardware, software, and system-level optimizations to minimize the energy consumed per useful computation. Research in battery-aware optimization is now moving from stand-alone devices to networks of wireless devices, specifically, ad hoc and distributed sensor networks. Computationally feasible mathematical models are now available that capture battery discharge characteristics in sufficient detail to let designers develop an optimization strategy that extracts maximum charge.

Battery modeling for energy aware system design

This paper presents a predictive model for the negative bias temperature instability (NBTI) of PMOS under both short term and long term operation. Based on the reaction-diffusion (R-D) mechanism, this model accurately captures the dependence of NBTI on the oxide thickness (tox), the diffusing species (H or H2) and other key transistor and design parameters. In addition, a closed form expression was derived for the threshold voltage change (DeltaVth ) under multiple cycle dynamic operation. Model accuracy and efficiency were verified with 90-nm experimental and simulation data. The impact of NBTI was further investigated on representative digital circuits

Predictive Modeling of the NBTI Effect for Reliable Design

As convolution layers contribute most operations in convolutional neural network (CNN) algorithms, an effective convolution acceleration scheme significantly affects the efficiency and performance of a hardware CNN accelerator. Convolution in CNNs involves three-dimensional multiply and accumulate (MAC) operations with four levels of loops, which results in a large design space. Prior works either employ limited loop optimization techniques, e.g. loop unrolling, tiling and interchange, or only tune some of the design variables after the accelerator architecture and dataflow are already fixed. Without fully studying the convolution loop optimization before the hardware design phase, the resulting accelerator can hardly exploit the data reuse and manage data movement efficiently. This work overcomes these barriers by quantitatively analyzing and optimizing the design objectives (e.g. required memory access) of the CNN accelerator based on multiple design variables. We systematically explore the trade-offs of hardware cost by searching the design variable configurations, and propose a specific dataflow of hardware CNN acceleration to minimize the memory access and data movement while maximizing the resource utilization to achieve high performance. The proposed CNN acceleration scheme and architecture are demonstrated on a standalone Altera Arria 10 GX 1150 FPGA by implementing end-to-end VGG-16 CNN model and achieved 645.25 GOPS of throughput and 47.97 ms of latency, which is a >3.2× enhancement compared to state-of-the-art FPGA implementations of VGG model.

Optimizing Loop Operation and Dataflow in FPGA Acceleration of Deep Convolutional Neural Networks

Negative-bias-temperature instability (NBTI) has become the primary limiting factor of circuit life time. In this paper, we develop a hierarchical framework for analyzing the impact of NBTI on the performance of logic circuits under various operation conditions, such as the supply voltage, temperature, and node switching activity. Given a circuit topology and input switching activity, we propose an efficient method to predict the degradation of circuit speed over a long period of time. The effectiveness of our method is comprehensively demonstrated with the International Symposium on Circuits and Systems (ISCAS) benchmarks and a 65-nm industrial design. Furthermore, we extract the following key design insights for reliable circuit design under NBTI effect, including: 1) During dynamic operation, NBTI-induced degradation is relatively insensitive to supply voltage, but strongly dependent on temperature; 2) There is an optimum supply voltage that leads to the minimum of circuit performance degradation; circuit degradation rate actually goes up if supply voltage is lower than the optimum value; 3) Circuit performance degradation due to NBTI is highly sensitive to input vectors. The difference in delay degradation is up to 5× for various static and dynamic operations. Finally, we examine the interaction between NBTI effect, and process and design uncertainty in realistic conditions.

Sarma Vrudhula

Papers

Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks

Battery modeling for energy aware system design

Predictive Modeling of the NBTI Effect for Reliable Design

Optimizing Loop Operation and Dataflow in FPGA Acceleration of Deep Convolutional Neural Networks

The Impact of NBTI Effect on Combinational Circuit: Modeling, Simulation, and Analysis