ACOCO: Adaptive Coding for Approximate Computing on Faulty Memories
TL;DR: The Adaptive Coding for approximate Computing (ACOCO) framework is presented, which provides an analysis-guided design methodology to develop adaptive codes for different computations on the data read from faulty memories.
Abstract: With scaling of process technologies and increase in process variations, embedded memories will be inherently unreliable. Approximate computing is a new class of techniques that relax the accuracy requirement of computing systems. In this paper, we present the A daptive Co ding for approximate Co mputing (ACOCO) framework, which provides us with an analysis-guided design methodology to develop adaptive codes for different computations on the data read from faulty memories. In ACOCO, we first compress the data by introducing distortion in the source encoder, and then add redundant bits to protect the data against memory errors in the channel encoder. We are thus able to protect the data against memory errors without additional memory overhead so that the coded data have the same bit-length as the uncoded data. We design the source encoder by first specifying a cost function measuring the effect of the data compression on the system output, and then design the source code according to this cost function. We develop adaptive codes for two types of systems under ACOCO. The first type of systems we consider, which includes many machine learning and graph-based inference systems, is the systems dominated by product operations. We evaluate the cost function statistics for the proposed adaptive codes, and demonstrate its effectiveness via two application examples: max-product image denoising and naive Bayesian classification. Next, we consider another type of systems: iterative decoders with min operation and sign-bit decision, which are widely applied in wireless communication systems. We develop an adaptive coding scheme for the min-sum decoder subject to memory errors. A density evolution analysis and simulations on finite length codes both demonstrate that the decoder with our adaptive code achieves a residual error rate that is on the order of the square of the residual error rate achieved by the nominal min-sum decoder.
Citations
More filters
TL;DR: In this paper, the authors consider the problem of computing a binary linear transformation when all logic gates in the computation are unreliable and derive a lower bound on the error probability of the computation of the linear transformation.
Abstract: We consider the problem of computing a binary linear transformation when all circuit components are unreliable. Two models of unreliable components are considered: probabilistic errors and permanent errors. We introduce the “ENCODED” technique that ensures that the error probability of the computation of the linear transformation is kept bounded below a small constant independent of the size of the linear transformation even when all logic gates in the computation are noisy. By deriving a lower bound, we show that in some cases, the computational complexity of the ENCODED technique achieves the optimal scaling in error probability. Further, we examine the gain in energy-efficiency from the use of a “voltage-scaling” scheme, where gate-energy is reduced by lowering the supply voltage. We use a gate energy-reliability model to show that tuning gate-energy appropriately at different stages of the computation (“dynamic” voltage scaling), in conjunction with ENCODED, can lead to orders of magnitude energy-savings over the classical “uncoded” approach. Finally, we also examine the problem of computing a linear transformation when noiseless decoders can be used, providing upper and lower bounds to the problem.
56 citations
Posted Content•
TL;DR: In this article, the authors consider the problem of computing a binary linear transformation using unreliable components when all circuit components are unreliable and derive a lower bound on the error probability of the linear transformation.
Abstract: We consider the problem of computing a binary linear transformation using unreliable components when all circuit components are unreliable. Two noise models of unreliable components are considered: probabilistic errors and permanent errors. We introduce the "ENCODED" technique that ensures that the error probability of the computation of the linear transformation is kept bounded below a small constant independent of the size of the linear transformation even when all logic gates in the computation are noisy. Further, we show that the scheme requires fewer operations (in order sense) than its "uncoded" counterpart. By deriving a lower bound, we show that in some cases, the scheme is order-optimal. Using these results, we examine the gain in energy-efficiency from use of "voltage-scaling" scheme where gate-energy is reduced by lowering the supply voltage. We use a gate energy-reliability model to show that tuning gate-energy appropriately at different stages of the computation ("dynamic" voltage scaling), in conjunction with ENCODED, can lead to order-sense energy-savings over the classical "uncoded" approach. Finally, we also examine the problem of computing a linear transformation when noiseless decoders can be used, providing upper and lower bounds to the problem.
47 citations
University of Cambridge1, University of Washington2, Massachusetts Institute of Technology3, Max Planck Society4, University of California, Los Angeles5, University of Texas at Austin6, University of Twente7, National University of Singapore8, École Polytechnique Fédérale de Lausanne9, Georgia Institute of Technology10, University of Toronto11, University of Illinois at Urbana–Champaign12, Cornell University13
TL;DR: This work presents a synthesis of research results on computing systems that only make as many errors as their end-to-end applications can tolerate, and introduces a formalization of terminology that allows for a coherent view across the techniques traditionally used by different research communities in their individual layer of focus.
Abstract: When a computational task tolerates a relaxation of its specification or when an algorithm tolerates the effects of noise in its execution, hardware, system software, and programming language compilers or their runtime systems can trade deviations from correct behavior for lower resource usage. We present, for the first time, a synthesis of research results on computing systems that only make as many errors as their end-to-end applications can tolerate. The results span the disciplines of computer-aided design of circuits, digital system design, computer architecture, programming languages, operating systems, and information theory. Rather than over-provisioning the resources controlled by each of these layers of abstraction to avoid errors, it can be more efficient to exploit the masking of errors occurring at one layer and thereby prevent those errors from propagating to a higher layer. We demonstrate the potential benefits of end-to-end approaches using two illustrative examples. We introduce a formalization of terminology that allows us to present a coherent view across the techniques traditionally used by different research communities in their individual layer of focus. Using this formalization, we survey tradeoffs for individual layers of computing systems at the circuit, architecture, operating system, and programming language levels as well as fundamental information-theoretic limits to tradeoffs between resource usage and correctness.
27 citations
Cites background or methods from "ACOCO: Adaptive Coding for Approxim..."
...[86], which proposes a redundancy-free adaptive code that can correct errors in data retrieved from potentially faulty cells....
[...]
...[86] present a simple yet effective coding scheme that uses a combination of a lossy source/channel coding to protect against hardware errors for iterative statistical inference....
[...]
TL;DR: This work proposes a principled approach to determine optimal non-uniform bit-line swings by formulating convex optimization problems and shows that energy-optimal swing assignment reduces energy consumption by half at a peak signal-to-noise ratio of 30 dB for an 8-bit accessed word.
Abstract: Conventional low-power static random access memories (SRAMs) reduce read energy by decreasing the bit-line voltage swings uniformly across the bit-line columns. This is because the read energy is proportional to the bit-line swings. On the other hand, bit-line swings are limited by the need to avoid decision errors especially in the most significant bits. We propose a principled approach to determine optimal non-uniform bit-line swings by formulating convex optimization problems. For a given constraint on mean squared error of retrieved words, we consider criteria to minimize energy (for low-power SRAMs), maximize speed (for high-speed SRAMs), and minimize energy-delay product. These optimization problems can be interpreted as classical water-filling, ground-flattening and water-filling, and sand-pouring and water-filling, respectively. By leveraging these interpretations, we also propose greedy algorithms to obtain optimized discrete swings. Numerical results show that energy-optimal swing assignment reduces energy consumption by half at a peak signal-to-noise ratio of 30 dB for an 8-bit accessed word. The energy savings increase to four times for a 16-bit accessed word.
17 citations
Cites background from "ACOCO: Adaptive Coding for Approxim..."
...In [20], the authors proposed adaptive coding techniques for different computations on the data read from faulty memories....
[...]
Posted Content•
University of Cambridge1, University of Washington2, Massachusetts Institute of Technology3, Max Planck Society4, University of California, Los Angeles5, University of Texas at Austin6, National University of Singapore7, École Polytechnique Fédérale de Lausanne8, University of Toronto9, University of Illinois at Urbana–Champaign10, Cornell University11
TL;DR: A synthesis of research results on computing systems that only make as many errors as their users can tolerate is presented, for the first time, from across the disciplines of computer aided design of circuits, digital system design, computer architecture, programming languages, operating systems, and information theory.
Abstract: When a computational task tolerates a relaxation of its specification or when an algorithm tolerates the effects of noise in its execution, hardware, programming languages, and system software can trade deviations from correct behavior for lower resource usage. We present, for the first time, a synthesis of research results on computing systems that only make as many errors as their users can tolerate, from across the disciplines of computer aided design of circuits, digital system design, computer architecture, programming languages, operating systems, and information theory. Rather than over-provisioning resources at each layer to avoid errors, it can be more efficient to exploit the masking of errors occurring at one layer which can prevent them from propagating to a higher layer. We survey tradeoffs for individual layers of computing systems from the circuit level to the operating system level and illustrate the potential benefits of end-to-end approaches using two illustrative examples. To tie together the survey, we present a consistent formalization of terminology, across the layers, which does not significantly deviate from the terminology traditionally used by research communities in their layer of focus.
15 citations
Cites background or methods from "ACOCO: Adaptive Coding for Approxim..."
...[80] which proposes a redundancy-free adaptive code that can correct errors in data retrieved from potentially faulty cells....
[...]
...[80] present a simple yet effective coding scheme that uses a combination of a lossy source/channel coding to protect against hardware errors for iterative statistical inference....
[...]
References
More filters
TL;DR: This work presents two algorithms based on graph cuts that efficiently find a local minimum with respect to two types of large moves, namely expansion moves and swap moves that allow important cases of discontinuity preserving energies.
Abstract: Many tasks in computer vision involve assigning a label (such as disparity) to every pixel. A common constraint is that the labels should vary smoothly almost everywhere while preserving sharp discontinuities that may exist, e.g., at object boundaries. These tasks are naturally stated in terms of energy minimization. The authors consider a wide class of energies with various smoothness constraints. Global minimization of these energy functions is NP-hard even in the simplest discontinuity-preserving case. Therefore, our focus is on efficient approximation algorithms. We present two algorithms based on graph cuts that efficiently find a local minimum with respect to two types of large moves, namely expansion moves and swap moves. These moves can simultaneously change the labels of arbitrarily large sets of pixels. In contrast, many standard algorithms (including simulated annealing) use small moves where only one pixel changes its label at a time. Our expansion algorithm finds a labeling within a known factor of the global minimum, while our swap algorithm handles more general energy functions. Both of these algorithms allow important cases of discontinuity preserving energies. We experimentally demonstrate the effectiveness of our approach for image restoration, stereo and motion. On real data with ground truth, we achieve 98 percent accuracy.
7,413 citations
"ACOCO: Adaptive Coding for Approxim..." refers methods in this paper
...In this example, we associate the 2-D grid factor graph for the “penguin” image (with size 179 × 122 and 256 gray-scale levels) to a Potts model [35] with coefficient ξ = 0....
[...]
Book•
01 Jan 1983
TL;DR: This book explains coding for Reliable Digital Transmission and Storage using Trellis-Based Soft-Decision Decoding Algorithms for Linear Block Codes and Convolutional Codes, and some of the techniques used in this work.
Abstract: 1. Coding for Reliable Digital Transmission and Storage. 2. Introduction to Algebra. 3. Linear Block Codes. 4. Important Linear Block Codes. 5. Cyclic Codes. 6. Binary BCH Codes. 7. Nonbinary BCH Codes, Reed-Solomon Codes, and Decoding Algorithms. 8. Majority-Logic Decodable Codes. 9. Trellises for Linear Block Codes. 10. Reliability-Based Soft-Decision Decoding Algorithms for Linear Block Codes. 11. Convolutional Codes. 12. Trellis-Based Decoding Algorithms for Convolutional Codes. 13. Sequential and Threshold Decoding of Convolutional Codes. 14. Trellis-Based Soft-Decision Algorithms for Linear Block Codes. 15. Concatenated Coding, Code Decomposition ad Multistage Decoding. 16. Turbo Coding. 17. Low Density Parity Check Codes. 18. Trellis Coded Modulation. 19. Block Coded Modulation. 20. Burst-Error-Correcting Codes. 21. Automatic-Repeat-Request Strategies.
3,848 citations
"ACOCO: Adaptive Coding for Approxim..." refers methods in this paper
...Based on our independent BSC memory error model, we use linear block codes as our channel codes, since they are known to be good for correcting independent bit flips [6], [29], [30]....
[...]
01 Jan 1956
2,140 citations
[...]
TL;DR: This summary of the state-of-the-art in iterative coding makes this decision more straightforward, with emphasis on the underlying theory, techniques to analyse and design practical iterative codes systems.
Abstract: Having trouble deciding which coding scheme to employ, how to design a new scheme, or how to improve an existing system? This summary of the state-of-the-art in iterative coding makes this decision more straightforward. With emphasis on the underlying theory, techniques to analyse and design practical iterative coding systems are presented. Using Gallager's original ensemble of LDPC codes, the basic concepts are extended for several general codes, including the practically important class of turbo codes. The simplicity of the binary erasure channel is exploited to develop analytical techniques and intuition, which are then applied to general channel models. A chapter on factor graphs helps to unify the important topics of information theory, coding and communication theory. Covering the most recent advances, this text is ideal for graduate students in electrical engineering and computer science, and practitioners. Additional resources, including instructor's solutions and figures, available online: www.cambridge.org/9780521852296.
2,100 citations
IBM1
TL;DR: This work analyzes the impact of the distribution entropy on the classification error, showing that low-entropy feature distributions yield good performance of naive Bayes and demonstrates that naive Baye works well for certain nearlyfunctional feature dependencies.
Abstract: The naive Bayes classifier greatly simplify learning by assuming that features are independent given class. Although independence is generally a poor assumption, in practice naive Bayes often competes well with more sophisticated classifiers. Our broad goal is to understand the data characteristics which affect the performance of naive Bayes. Our approach uses Monte Carlo simulations that allow a systematic study of classification accuracy for several classes of randomly generated problems. We analyze the impact of the distribution entropy on the classification error, showing that low-entropy feature distributions yield good performance of naive Bayes. We also demonstrate that naive Bayes works well for certain nearlyfunctional feature dependencies, thus reaching its best performance in two opposite cases: completely independent features (as expected) and functionally dependent features (which is surprising). Another surprising result is that the accuracy of naive Bayes is not directly correlated with the degree of feature dependencies measured as the classconditional mutual information between the features. Instead, a better predictor of naive Bayes accuracy is the amount of information about the class that is lost because of the independence assumption.
2,046 citations
"ACOCO: Adaptive Coding for Approxim..." refers background in this paper
...sifier has been proven effective in many practical applications, including text classification, medical diagnosis, and systems performance management [36]....
[...]