This paper presents improved architectures for a floating-point fused two-term dot product unit. The floating-point fused dot product unit is useful for a wide variety of digital signal processing (DSP) applications including complex multiplication and fast Fourier transform (FFT) and discrete cosine transform (DCT) butterfly operations. In order to improve the performance, a new alignment scheme, early normalization, a four-input leading zero anticipation (LZA), a dual-path algorithm, and pipelining are applied. The proposed designs are implemented for single precision and synthesized with a 45nm standard cell library. The proposed dual-path design reduces the latency by 25% compared to the traditional floating-point fused dot product unit. Based on a data flow analysis, the proposed design can be split into three pipeline stages. Since the latencies of the three stages are fairly well balanced, the throughput is increased by a factor of 2.8 compared to the non-pipelined dual-path design.

Improved Architectures for a Floating-Point Fused Dot Product Unit

Intel’s Nervana Neural Network Processor for Training (NNP-T) contains at its core an advanced floating point dot product design to accelerate the matrix multiplication operations found in many AI applications. Each Matrix Processing Unit (MPU) on the Intel NNP-T can process a 32x32 BFloat16 matrix multiplication every 32 cycles, accumulating the result in single precision (FP32). To reduce hardware costs, the MPU uses a fused many-term floating point dot product design with block alignment of the many input terms during addition, resulting in a unique datapath with several interesting design trade-offs. In this paper, we describe the details of the MPU pipeline, discuss the trade-offs made in the design, and present information on the accuracy of the computation as compared to traditional FMA implementations.

Intel Nervana Neural Network Processor-T (NNP-T) Fused Floating Point Many-Term Dot Product

We explore the floating-point arithmetic implemented in the NVIDIA tensor cores, which are hardware accelerators for mixed-precision matrix multiplication available on the Volta, Turing, and Ampere microarchitectures. Using Volta V100, Turing T4, and Ampere A100 graphics cards, we determine what precision is used for the intermediate results, whether subnormal numbers are supported, what rounding mode is used, in which order the operations underlying the matrix multiplication are performed, and whether partial sums are normalized. These aspects are not documented by NVIDIA, and we gain insight by running carefully designed numerical experiments on these hardware units. Knowing the answers to these questions is important if one wishes to: (1) accurately simulate NVIDIA tensor cores on conventional hardware; (2) understand the differences between results produced by code that utilizes tensor cores and code that uses only IEEE 754-compliant arithmetic operations; and (3) build custom hardware whose behavior matches that of NVIDIA tensor cores. As part of this work we provide a test suite that can be easily adapted to test newer versions of the NVIDIA tensor cores as well as similar accelerators from other vendors, as they become available. Moreover, we identify a non-monotonicity issue affecting floating point multi-operand adders if the intermediate results are not normalized after each step.

Numerical behavior of NVIDIA tensor cores.

Fast Fourier transform (FFT) coprocessor, having a significant impact on the performance of communication systems, has been a hot topic of research for many years. The FFT function consists of consecutive multiply add operations over complex numbers, dubbed as butterfly units. Applying floating-point (FP) arithmetic to FFT architectures, specifically butterfly units, has become more popular recently. It offloads compute-intensive tasks from general-purpose processors by dismissing FP concerns (e.g., scaling and overflow/underflow). However, the major downside of FP butterfly is its slowness in comparison with its fixed-point counterpart. This reveals the incentive to develop a high-speed FP butterfly architecture to mitigate FP slowness. This brief proposes a fast FP butterfly unit using a devised FP fused-dot-product-add (FDPA) unit, to compute AB ± CD ± E, based on binary-signed-digit (BSD) representation. The FP three-operand BSD adder and the FP BSD constant multiplier are the constituents of the proposed FDPA unit. A carry-limited BSD adder is proposed and used in the three-operand adder and the parallel BSD multiplier so as to improve the speed of the FDPA unit. Moreover, modified Booth encoding is used to accelerate the BSD multiplier. The synthesis results show that the proposed FP butterfly architecture is much faster than previous counterparts but at the cost of more area.

Floating-Point Butterfly Architecture Based on Binary Signed-Digit Representation

This paper makes a paradigm shift in the assumed notion of common subexpressions for complexity reduction of multiple constant multiplications implementation. Our proposed unified adder/subtractor (UAS)-based common subexpression elimination (CSE) algorithm is inspired by the recent advancement in complex arithmetic component mapping for datapath synthesis of digital systems. A dedicated UAS operator is designed at gate level to achieve arithmetic reduction for concurrent computation of the sum and difference of two input signals. To maximize computation reuse, dual subexpression is defined to enable a UAS to be shared by the otherwise incompatible odd and even common subexpressions. The three different types of common subexpression are uniquely encoded by a quadruple in the proposed data structure. Constant coefficients are represented by signed digits in Cartesian coordinate system from which nonoverlapping pairs of nonzero digits are parsed for dual, even, and odd subexpressions to maximize the reuse of all three types of arithmetic resources. The effectiveness of our proposed UAS-based CSE in overcoming the complexity reduction bottleneck are demonstrated by comparing the synthesis results obtained from six benchmark finite impulse response filters, an electroencephalogram filter bank, fast Fourier transform, and discrete cosine transform multipliers designed by ten algorithms. The results show a noteworthy 27.2% reduction in area-time complexity of our method over the baseline canonical signed digit implementation. Our solutions are also more power efficient, with average power saving of 12.0% over those designed by other algorithms in comparison.

A New Paradigm of Common Subexpression Elimination by Unification of Addition and Subtraction

This paper presents improved architectures for a fused floating-point add-subtract unit. The fused floating-point add-subtract unit is useful for digital signal processing (DSP) applications such as fast Fourier transform (FFT) and discrete cosine transform (DCT) butterfly operations. To improve the performance of the fused floating-point add-subtract unit, a dual-path algorithm and pipelining are employed. The proposed designs are implemented for both single and double precision and synthesized with a 45-nm standard-cell library. The fused floating-point add-subtract unit saves 40% of the area and power consumption compared to a discrete floating-point add-subtract unit. The proposed dual-path design reduces the latency by 30% compared to the discrete design with area and power consumption between that of the discrete and fused designs. Based on a data flow analysis, the proposed fused dual-path floating-point add-subtract unit can be split into two pipeline stages. Since the latencies of two pipeline stages are fairly well balanced, the throughput is increased by 80% compared to the nonpipelined dual-path design.

/pdf/improved-architectures-for-a-fused-floating-point-add-3229s4ogw9.pdf

Improved Architectures for a Fused Floating-Point Add-Subtract Unit

This paper presents improved architectures for a fused floating-point three-term adder. The fused floating-point three-term adder performs two additions in a single unit to achieve better performance and better accuracy compared to a network of traditional floating-point two-term adders, which is referred to as a discrete design. In order to further improve the performance of the three-term adder, several optimization techniques are applied including a new exponent compare and significand alignment, dual-reduction, early normalization, three-input leading zero anticipation, compound addition/rounding and pipelining. The proposed design is implemented for both single and double precision and synthesized with a 45 nm CMOS standard-cell library. The improved fused floating-point three-term adder reduces the area and power consumption by about 20% and reduces the latency by about 35% compared to a discrete floating-point three-term adder. Based on the data flow analysis, the proposed three-term adder can be split into three pipeline stages. Since the latencies of three pipeline stages are fairly well balanced, the throughput is increased to 2.7 times that of the non-pipelined design.

A Fused Floating-Point Three-Term Adder

This paper presents a novel design for a fused floating-point four-term dot product unit. The dot product is one of the most frequently used operations for a wide variety of graphics, digital signal processing (DSP) and statistical applications. The proposed design computes the four-term dot product in a single unit to achieve better performance and accuracy compared to a network of traditional floating-point multipliers and adders, which is referred to as a discrete design. In order to further improve the performance of the fused floating-point four-term dot product unit, a new exponent compare and significand alignment scheme, dual-reduction, early normalization, four-input Leading Zero Anticipation (LZA), and compound addition and rounding are applied. The proposed design is implemented for both single and double precision and synthesized with a 45 nm CMOS standard-cell library. In order to evaluate the improvement of the proposed design, the area, latency, and power consumption are investigated. The proposed design reduces the area, power consumption and latency by about 40% compared to the discrete design.

A Fused Floating-Point Four-Term Dot Product Unit

This paper examines a low-power consumption dual-path fused floating-point add-subtract unit and compares it with previous fused add-subtract units such as the single path fused add-subtract unit with/without enhanced aligning and compound adder-round unit and the high-speed dual-path fused add-subtract unit The high-speed dual-path fused add-subtract unit has less latency than the single-path fused add-subtract units, but with higher power consumption To reduce the power consumption, a dual-path fused add-subtract unit with a simplified far/close path scheme is proposed; the significand addition, subtraction and round units are not included in the far/close paths while each path of the high-speed version has these units The significand adder and subtractor are shared by far/close path The power consumption of the proposed design is 18% lower than the high-speed dual-path fused add-subtract unit at a 16% cost in latency; however, the proposed dual-path fused add-subtract unit is 11–20% faster than the two single-path fused add-subtract units

Jongwook Sohn

Papers

Improved Architectures for a Fused Floating-Point Add-Subtract Unit

Improved Architectures for a Floating-Point Fused Dot Product Unit

A Fused Floating-Point Three-Term Adder

A Fused Floating-Point Four-Term Dot Product Unit

A low-power dual-path floating-point fused add-subtract unit