scispace - formally typeset
SciSpace - Your AI assistant to discover and understand research papers | Product Hunt

Proceedings ArticleDOI

Exploiting Posit Arithmetic for Deep Neural Networks in Autonomous Driving Applications

09 Jul 2018-pp 1-6

TL;DR: It is argued that a new representation for floating point numbers called Posit is particularly advantageous, allowing for a better trade-off between computation accuracy and implementation complexity.

AbstractThis paper discusses the introduction of an integrated Posit Processing Unit (PPU) as an alternative to Floating-point Processing Unit (FPU) for Deep Neural Networks (DNNs) in automotive applications. Autonomous Driving tasks are increasingly depending on DNNs. For example, the detection of obstacles by means of object classification needs to be performed in real-time without involving remote computing. To speed up the inference phase of DNNs the CPUs on-board the vehicle should be equipped with co-processors, such as GPUs, which embed specific optimization for DNN tasks. In this work, we review an alternative arithmetic that could be used within the co-processor. We argue that a new representation for floating point numbers called Posit is particularly advantageous, allowing for a better trade-off between computation accuracy and implementation complexity. We conclude that implementing a PPU within the co-processor is a promising way to speed up the DNN inference phase.

Summary (3 min read)

Introduction

  • For datafusion a grid-based approach may be used to determine the occupancy probability (Bayesian approach) of a cell, or the belief function (Dempster-Shafer approach), by evaluating the current sensor reading and the history from past cycle [11].
  • To accelerate DNN computing in automotive applications, this paper discusses the introduction of the Posit Processing Unit (PPU) as alternative to the Floating-point Processing Unit (FPU).
  • In Sections IV the authors argue that 16- bit Posits can replace conventional FPUs, since they are more accurate if the same number of bits is used.

II. ALTERNATIVE REAL NUMBER REPRESENTATIONS

  • Representing real numbers in electronic computers requires the selection of a method to map real numbers into a sequence of bits.
  • Hardware realization of specific elementary functions are also desired.

A. Type-I Unums

  • Type-I uniform numbers have been introduced in Gustafson’s book [20].
  • While the sign, exponent, and fraction bit fields take their definition from IEEE 754, the exponent and fraction field lengths vary automatically, from a single bit up to some user specified maximum .
  • Type-I Unums have been recently implemented in hardware, on FPGA [16, 22], also known as Hardware implementations.
  • This is due to both the variable length operands and the interval arithmetic, which is more complex than arithmetic between floats.
  • The authors believe that Type-I Unums are not particularly appealing for DNN in autonomous driving applications, due to the disadvantages mentioned above.

B. Type-II Unums

  • Type-II Unums have been introduced again by Prof. Gustafson in [23].
  • Here the key idea was to represent both exact real numbers and the open interval between two consecutive exact real numbers represented.
  • In addition, computing the opposite and the reciprocal takes the same time Disadvantages:.
  • The SORN approach needs look-up tables requiring large amounts of ROM/RAM.
  • Unums have not been implemented in hardware yet.

C. Other formats

  • Jorgensen has recently patented a format which retains the correct bounds after each computation and the corresponding hardware [24].
  • This proposal is another evidence of the need of novel formats for real numbers and novel hardware implementations, and, although interesting, is too similar to Type-I Unums.
  • On the other hand Type-I Unums are free from patents rights.
  • In conclusion of this section, none of these newly proposed floating point representations are satisfactory for DNNs.
  • On the contrary, the Posit data format, explained in the next section has promising properties.

III. THE POSIT REPRESENTATION

  • Posits are a by-product of the Type-II Unums numbers described in previous section and they have been introduced very recently in [25], again by Gustafson.
  • Please observe how the two Posits representations shown in the figure have a different number of bits reserved for the fraction field (8 and 9, respectively), having different lengths for the regime fields (4 vs 3).
  • Please notice that the floating-point standard wastes a lot of representations for NaNs, which also make the hardware for comparing floats very complex.
  • Thus comparing two Posits can be done in ALU (by type re-casting to signed integers): negative Posits are expressed using complement two as integers, and the other three fields allow direct ordering.

A. Dimensioning Posits

  • As mentioned the Posit representation is parametrized over the total number of bits to use and the number of them to reserve to the exponent.
  • Depending on the task at hand, these two numbers can chosen accordingly.
  • Thus the authors know that 16-bit Posits are sufficient too.
  • In addition the authors observe that the Posit representation is compatible with the flexpoint idea introduced in [15].
  • Indeed, it is possible to create Posits tensors that share an extra exponent field.

IV. HARDWARE IMPLEMENTATION ISSUES

  • The implementation of a PPU will take advantage of the operations over Posits that can be traced back to three main levels of Posit representation: i) binary form, ii) decoded form, and iii) expanded form.
  • The first one is the binary representation of Posits discussed so far that can be used for some operations, with the effect of fastest speed and minimal complexity.
  • The second form decodes the Posit in the three components of regime, exponent and fraction by parsing the variable length encoding of the regime field.
  • Inversion, doubling and halving can be efficiently computed by working on this intermediate representation, in addition to the fundamental arithmetic operations when fraction is zero.
  • The third level requires a more sophisticated logic that, anyway, in comparison to IEEE floating points, can take advantage of the separation between regime and exponent parts.

A. The advantage of using shared registers between ALU and PPU

  • The property of Posit representation that allows to perform comparisons using the integer comparison can be exploited for a subset of DNN operations.
  • The authors are referring on particular to the argmax operation at the end of the inference, to the ReLu activation function, and to Max pooling in convolutional neural networks (see next subsection).
  • Some Posit operation could require custom instructions while others (such as the comparison) could be obtained via integer instructions.
  • Figure 7 shows the schematic of registers within the co-processor, to reuse the ALU for comparing Posits.
  • This approach, depicted in Figure 7, is common to modern architecture (e.g. Intel Core CPUs) that generalize this properties to larger register banks, up to 512 bits.

B. Specific DNN functions to implement in hardware

  • In existing custom hardware for Tensor operations (e.g. Nvidia Tensor units and Google TPU) there are specific operations that are structured in hardware due to their repetitive nature.
  • In the following the authors are considering some fundamental DNN operations that can take advantage of Posit representation and that can be directly executed.
  • DNN functions that are interesting to implement in hardware are the following:.

C. PPU emulation library

  • Posits library has been implemented using a C++ template library parametrized over Posit length and exponent length.
  • The implementation takes advantage of the representation levels discussed in Section IV.
  • Moreover level 3 operations of small Posits (e.g. less than 14 bits) can be implemented in the library using lookup tables.
  • The use of a look-up table is interesting for those applications requiring a low precision and for those computers having sufficient cache memory, like current desktop PCs.
  • In particular the library can be used for working with the C++ Eigen template matrices library [29].

D. Summary: the advantages of Posits over Floats

  • More in general, the authors confirm that Posits are an interesting data format for low-precision arithmetic.
  • In particular, using Posit16 the authors are widening the range of applications for which 16 bits are enough, thus saving bandwidth, storage and energy consumption in comparison to Float32.

Did you find this useful? Give us your feedback

...read more

Content maybe subject to copyright    Report

Exploiting Posit Arithmetic for Deep Neural Networks
in Autonomous Driving Applications
Marco Cococcioni
1
, Emanuele Ruffaldi
2
and Sergio Saponara
1
Abstract This paper discusses the introduction of an in-
tegrated Posit Processing Unit (PPU) as an alternative to
Floating-point Processing Unit (FPU) for Deep Neural Networks
(DNNs) in automotive applications. Autonomous Driving tasks
are increasingly depending on DNNs. For example, the detection
of obstacles by means of object classification needs to be
performed in real-time without involving remote computing.
To speed up the inference phase of DNNs the CPUs on-board
the vehicle should be equipped with co-processors, such as
GPUs, which embed specific optimization for DNN tasks. In this
work, we review an alternative arithmetic that could be used
within the co-processor. We argue that a new representation for
floating point numbers called Posit is particularly advantageous,
allowing for a better trade-off between computation accuracy
and implementation complexity. We conclude that implement-
ing a PPU within the co-processor is a promising way to speed
up the DNN inference phase.
I. INTRODUCTION
Assisted and Autonomous Driving (AD) require the un-
derstanding of the physical environment outside the vehicle
as shown in Figure 1. New on-board automotive computing
architectures [1]–[6] will exploit powerful embedded High
Performance Computing (eHPC) platforms, such as NVDIA
Pegasus or Intel GO, implementing in real-time the following
vehicle perception and autonomous decision tasks:
Observation: building a model of the surrounding en-
vironment, where inputs are the direct observations
produced by sensors (visual and infrared cameras, radar,
sonar, lidar [7]) and output is a geometrical and topo-
logical representation of the environment.
Perception: localization of the vehicle, i.e. estimating its
path, position and orientation within a map, by fusing
global and relative data (e.g. Global Navigation Satellite
signals fused with local accelerometer and gyro inertial
sensors); detection of all static and dynamic obstacles
(e.g. landmarks, road and traffic signs, vehicles, pedes-
trian, bikers) and their classification depending on how
well they match up with a library of pre-determined
shape and motion descriptors.
Planning and decision: move the car, which requires
Artificial Intelligence (AI) for adaptive route planning
and trajectory control used to direct the vehicle to its
destination, avoiding obstacles, following traffic rules,
1
University of Pisa, Department of Information Engineering,
Via Caruso 16, Pisa - 56123 Italy {marco.cococcioni,
sergio.saponara}@unipi.it
2
MMI s.p.a., Via del Paduletto 10A, 56011, Calci (PI), Italy
emanuele.ruffaldi@mmimicro.com
and predicting the behavior of neighbor vehicles, bikers
and pedestrian.
Online map data is required to provide long range planning
information such as lane end, speed limits, construction sites
and other changing road conditions. All these operations
have to be repeated in a time scale of about 5 ms (200 Hz)
with stringent low-latency requirements [8, 9]. To support
the functions of mapping, localization and object identi-
fication, a perception process must be implemented (e.g.
[10]). Perception results from fusion of all surround sensing
and online map data into single surround model. For data-
fusion a grid-based approach may be used to determine the
occupancy probability (Bayesian approach) of a cell, or the
belief function (Dempster-Shafer approach), by evaluating
the current sensor reading and the history from past cycle
[11]. Grid occupancy is calculated from sensor data, with
explicit modeling of uncertainties. Grid cells can bear addi-
tional information such as moving object speed, which can
be used to predict likely behavior.
The current and future trend is solving such difficult tasks
by using Deep Neural Networks (DNNs) [12]–[17] trained
on millions of frames on a supercomputer based on GPUs
with units specialized for tensor operations. Once the DNN
has been trained and validated, it can be used in real-time
on-board a vehicle (a car or truck or a public transport
bus/taxi), to understand the environment and help and inform
higher-level of the decision and control process. The speed
of the DNN inference is crucial in this kind of latency-
critical applications. But even more crucial is the fact that
autonomous driving is a safety critical application, thus it
requires a reliable software implementation of DNNs on
a reliable hardware. In summary, a fast and reliable DNN
module is required, either implemented in software or in
hardware. Regarding the inference phase (the only phase that
need to be performed on-board the vehicle), 8 bit or less
are enough on non-safety critical applications [17]–[19], by
using vector quantization or integer/fixed-point arithmetic.
On the contrary, safety critical applications (see critical
automotive standards, such as the ISO 26262 functional
safety specification) still require 16 or 32 bits and thus
floating point representations must be preferred over fixed
point/integer representations.
To accelerate DNN computing in automotive applications,
this paper discusses the introduction of the Posit Processing
Unit (PPU) as alternative to the Floating-point Process-
ing Unit (FPU). After discussing in Section II alternative
representations for real numbers, the new Posits format is

Fig. 1. AD architecture with perception (on the left) based on sensor data
fusion and planning (on the right). Both functions can take advantage of AI
approaches in particular deep learning techniques.
proposed in Section III. In Sections IV we argue that 16-
bit Posits can replace conventional FPUs, since they are
more accurate if the same number of bits is used. Moreover,
Posits enjoy other interesting properties, like the possibility
to compare two Posits by treating them as two integers on
ALU. Hardware/ Software implementations issues of a PPU
are also discussed, together with results we obtained by em-
ulating a PPU exploiting a General Purpose microprocessor
(GPP) or a microcontroller. Conclusions and future hints of
the use of PPU for the DNN training phase are presented in
Section V.
II. ALTERNATIVE REAL NUMBER REPRESENTATIONS
Representing real numbers in electronic computers re-
quires the selection of a method to map real numbers into a
sequence of bits. In addition, a circuitry is typically required
to perform in hardware (and thus, in a fast way) both
the basic four arithmetic operations (+, , ×, /) and the
comparison operators of two numbers (<, ==, >, , ).
Hardware realization of specific elementary functions are
also desired.
A. Type-I Unums
Type-I uniform numbers have been introduced in
Gustafson’s book [20]. The Type-I Unums data format is a
superset of IEEE 754 Standard floating-point format; it uses
a “ubit” at the end of the fraction to indicate whether a real
number is an exact float or lies in the open interval between
adjacent floats. While the sign, exponent, and fraction bit
fields take their definition from IEEE 754, the exponent and
fraction field lengths vary automatically, from a single bit
up to some user specified maximum (see Figure 2). Type-I
Unums provide a compact way to express interval arithmetic,
but their variable length demands extra management. They
can express IEEE float behavior, via an explicit rounding
function.
Advantages: superset of floats.
Disadvantages: variable length; require complex inter-
val arithmetic.
0
1 2
3
4
5 6
7
8
9
10
11 12
13
14
15 16
17
18
19
20
21 22
23
24
25 26
27
28
29
30 31
S
Exp (8 bits)
Fraction (12 bits)
U
Esize Fsize
| {z }
Utag
Fig. 2. An example of Type-I Unum. The first three fields are the same of
floating point numbers, the remaining ubit, esize, fsize are additional fields.
Software implementations: Type-I Unums have been
implemented in Matlab and applied to a Model Predic-
tive Control Problems [21]. In that work, the authors
showed a save of 80% of storage space with respect the
use of Floats, without loosing in accuracy.
Hardware implementations: Type-I Unums have been
recently implemented in hardware, on FPGA [16, 22].
The authors conclude that implementing in hardware
Type-I Unums requires a bigger area than that of an
FPU. This is due to both the variable length operands
and the interval arithmetic, which is more complex
than arithmetic between floats. We believe that Type-I
Unums are not particularly appealing for DNN in au-
tonomous driving applications, due to the disadvantages
mentioned above.
B. Type-II Unums
Type-II Unums have been introduced again by Prof.
Gustafson in [23]. The basic requirement in designing Type-
II Unums was a fixed length data type. Fixed length is
very important when working with problems showing data
parallelism and, more in general, to efficiently work with
arrays (vectors, matrices, tensors, etc.). Another important
designing criteria was the desire to not overlook any real
number. All real numbers from inf to + inf should be
represented, although with different precisions. Here the
key idea was to represent both exact real numbers and the
open interval between two consecutive exact real numbers
represented. Then Gustafson exploited the idea of “projective
reals”, e.g., the idea of mapping the real line to a circle, with
the intent of constructing a bijection between the line and the
circle. By using this approach, Gustafson was able to express
the opposite of a number in the circle via “horizontal flip-
ping”, while, more interestingly, the reciprocal of a number is
obtained exactly by “vertical flipping” of the representation.
This is a very important property of Type-II Unums,
since reciprocating a number is straightforward and thus
computing the division requires a similar amount of time
of computing the product (by computing a/b as a × b
1
).
This means that the “bad boy” of computing (dividing
two real numbers) is removed, and all the four arithmetic
operations require similar amount of time. Figure 3 shows
Type-II Unums when using only 4 bits, represented on the
mentioned circle. Finally, the idea of Type-II Unums was
the computation of accurate and reproducible results, even
on parallel computers. Based on this Gustafson introduced
the idea of Set of Real Numbers (SORNs), an intriguing
way to represent any set of Type-II Unums. Unfortunately
operations on SORNs require a look-up table, thus limiting

Fig. 3. Type-II Unums on 4 bits. This kind of numbers can be represented
on a circle. Even more interesting, the representation of the reciprocal of a
number can be easily obtained by working on the representation itself, by
performing a rotation on the horizontal axis.
the total length of Type-II Unums on current hardware to 20
bits or less.
Advantages: they have fixed size, in contrast to Type-
I Unums. In addition, computing the opposite and the
reciprocal takes the same time
Disadvantages: The SORN approach needs look-up
tables requiring large amounts of ROM/RAM.
Hardware implementations: Type-II Unums have not
been implemented in hardware yet. The SORN ap-
proach they are based allows using a look-up table, for
numbers up to 20 bits.
C. Other formats
Jorgensen has recently patented a format which retains the
correct bounds after each computation and the corresponding
hardware [24]. This proposal is another evidence of the
need of novel formats for real numbers and novel hardware
implementations, and, although interesting, is too similar
to Type-I Unums. On the other hand Type-I Unums are
free from patents rights. In conclusion of this section, none
of these newly proposed floating point representations are
satisfactory for DNNs. On the contrary, the Posit data format,
explained in the next section has promising properties.
III. THE POSIT REPRESENTATION
Posits are a by-product of the Type-II Unums numbers
described in previous section and they have been introduced
very recently in [25], again by Gustafson. The Posit represen-
tation is depicted in Figure 4. A Posit contains a maximum
of four fields: the sign field (1 bit), the regime field (variable-
length field), the exponent bit (fixed-length field which can
also be of zero bits), and the fraction field (variable-length
field, which can even be absent for some configurations). The
length of the exponent field is decided a-priori, together with
the total length for Posits. These two lengths characterize
different types of Posit representations. The length of the
regime field is determined using a run-length method: the
number of consecutive 0 after the sign bit and before the
first 1 bit is the regime length (a regime field can be also
made by a sequence of 1, until the first 0 is encountered:
in that case the number of consecutive 1 is the regime
0
1 2
3
4
5 6
7
8
9
10
11 12
13
14
15 16
17
18
19
20
21 22
23
24
25 26
27
28
29
30 31
S
Regime (1..re) Exponent (0..es)
Fraction (...)
Fig. 4. An example of Posit data type.
0
1 2
3
4
5 6
7
8
9
10
11 12
13
14
15
S R E F
0 0 0 0
1 1
0
1 1 1
0
1 1 1
0
1
0
1 2
3
4
5 6
7
8
9
10
11 12
13
14
15
S R E F
0
1 1
0 0
1 1
0 0 0
1
0
1
0 0 0
Fig. 5. Two examples of 16-bit Posit with 3 bits for exponent (es=3). In
the upper the numerical value is: +256
3
· 2
5
· (1 + 221/256) (221/256 is
the value of the fraction, 1 + 221/256 is the value of the mantissa). The final
value is therefore 1.907348 × 10
6
· (1 + 221/256) = 3.55393 × 10
6
.
In the lower the numerical value is: +256
+1
· 2
3
· (1 + 40/512) (40/512
is the value of the fraction, 1 + 40/512 is the value of the mantissa). The
final value is therefore 2048 · (1 + 40/512) = 2208.
length, but this time its value is negative). Once the length
of the regime is known, the length of the mantissa can be
determined, as the number of remaining bits (after skipping
the exponent bits). The formula that allows to retrieve the real
number is available [25] and two examples of its application
are shown in Figure 5. Please observe how the two Posits
representations shown in the figure have a different number
of bits reserved for the fraction field (8 and 9, respectively),
having different lengths for the regime fields (4 vs 3).
Similarly to Type-II Unums, Posits can be put on a circle
sharing the concept of projection of reals over a circle, but
different design decisions allow to implement Posit operation
without imposing the use of a look-up table. In Figure 6 the
(semi)-circles of 3-bit, 4-bit and 5-bit Posits are shown.
Posits enjoy many really interesting properties, such as:
unique representation for zero
no representations wasted for Not-A-Number (when
using Posits, an exception is raised instead of reserving
one or more representations for NaNs). Please notice
that the floating-point standard wastes a lot of repre-
sentations for NaNs, which also make the hardware for
comparing floats very complex.
Even more interestingly, Posits are sorted like signed
integers (when the latters are represented using the two’s
complement). Thus comparing two Posits can be done in
ALU (by type re-casting to signed integers): negative Posits
are expressed using complement two as integers, and the
other three fields allow direct ordering.
We think the brightest idea of the Posit representation is
to reserve more bits to the mantissa for small real numbers
(closer to zero) and less for large real numbers, within a
fixed length format (the total length is fixed, although the
length of the regime and that of the mantissa vary).
Finally observe that, since the Posit format standardization
process is still on its infancy, we are on time to provide useful
feedbacks to the standardization committee, providing them
suggestions based on the experience in their use with DNN.

Fig. 6. Posits semi-circles for different numbers of bits (3, 4 and 5).
A. Dimensioning Posits
As mentioned the Posit representation is parametrized over
the total number of bits to use and the number of them to
reserve to the exponent. Depending on the task at hand,
these two numbers can chosen accordingly. For instance,
as regards the DNN inference phase we know that 16-bit
floats are sufficient enough. Thus we know that 16-bit Posits
are sufficient too. However, how many bits to reserve to the
exponent part of the Posit can still be optimized on the task
at hand. In addition we observe that the Posit representation
is compatible with the flexpoint idea introduced in [15].
Indeed, it is possible to create Posits tensors that share
an extra exponent field. Thus, when dimensioning Posits
this extra exponent field must be dimensioned together
with the dimensioning of Posits. Finally we point out how
that stochastic rounding [26] could be integrated in Posit
arithmetic too (even if this adds an extra logic and thus
additional transistors).
IV. HARDWARE IMPLEMENTATION ISSUES
The implementation of a PPU will take advantage of the
operations over Posits that can be traced back to three main
levels of Posit representation: i) binary form, ii) decoded
form, and iii) expanded form. Indeed we associate to any
operation a Posit representation level. The first one is the
binary representation of Posits discussed so far that can be
used for some operations, with the effect of fastest speed
and minimal complexity. The second form decodes the Posit
in the three components of regime, exponent and fraction
by parsing the variable length encoding of the regime field.
Inversion, doubling and halving can be efficiently computed
by working on this intermediate representation, in addition
to the fundamental arithmetic operations when fraction is
zero. The third level requires a more sophisticated logic that,
anyway, in comparison to IEEE floating points, can take
advantage of the separation between regime and exponent
parts.
Dispatcher
ALU# PPU#
Memory
Store
DMA
Instruction Stream Decoder
Memory
Load
Memory Control
(Cache and Memory)
Unified Register File
(vector registers)
Execution Units
Fig. 7. Proposed superscalar architecture for a processor (or a co-processor)
that integrates the PPU. Thanks to register sharing and the properties of
Posit some operations can be performed using the ALU. Multiple ALU and
PPU and execution units are possible thanks to the parallel structure of the
superscalar architecture.
A. The advantage of using shared registers between ALU
and PPU
The property of Posit representation that allows to perform
comparisons using the integer comparison can be exploited
for a subset of DNN operations. We are referring on partic-
ular to the argmax operation at the end of the inference,
to the ReLu activation function, and to Max pooling in
convolutional neural networks (see next subsection). The
manipulation of a Posit number using integer operations or
bitwise operations justifies the proposal of adopting PPU as
an integrated part of a CPU, namely as an Execution Unit.
Some Posit operation could require custom instructions while
others (such as the comparison) could be obtained via integer
instructions. Figure 7 shows the schematic of registers within
the co-processor, to reuse the ALU for comparing Posits.
In the instructions’ opcode we must be able to specify
whether the 32-bit registry contains an operand of 8, 16, or 32
bits, being agnostic on its content (an address, an unsigned,
an integer or a posit). This approach, depicted in Figure 7, is
common to modern architecture (e.g. Intel Core CPUs) that
generalize this properties to larger register banks, up to 512
bits.
B. Specific DNN functions to implement in hardware
In existing custom hardware for Tensor operations (e.g.
Nvidia Tensor units and Google TPU) there are specific
operations that are structured in hardware due to their
repetitive nature. In the following we are considering some
fundamental DNN operations that can take advantage of
Posit representation and that can be directly executed. DNN
functions that are interesting to implement in hardware are
the following:

Fig. 8. The Sigmoid function and its approximation using Posits.
Sigmoid function:
1
e
x
+1
. This function is widely used
as activation function in DNNs. However, its evaluation
is time consuming, since it requires and exponentiation
and a division. As recently observed [25], this function
can be easily approximated in hardware by doing simple
bit-cloning and masking on the Posit representation.
Figure 8 shows how closely the approximated version
compares with the exact counterpart.
ReLu: max(0, x) (Rectified Linear Unit). This func-
tion is an appealing alternative to the exact Sigmoid
function, reducing the problem of vanishing gradient
while being faster. As such, it has been widely used in
DNNs. However, when using Posits, the approximated
sigmoid function discussed above can be used on its
place, since the two require comparable time (being
both executed in hardware). Thus computer scientist are
no longer forced to use a non-differentiable and less
expressive function, like the ReLu, and can continue
to use the more expressive Sigmoid function (although
approximated).
MaxPool: max pooling is a mathematical function
widely used in convolutional neural networks. MaxPool
can be thought as a max filter applied to an image block-
wise and thus it requires many comparisons between
real numbers. Having the possibility to compare two
Posits using the ALU we saving a significant of transis-
tors (and thus we save energy consumption at run-time)
by not having to implement the comparison operators
within the PPU.
dot product:
P
i
a
i
· b
i
. This function is crucial in
multi-layer perceptrons DNNs, since the dot product
between two vectors of real numbers a and b is routinely
computed between the inputs (or each intermediate
representation) and the layer weights. In addition, the
standard (row-by-column) matrix by vector product is
nothing more that a series of dot products, and it is
the basic operation in convolutional DNNs. The dot
product of two real vectors represented using the Posit
data format needs to be carefully designed in hardware.
This is a special function that falls within the category of
fused operations. The key observation is that, although
the final result of the dot product fits a given number
of bits, the intermediate term may requires many more,
in order to not loose accuracy [27]. First versions of
the IEEE standard for floats (such as the 754 of 1985)
did not specify the number of bits to use for fused
TABLE I
DIFFERENT SOLUTIONS TO COMPUTE WITH POSITS
Posits up to 14 bits Larger Posits
Solution A
Hardware PPU Hardware PPU
Solution B
Look-up table on
ROM/RAM plus
some ALU
Emulated PPU using ALU
only (not the FPU), by in-
cluding the developed C++
Posit class
operations. Only in the last revision of the standard (the
2008 version) it has been standardized.
C. PPU emulation library
In this sub-section we report our experience with C++ im-
plementation of Posits on an emulation platform represented
by a GPP processor, e.g. an Intel Core i7. Posits library has
been implemented using a C++ template library parametrized
over Posit length and exponent length. The implementation
takes advantage of the representation levels discussed in
Section IV. Integer ALU are used for the level 3 operations
much like the SoftFloat [28] does with IEEE floating points.
Moreover level 3 operations of small Posits (e.g. less than 14
bits) can be implemented in the library using lookup tables.
The use of a look-up table is interesting for those applications
requiring a low precision and for those computers having
sufficient cache memory, like current desktop PCs. Table I
summarizes the different implementation options. Thanks to
the library, computations with real numbers can be performed
using a fully software implementation of Posits, being able
to save memory and bandwidth. In particular the library can
be used for working with the C++ Eigen template matrices
library [29].
D. Summary: the advantages of Posits over Floats
When performing the inference of a trained DNN in an
autonomous driving application the advantages are:
1) More efficient use of bits (less bit/higher accuracy)
2) 16-bit Posits are more accurate than 16-bit Floats
3) The PPU within the processor/co-processor requires
less transistors, because the comparison operator does
not need to be implemented (use the one of the ALU)
4) The Sigmoid function can be easily computed in
hardware, due to a numerical property of Posits.
More in general, we confirm that Posits are an interesting
data format for low-precision arithmetic. In particular, using
Posit16 we are widening the range of applications for which
16 bits are enough, thus saving bandwidth, storage and
energy consumption in comparison to Float32.
V. CONCLUSIONS AND FUTURE WORK
To accelerate DNN computing in automotive applications,
the paper has discussed the introduction of PPU as alternative
to classic FPU. Once introduced the Posits as alternative
representation for real numbers, we showed that 16-bit Posits
should replace conventional FPUs, since they are more

Citations
More filters

Proceedings ArticleDOI
25 Mar 2019
Abstract: The recent surge of interest in Deep Neural Networks (DNNs) has led to increasingly complex networks that tax computational and memory resources. Many DNNs presently use 16-bit or 32-bit floating point operations. Significant performance and power gains can be obtained when DNN accelerators support low-precision numerical formats. Despite considerable research, there is still a knowledge gap on how low-precision operations can be realized for both DNN training and inference. In this work, we propose a DNN architecture, Deep Positron, with posit numerical format operating successfully at ≤8 bits for inference. We propose a precision-adaptable FPGA soft core for exact multiply-and-accumulate for uniform comparison across three numerical formats, fixed, floating-point and posit. Preliminary results demonstrate that 8-bit posit has better accuracy than 8-bit fixed or floating-point for three different low-dimensional datasets. Moreover, the accuracy is comparable to 32-bit floating-point on a Xilinx Virtex-7 FPGA device. The trade-offs between DNN performance and hardware resources, i.e. latency, power, and resource utilization, show that posit outperforms in accuracy and latency at 8-bit and below.

52 citations


Proceedings ArticleDOI
13 Mar 2019
TL;DR: A unified analysis quantifies the trade-offs between overall network efficiency and performance across five classification tasks and indicates that posits are a natural fit for DNN inference, outperforming at ≤8-bit precision and can be realized with competitive resource requirements relative to those of floating point.
Abstract: Deep neural networks (DNNs) have been demonstrated as effective prognostic models across various domains, e.g. natural language processing, computer vision, and genomics. However, modern-day DNNs demand high compute and memory storage for executing any reasonably complex task. To optimize the inference time and alleviate the power consumption of these networks, DNN accelerators with low-precision representations of data and DNN parameters are being actively studied. An interesting research question is in how low-precision networks can be ported to edge-devices with similar performance as high-precision networks. In this work, we employ the fixed-point, floating point, and posit numerical formats at ≤8-bit precision within a DNN accelerator, Deep Positron, with exact multiply-and-accumulate (EMAC) units for inference. A unified analysis quantifies the trade-offs between overall network efficiency and performance across five classification tasks. Our results indicate that posits are a natural fit for DNN inference, outperforming at ≤8-bit precision, and can be realized with competitive resource requirements relative to those of floating point.

28 citations


Cites background from "Exploiting Posit Arithmetic for Dee..."

  • ...review the effectiveness of posits for autonomous driving functions [6]....

    [...]


Posted Content
TL;DR: This work first describes an algorithm for multiplying two posit numbers, even when the number of exponent bits is zero, which is particularly interesting because it allows the deployment of a fast sigmoid function.
Abstract: The posit number system is arguably the most promising and discussed topic in Arithmetic nowadays. The recent breakthroughs claimed by the format proposed by John L. Gustafson have put posits in the spotlight. In this work, we first describe an algorithm for multiplying two posit numbers, even when the number of exponent bits is zero. This configuration, scarcely tackled in literature, is particularly interesting because it allows the deployment of a fast sigmoid function. The proposed multiplication algorithm is then integrated as a template into the well-known FloPoCo framework. Synthesis results are shown to compare with the floating point multiplication offered by FloPoCo as well. Second, the performance of posits is studied in the scenario of Neural Networks in both training and inference stages. To the best of our knowledge, this is the first time that training is done with posit format, achieving promising results for a binary classification problem even with reduced posit configurations. In the inference stage, 8-bit posits are as good as floating point when dealing with the MNIST dataset, but lose some accuracy with CIFAR-10.

11 citations


Cites methods from "Exploiting Posit Arithmetic for Dee..."

  • ...Performing low-precision inference can be extremely helpful in embedded systems and applications that make use of DL techniques such as Autonomous Driving [37]....

    [...]


Journal ArticleDOI
10 Mar 2020-Sensors
TL;DR: This paper proposes a new class of Posit operators called L1 operators, which consists of fast and approximated versions of existing arithmetic operations or functions only using integer arithmetic, and points out how small Posits are very interesting while PPUs become widespread.
Abstract: With increasing real-time constraints being put on the use of Deep Neural Networks (DNNs) by real-time scenarios, there is the need to review information representation. A very challenging path is to employ an encoding that allows a fast processing and hardware-friendly representation of information. Among the proposed alternatives to the IEEE 754 standard regarding floating point representation of real numbers, the recently introduced Posit format has been theoretically proven to be really promising in satisfying the mentioned requirements. However, with the absence of proper hardware support for this novel type, this evaluation can be conducted only through a software emulation. While waiting for the widespread availability of the Posit Processing Units (the equivalent of the Floating Point Unit (FPU)), we can already exploit the Posit representation and the currently available Arithmetic-Logic Unit (ALU) to speed up DNNs by manipulating the low-level bit string representations of Posits. As a first step, in this paper, we present new arithmetic properties of the Posit number system with a focus on the configuration with 0 exponent bits. In particular, we propose a new class of Posit operators called L1 operators, which consists of fast and approximated versions of existing arithmetic operations or functions (e.g., hyperbolic tangent (TANH) and extended linear unit (ELU)) only using integer arithmetic. These operators introduce very interesting properties and results: (i) faster evaluation than the exact counterpart with a negligible accuracy degradation; (ii) an efficient ALU emulation of a number of Posits operations; and (iii) the possibility to vectorize operations in Posits, using existing ALU vectorized operations (such as the scalable vector extension of ARM CPUs or advanced vector extensions on Intel CPUs). As a second step, we test the proposed activation function on Posit-based DNNs, showing how 16-bit down to 10-bit Posits represent an exact replacement for 32-bit floats while 8-bit Posits could be an interesting alternative to 32-bit floats since their performances are a bit lower but their high speed and low storage properties are very appealing (leading to a lower bandwidth demand and more cache-friendly code). Finally, we point out how small Posits (i.e., up to 14 bits long) are very interesting while PPUs become widespread, since Posit operations can be tabulated in a very efficient way (see details in the text).

10 citations


Book ChapterDOI
11 Sep 2019
TL;DR: This paper presents a fast approximation of another activation function widely used in DNNs: thehyperbolic tangent, and shows how the approximated hyperbolic function outperforms the approximating sigmoid counterpart.
Abstract: Deep Neural Networks (DNNs) are being used in more and more fields. Among the others, automotive is a field where deep neural networks are being exploited the most. An important aspect to be considered is the real-time constraint that this kind of applications put on neural network architectures. This poses the need for fast and hardware-friendly information representation. The recently proposed Posit format has been proved to be extremely efficient as a low-bit replacement of traditional floats. Its format has already allowed to construct a fast approximation of the sigmoid function, an activation function frequently used in DNNs. In this paper we present a fast approximation of another activation function widely used in DNNs: the hyperbolic tangent. In the experiment, we show how the approximated hyperbolic function outperforms the approximated sigmoid counterpart. The implication is clear: the posit format shows itself to be again DNN friendly, with important outcomes.

10 citations


References
More filters

Journal ArticleDOI
28 May 2015-Nature
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

33,931 citations


Posted Content
TL;DR: The results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy.
Abstract: Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network's behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

1,233 citations


"Exploiting Posit Arithmetic for Dee..." refers background in this paper

  • ...that stochastic rounding [26] could be integrated in Posit arithmetic too (even if this adds an extra logic and thus additional transistors)....

    [...]


Proceedings Article
06 Jul 2015
Abstract: Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of lowprecision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network's behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

861 citations


01 Jan 2011
TL;DR: This paper uses speech recognition as an example task, and shows that a real-time hybrid hidden Markov model / neural network (HMM/NN) large vocabulary system can be built with a 10× speedup over an unoptimized baseline and a 4× speed up over an aggressively optimized floating-point baseline at no cost in accuracy.
Abstract: Recent advances in deep learning have made the use of large, deep neural networks with tens of millions of parameters suitable for a number of applications that require real-time processing. The sheer size of these networks can represent a challenging computational burden, even for modern CPUs. For this reason, GPUs are routinely used instead to train and run such networks. This paper is a tutorial for students and researchers on some of the techniques that can be used to reduce this computational cost considerably on modern x86 CPUs. We emphasize data layout, batching of the computation, the use of SSE2 instructions, and particularly leverage SSSE3 and SSE4 fixed-point instructions which provide a 3× improvement over an optimized floating-point baseline. We use speech recognition as an example task, and show that a real-time hybrid hidden Markov model / neural network (HMM/NN) large vocabulary system can be built with a 10× speedup over an unoptimized baseline and a 4× speedup over an aggressively optimized floating-point baseline at no cost in accuracy. The techniques described extend readily to neural network training and provide an effective alternative to the use of specialized hardware.

798 citations


Proceedings Article
13 Feb 2018
Abstract: Researches on deep neural networks with discrete parameters and their deployment in embedded systems have been active and promising topics. Although previous works have successfully reduced precision in inference, transferring both training and inference processes to low-bitwidth integers has not been demonstrated simultaneously. In this work, we develop a new method termed as "WAGE" to discretize both training and inference, where weights (W), activations (A), gradients (G) and errors (E) among layers are shifted and linearly constrained to low-bitwidth integers. To perform pure discrete dataflow for fixed-point devices, we further replace batch normalization by a constant scaling layer and simplify other components that are arduous for integer implementation. Improved accuracies can be obtained on multiple datasets, which indicates that WAGE somehow acts as a type of regularization. Empirically, we demonstrate the potential to deploy training in hardware systems such as integer-based deep learning accelerators and neuromorphic chips with comparable accuracy and higher energy efficiency, which is crucial to future AI applications in variable scenarios with transfer and continual learning demands.

232 citations


Frequently Asked Questions (2)
Q1. What have the authors contributed in "Exploiting posit arithmetic for deep neural networks in autonomous driving applications" ?

This paper discusses the introduction of an integrated Posit Processing Unit ( PPU ) as an alternative to Floating-point Processing Unit ( FPU ) for Deep Neural Networks ( DNNs ) in automotive applications. In this work, the authors review an alternative arithmetic that could be used within the co-processor. The authors argue that a new representation for floating point numbers called Posit is particularly advantageous, allowing for a better trade-off between computation accuracy and implementation complexity. The authors conclude that implementing a PPU within the co-processor is a promising way to speed up the DNN inference phase. 

As further evolution of the work the authors are analyzing whether the use of Posits can be helpful even during the training phase of DNNs. Such servers can be equipped with co-processors as well.