10 August 2022
POLITECNICO DI TORINO
Repository ISTITUZIONALE
An Error-Detection and Self-Repairing Method for Dynamically and Partially Reconfigurable Systems / SONZA
REORDA, Matteo; Sterpone, Luca; Ullah, Anees. - In: IEEE TRANSACTIONS ON COMPUTERS. - ISSN 0018-9340. -
ELETTRONICO. - 66:6(2017), pp. 1022-1033. [10.1109/TC.2016.2607749]
Original
An Error-Detection and Self-Repairing Method for Dynamically and Partially Reconfigurable Systems
IEEE postprint/Author's Accepted Manuscript
Publisher:
Published
DOI:10.1109/TC.2016.2607749
Terms of use:
openAccess
Publisher copyright
©2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any
current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating
new collecting works, for resale or lists, or reuse of any copyrighted component of this work in other works.
(Article begins on next page)
This article is made available under terms and conditions as specified in the corresponding bibliographic description in
the repository
Availability:
This version is available at: 11583/2658319 since: 2016-11-30T14:11:58Z
IEEE
0018-9340 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TC.2016.2607749, IEEE
Transactions on Computers
2 IEEE TRANSACTIONS ON COMPUTERS
An Error-Detection and Self-Repairing Method for
Dynamically and Partially Reconfigurable Systems
Abstract— Reconfigurable systems are gaining an increasing interest in the domain of safety-critical applications, for example
in the space and avionic domains. In fact, the capability of reconfiguring the system during run-time execution and the high
computational power of modern Field Programmable Gate Arrays (FPGAs) make these devices suitable for intensive data
processing tasks. Moreover, such systems must also guarantee the abilities of self-awareness, self-diagnosis and self-repair in
order to cope with errors due to the harsh conditions typically existing in some environments. In this paper we propose a self-
repairing method for partially and dynamically reconfigurable systems applied at a fine-grain granularity level. Our method is
able to detect, correct and recover errors using the run-time capabilities offered by modern SRAM-based FPGAs. Fault injection
campaigns have been executed on a dynamically reconfigurable system embedding a number of benchmark circuits.
Experimental results demonstrate that our method achieves full detection of single and multiple errors, while significantly
improving the system availability with respect to traditional error detection and correction methods.
Index Terms— Self-Repair; Partial and Dynamic Reconfiguration; Single Event Upsets (SEUs); Multiple Event Upsets (MEUs)
——————————
——————————
1 INTRODUCTION
ECHNOLOGY scaling in the nano-metric domain and
beyond supports the increasing usage of high perfor-
mance and miniaturized embedded systems. Howev-
er, the quest for pushing the limits of technology to the
ultra-nano scale devices has exacerbated concerns related
to power consumption and reliability that have not be
envisioned before. In particular, one of the major issues in
safety-critical applications (especially in the space and
avionic domains) is the run-time mitigation of various
radiation-induced fault effects, which may provoke tran-
sient and permanent modifications of the electronic cir-
cuit’s behavior. The problem is widely known and vari-
ous methods have been developed and proposed in the
area during the last decade. The ubiquity of embedded
systems for safety-critical applications operating in radia-
tion environments demands continuous and successful
operations of the system by autonomously overcoming
possible malfunctions. This condition requires the abili-
ties of autonomous error detection, self-diagnosis and
self-repair [1].
Among the available technology solutions, the adop-
tion of SRAM-based FPGAs is the most suitable for the
realization of dynamically and partially reconfigurable
systems; however, when used in harsh environments,
SRAM-based FPGAs have to withstand the radiation ef-
fects in the form of Single Event Upsets (SEUs) and Mul-
tiple Event Upsets (MEUs), especially affecting their con-
figuration memory [2].
The increased probability of MEUs hitting the config-
uration memory of an FPGA can limit the effectiveness of
traditional redundancy-based fault-tolerance approaches
[3]. In fact, particles can hit the same logic group of circuit
replicas enabling erroneous results to propagate. To cope
with this scenario, researchers have recently investigated
the fine-grain redundancy and its resilience to MEUs
[4][5][6]. However, for proper shielding against high fail-
ure rate while minimizing redundancy overhead in terms
of area, speed and power consumption, systems should
be designed with accurate mixed-grain redundancy and
self-repair properties which are not feasible for fine-grain
redundancy.
State-of-the-art SRAM-based FPGAs have the technol-
ogy supporting run-time dynamic and partial reconfigu-
ration (DPR), which can be used for adaptive behavior as
well as for fault repairing [7]. A self-repairing system
adopting the partial dynamic reconfiguration capabilities
of SRAM-based FPGAs is often divided in two parts,
called static region and dynamic region. The logic and rout-
ing resources and the corresponding configuration
memory frames individuated by means of clock regions
and major and minor columns, illustrated in figure 1, are
organized in a static region, also called base region. The
static region typically consists of a microprocessor, some
memory modules and input/output ports, as described in
figure 2. In general, these components are not re-
configured and their full functionality is constantly re-
quired for implementing the correct operations of the
system; for this reason the static region is often hardened
using a traditional redundancy-based approach, such as
Triple Modular Redundancy (TMR) [7]. The static region
is also responsible for the reconfiguration of the modules
placed into the reconfigurable regions. On the contrary,
the components in the dynamic region correspond to par-
tially reconfigurable resources that can be configured in
different ways depending on the system requirements [8].
The dynamically reconfigurable regions idea is extended
in this paper so that the system is able to also correct the
identified errors by applying internal reconfiguration.
M. Sonza Reorda, Fellow, IEEE, L. Sterpone, Member, IEEE,
A. Ullah, Student Member, IEEE
xxxx-xxxx/0x/$xx.00 © 200x IEEE Published by the IEEE Computer Society
T
————————————————
M. Sonza Reorda, L. Sterpone and A. Ullah are with the Dipartimen-
to di Automatica e Informatica (DAUIN), Politecnico di Torino, To-
rino, Italy. For any information please refer to:
luca.ster
p
one@
p
olito.it (contact author).
0018-9340 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TC.2016.2607749, IEEE
Transactions on Computers
SONZA ET AL.: AN ERROR-DETECTION AN SELF-REPAIRING METHOD FOR DYNAMICALLY AND PARTIALLY RECONFIGURABLE SYSTEMS 3
The proposed approach provides significant advantages
compared to already developed solutions [9][10], mainly
because it increases the error detection and correction
capabilities while introducing comparable area and per-
formance overhead.
In order to practically prove the effectiveness of the
approach, we developed a complete set of tools for the
automatic generation of the constraints used for the parti-
tioning of the dynamic regions. The developed set of tools
directly acts at the physical level, automatically inserting
a carry chain into the physical net-list and adding com-
parator check flags into the circuitry; moreover, the tool is
able to cleverly place the different partitions of the dy-
namic region into proper sub-regions, thus allowing SEUs
and MEUs correction. The proposed approach drastically
improves the solution in [9] which uses the built-in slice
carry chain for error detection, only.
Our approach introduces a minimal area overhead,
which is strictly dependent upon the number of user-
defined partitions. On the average, the overhead intro-
duced by our approach is around 11% with respect to the
duplication-based approach; hence, the proposed tech-
nique is using far less computational resources if com-
pared to the standard TMR solution. Furthermore, correc-
tion is performed on a single reconfigurable frame, which
is the smallest amount of reconfigurable information that
can be read or written; therefore, we can achieve the
highest availability limits offered by the current reconfig-
urable technology.
The paper is organized as follows. Section 2 gives an
overview of the soft error detection and correction meth-
ods implemented with modern SRAM-based FPGAs and
summarizes the major contributions of this paper. Section
3 describes the proposed method, while the developed
design flow is illustrated in Section 4. Experimental re-
sults on the selected case study and their analysis are pre-
sented in Section 5. Finally, conclusions and future works
are described in Section 6.
2 PREVIOUS WORKS
State-of-the-art SRAM-based FPGAs are heterogene-
ous devices containing several macro blocks, like Digital
Signal Processors (DSPs), Block RAMS (BRAMs) and IO
Blocks (IOBs), along with Configurable Logic Blocks
(CLBs) inside the FPGA reconfiguration fabric. Each of
these resource types is arranged in columns that span
from top to bottom of the device realizing a column of
CLBs, IOBs and BRAM memories interconnected by a
mesh of heterogeneous routing resources. Each SRAM-
based FPGA chip is organized in a number of rows de-
pendent on the manufacturer families or specific part. The
most advanced devices have CLB rows connected to
global resources as well as local clock sources [11]. In or-
der to harden circuits implemented on SRAM-based
FPGAs, different architecture level techniques have been
proposed in the past. However, we can broadly classify
them into two main techniques, namely fault masking and
fault correction. In the next part of this section we will pre-
sent a detailed discussion of the previous research work
in each category.
In the recent years, two different mitigation approach-
es have been proposed to mitigate SEUs affecting the con-
figuration memory of SRAM-based FPGAs. On one side,
full hardware redundancy obtained thanks to Triple
Modular Redundancy (TMR) is used to identify and cor-
rect logic values. This solution presents a large overhead
in terms of area, power and especially delay, since it trip-
licates all the combinational and sequential logic, and the
architecture introduces delay penalties for the voter
propagation time and the routing congestion. On the oth-
er side, redundancy approaches are nowadays combined
with scrubbing, that consists in periodically reloading the
complete content of the FPGA’s configuration memory. A
more complex system is used to correct the information in
the configuration memory by using read-back and partial
configuration procedures. Through the read-back process
the content of the FPGA’s configuration memory is read
and compared with the expected value, which is stored in
a dedicated memory located outside of the FPGA. With
the advent of modern SRAM-based FPGAs this operation
may be performed through dynamic reconfiguration. In
details, with dynamic reconfiguration, the FPGA configu-
ration memory can be read-back continuously without
interfering with the circuit functionality and if any upset
is detected it can be selectively re-written with the correct
values, thus avoiding the accumulation of radiation-
induced errors [12]. However, the main drawback of this
technique is the huge detection and correction time that
makes it useless for real-time operations and ineffective
versus the single point of failure induced by configura-
tion memory bit-flips. Spatial redundancy using Triple
Modular Redundancy (TMR) is complementarily used
with the read-back and correction techniques: on one side
TMR can tolerate faults with the limitation of withstand-
ing a single fault per voting group [13], on the other side
read-back and correction avoids the accumulation of er-
rors within the configuration memory. The combination
of TMR and self-healing using dynamic partial reconfigu-
ration has been previously used in [14][15]. However, the
results achievable with this combined solution are com-
putationally expensive and area hungry.
Reconfiguration at the gate level is used in fine-grain
approaches [16] with particular efficiency from the point
of view of the area overhead, although it suffers from a
complex and not flexible control mechanism. Further-
more, because of the adopted fine granularity, this ap-
proach is infeasible for system-level healing. A self-
healing partial dynamic reconfigurable design methodol-
ogy has been proposed in [17]. However, the method in-
serts control circuitry by partitioning the circuit for error
• Minor Column
• Rows (Clock
Regions)
• Major Column
Frame 0
Frame 35
Frame 1
Frame 25
CLB Major Column
20 CLB High
Fig. 1. Resource and Frame Layout of Modern SRAM based FPGAs
0018-9340 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TC.2016.2607749, IEEE
Transactions on Computers
4 IEEE TRANSACTIONS ON COMPUTERS
localization and detection purposes. This requires a sig-
nificant overhead.
A methodology for fault tolerant architectures using
on-line checkers for fault detection and localization was
introduced in [18]. On-line checkers for TMR- and dupli-
cation-based systems were combined with partial dynam-
ic reconfiguration in [19] in such a way that detection and
localization of faults will be performed by the checker,
while reconfiguration will recover from the error. The
detection and localization of errors is implemented as a
partial reconfigurable module which is itself subject to
errors. A previous approach based on fine-granularity
error masking has been developed in [20]; however, such
solution can only be applied to a TMR technique with a
majority voter logic scheme. Vice versa, a first overview
of recovery architectures for high computational systems
based on SRAM-based FPGAs has been presented in [21].
2.1 Main contribution
The main contribution of the present work, which is
based on the platform preliminarily presented in [8], is
the description of an autonomous recovery approach that
can be applied to Partially Reconfigurable Modules
(PRMs) when errors are detected inside them. The ap-
proach is implemented by the static region providing ef-
fective capabilities of error detection and correction of
faults within the dynamic region. Our approach allows
resilience to MEUs, since we adopt a static region protect-
ed with a fine-grain redundancy approach, as described
by [3]. In particular, we propose a new fine-grain fault
detection mechanism applied to FPGA resources: the ap-
proach is based on the comparison of Look-Up Tables
(LUTs) outputs by using the logic available to allow carry
propagation, which is generally used for fast arithmetic
computations and mostly not inferred by design tools,
following the approach preliminarily introduced in [9] for
fault detection. In details, the proposed method is charac-
terized by the ability of detecting MEUs into the FPGA’s
configuration memory, as well as to recover any number
of faults in the dynamic partition, thus improving previ-
ously developed approaches, as presented in [9], that
cannot deal with MEUs. Our solution is adaptable to all
modern SRAM-based FPGAs equipped with an Internal
Configuration Access Port (ICAP) and based on a LUT-
slice architecture.
3 THE PROPOSED METHOD
The proposed method consists of two flows: one applied
to the dynamically reconfigurable region for implement-
ing error detection, the other one for instrumenting the
circuit mapped on the FPGA so that it supports the execu-
tion of the self-repairing method against single and mul-
tiple-bit errors.
A dynamically reconfigurable system, from the architec-
tural perspective, is partitioned into static and dynamic
regions as illustrated in figure 2. The static region consists
of a processor with a static-RAM, some general purpose
IOs, flash memories, and hardware resources for manag-
ing the internal configuration access port connected to the
processor local bus. The static region contains the main
processor, which is in charge of controlling the partially
reconfigurable system operational functionalities: there-
fore, it is very important to tolerate and recover errors in
these modules. In this paper we assume that this region is
implemented using Triple Modular Redundancy. By suit-
ably mapping the three copies of the circuit elements on
the device the static region can be protected against any
single point of failure.
The dynamic region consists of the resources imple-
menting the user’s circuit. The proposed approach mainly
focuses on the dynamic region, and exploits reconfigura-
tion at the individual frame level for error detection and
correction. The dynamic region can be organized into a
Single Bit Error (SBE) region, Multi Bit Error (MBE) re-
gion and Coarse-Grain Error region. It is first necessary to
introduce some definitions related to the major character-
istics of current SRAM-based FPGAs: modern FPGAs are
row-wise divided into a number of clock regions for dy-
namic partial reconfiguration, while column-wise are or-
ganized in major columns of resources, such as CLBs,
DSPs or IOs. Each major column spans the whole height
of the device but it is configured in each clock region
(row) by a separate reconfigurable frame (RF). Each RF
contains a different number of “minor frames”, each hav-
ing a height equal to the clock region (row) and num-
bered from left to right. For example, in Xilinx Virtex-5
devices [11] a CLB RF consists of 36 minor frames (hereby
simply referred to as frames), which are responsible for
the configuration of LUTs and their routing, while con-
figuration bits for a single LUT are distributed over mul-
tiple frames. From the point of view of the circuit archi-
tecture, the proposed method is based on the Duplication
With Comparison (DWC) technique applied at two dif-
ferent levels of granularity, herein called Coarse-grained
DWC (C-DWC) and Fine-grained DWC (F-DWC).
The C-DWC is applied for slices that use the carry
chain for computations such as fast additions or multipli-
cations. In this case, the duplication is performed at the
module level and the outputs are compared at the physi-
cal level by LUT elements configured to implement XOR
combinational functions. Our approach is able to directly
modify the circuit physical description in order to use the
XOR logic function to compare the module’s outputs. In
case of error, the software tools running on the reconfigu-
rable system partially rewrite the C-DWC region.
F-DWC is applied at the place and route level, by suitably
duplicating each LUT function in two copies that are
placed in a single slice using two consecutive LUT posi-
tions. The outputs of the two LUTs are then compared
with hardwired physical resources built into the slice in
!
!
"
"
#
$
Fig. 2. Placement space division into Single Bit Error region, Multipl
e
Bit Error region and the Coarse Grain Error region.
0018-9340 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TC.2016.2607749, IEEE
Transactions on Computers
SONZA ET AL.: AN ERROR-DETECTION AN SELF-REPAIRING METHOD FOR DYNAMICALLY AND PARTIALLY RECONFIGURABLE SYSTEMS 5
the form of a carry chain by using internal and not pro-
grammable resources, such as hardwired MUXCYs and
XORCYs.
The outputs generated by the XORCY functions are con-
nected in a chain of OR logic functions in order to provide
a single error detection flag for each column. Practically,
the F-DWC approach can be adopted by acting at the
Hardware Design Language (HDL) level: the combina-
tional functions are duplicated and both copies of the cir-
cuit LUTs are placed in a single FPGA slice using two
consecutive available LUT positions. Please note that the
outputs of any pair of LUTs pass through the carry chain
and at each pair position of the XORCY generate a com-
parison signal called check flag. Since we are generating a
check flag for each pair of LUTs the number of check flags
may drastically increase. This means that a considerable
amount of routing resources could be required by the
implementation of these check flags because they have to
be routed to the static region for the detection and correc-
tion of possible errors. Moreover, any such scheme will
not only have a large overhead, but it will also be fruitless
because the smallest unit of reconfiguration is a frame.
In order to have a single check flag for each frame we
propose to merge the individual check flags in two differ-
ent ways. The check flags in the SBE region are merged
through the built-in slice carry chain as shown in figure 3
(further details will be provided in section 3.1.1) where
the hardwired resources XORCY and MUXCY are labeled
as Xi and Mi, respectively. Furthermore, a whole column
of slices is connected through the carry chains to produce
a single flag for each column of slices (see for example flag
3 in the SBE region of figure 2).
In this way we achieve a huge reduction in the num-
ber of check flags, but we can only detect Single Event
Upsets (SEUs) in the SBE region, because multiple LUT
pairs are connected together by a long chain of XORs and
XNORs and thus an even number of errors will go unde-
tected due to the logic configuration of the detector.
In the Multiple Bit Error (MBE) region each pair of
LUTs generates a check flag and thus we have two check
flags per slice. The number of check flags can be reduced
by OR-ing some of the flags corresponding to the slices in
the same slice column, as shown in the magnified MBE
region in figure 4. Although some higher overhead is in-
troduced in this way, we have the ability to detect Multi-
ple Event Upsets (MEUs) in the frames mapped on this
region; in fact, the individual check flags are not merged
along the carry chain passing through multiple XORs, as
it happened in the SBE region.
3.1 Error Detection Method
In order to fully explain our proposal, in this section
we will specifically refer to the architecture of Xilinx Vir-
tex-5 FPGAs. As described in the previous section, the
error detection mechanism implemented in the reconfigu-
rable region is based on LUT-based checkers and carry
chains for propagating the check flags. Please note that
the LUT checkers are only deployed when the carry chain
is unavailable for comparison purposes. This allows re-
ducing the performance degradation of the circuit im-
plemented with our method, although in this case the
detection mechanism is implemented at the modular lev-
el. In this section, we focus on the method adopted for the
error detection using the carry chains for comparison; a
more detailed explanation of both the LUT checkers and
the carry chains insertion inside the physical place and
route description of the circuit will be given in Section 4.3.
3.1.1 Single-bit error detection
In order to detect single-bit errors, we propose to du-
plicate each original LUT function into two identical
LUTs. Furthermore, we place the two LUTs in a single
FPGA slice, where we set the Carry Input and the generic
AX inputs to 1 and 0, respectively, as illustrated in figure
3. Consequently, the hardwired XORCY logic gate in the
bottom of the slice is acting as an inverter, while the
MUXCY multiplexer in the bottom first position is simply
acting as a buffer to pass the value of LUT A.
The multiplexer “M2” receives an inverted (through
the AMUX_2_BX hardwired connection) and buffered
copy of the LUT A output at its “0” and “1” inputs while
the selection line is tied to LUT B (which is the copy of
LUT A) thus effectively performing the EX-NOR function.
The XOR gate named “X2” receives LUT A and LUT B
outputs on its inputs. Similarly, LUT C and LUT D can
also be connected with such a scheme by extending the
EX-NORs and EX-ORs along the slice. In fact, this scheme
can be extended to an entire clock region covering 20
CLBs using the COUT and CIN of slices, thus generating
two flags for the even and odd slice columns of the same
CLB, respectively. This convergence strategy can only be
applied if the CLB column has no empty slices. In case the
CLB column contains empty slices the dedicated COUT
connection cannot be used to propagate the flag signal
upwards along the column. For such a case, an ORing
LUT is introduced in the CLB column and placed in an
available empty slice. This will be discussed in greater
details in Section 4.3. It is interesting to investigate an
upper bound on the number of check flags that can be
generated for the most complex design. The flag signal is
generated per CLB tile columns and is directly related to
the device rows and columns. For example, for the Virtex-
5 VLX110T device the maximum number of check flags
for any design cannot be greater than 1,280 (160x8) [22]
[23]. As the FPGA must contain the control processor the
actual number will be quite less than 1,280 and will de-
termine the size of the GPIO port that is used by the con-
troller to detect errors. Then, it is possible to pinpoint sin-
gle bit upsets in any of the four LUTs in any slice column
in a clock region. However, errors affecting flip-flops
Fig. 3. Single-Bit Error detection scheme implemented in a single
slice.