Ethernet-Based Real-Time and Industrial
Communications
JEAN-DOMINIQUE DECOTIGNIE, FELLOW, IEEE
Invited Paper
Despite early attempts to use Ethernet in the industrial context,
only recently has it attracted a lot of attention as a support for in-
dustrial communication. A number of vendors are offering indus-
trial communication products based on Ethernet and TCP/IP as a
means to interconnect field devices to the first level of automation.
Others restrict their offer to communication between automation
devices such as programmable logic controllers and provide inte-
gration means to existing fieldbuses. This paper first details the re-
quirements that an industrial network has to fulfill. It then shows
how Ethernet has been enhanced to comply with the real-time re-
quirements in particular in the industrial context. Finally, we show
how the requirements that cannot be fulfilled at layer 2 of the OSI
model can be addressed in the higher layers adding functionality to
existing standard protocols.
Keywords—Ethernet, fieldbuses, industrial communications,
real time.
I. INTRODUCTION
Ethernet is now the dominant local area networking
solution in the home and office environment. It is fast, easy
to install and the interface ICs are cheap. Most computerized
equipments now come with a built-in Ethernet interface.
These are some of the reasons why a number of manu-
facturers of industrial control systems are now advocating
the use of Ethernet, with various modifications, to support
real-time communications at the factory floor. These pro-
posals are often referred as “Industrial Ethernet” even if they
correspond to different and most of the time incompatible
solutions. The objective of this paper is first to explain why
Ethernet is not considered as a suitable solution to support
industrial communications and then to review and analyze
the different additions to Ethernet to achieve an adequate
solution.
Despite early attempts to use Ethernet as a real-time com-
munication vehicle in the factories [1], [2], practitioners were
Manuscript received March 30, 2004; revised January 31, 2005.
The author is with the Swiss Center for Electronics and Microtechnology
(CSEM), 2007 Neuchâtel, Switzerland (e-mail: decotignie@ieee.org.).
Digital Object Identifier 10.1109/JPROC.2005.849721
reluctant to adopt this technology because of its intrinsic non-
determinism. Let us assume that two stations are waiting to
transmit a message while the medium is used by another sta-
tion. As soon as that station finishes transmitting, one the two
stations will succeed in transmitting its waiting message. We
cannot know in advance which station because the choice is
random. For this reason, it is not possible to give an upper
bound to the time required to transmit a message from one
station to another. It is only possible to assess the probability
that this time will not exceed a given value. This random as-
pect was rejected because the users wanted real-time guar-
antees (maximum transfer delay, jitter in the transmissions,
and available bandwidth). These early years has seen the de-
velopment of a variety of protocols providing deterministic
behavior such as Token Bus, Token Ring, and a number of
fieldbuses or control networks such as WorldFIP [3], Profibus
[4], P Net [5], Interbus [6], AS-Interface [7], SERCOS [8],
LonWorks [9], MVB [10], MIL-STD 1553 [11], DeviceNet
[12], SDS [13], or CAN [14].
Most of those solutions were developed two decades ago
and are now considered as too limited when compared to the
available performances (mainly in terms of throughput) of
non-real-time networks such as Ethernet or ATM. The re-
quirements have also evolved toward the transmission of a
higher quantity of information. There is hence a need to up-
grade the existing systems toward higher speeds and perfor-
mances. This is may not be feasible in some cases due to
limitations in the principles of operation (CAN, for instance),
but more importantly, the cost of the new developments nec-
essary for the upgrades allied to a relatively small market is
becoming a major impediment for the manufacturers. On the
other hand, Ethernet and the associated protocols IP, UDP,
and TCP have become so widely used that costs of interface
circuits have been drastically lowered. Furthermore, Ethernet
is now available at higher and higher speeds (1 Gb/s is now
currently offered and 10 Gb/s will soon be), and this trend to-
ward increasing speeds is likely to last. All these reasons have
prompted a number of vendors to offer industrial communi-
0018-9219/$20.00 © 2005 IEEE
1102 PROCEEDINGS OF THE IEEE, VOL. 93, NO. 6, JUNE 2005
cation solutions based on off-the-shelf Ethernet ICs with var-
ious modifications of the protocols to improve predictability.
As we will see later, the intent of the modifications was to
provide real-time guarantees. At the same time, many users
are seriously considering switching to Ethernet based solu-
tions to support communications at the plant floor. In this
paper, we review the different approaches that can be used
to provide real-time guarantees (such as maximum transfer
delay, jitter in the transmissions, and available bandwidth)
using Ethernet and the associated protocols IP [15], UDP,
and TCP [16] as a communication means. The objective is to
assess the guarantees that each proposal can offer. Both soft
and hard real-time guarantees will be considered. We will,
however, restrict our scope to industrial communications, be
they control networks [17], fieldbuses [18], or sensor and ac-
tuator buses [19]. There exist other types of environments
where real-time guarantees are required. Communication of
multimedia streams (radio and TV channels, for instance)
and sensor networks [20] are examples of them. Most of the
analysis presented here can be applied to these environments,
but differences exist that may lead to divergent conclusions.
Analyses of the use of Ethernet in factories have been previ-
ously published [21], [22]. They present interesting but only
partial insights.
Considering the OSI model [23] as a basis, real-time guar-
antees result from a combined effort at all layers. Starting
at the highest layer, the application layer, the interaction
model plays a large role in the guarantees [24]. This may
be illustrated as follows. Let us assume a client–server
interaction model. A client may issue a request. The network
transports the request to the server. The server responds and
the client gets the response through the network. If the server
takes an unbounded time to respond, even if the network
offers real-time guarantees, the client will have no guaran-
tees and the real-time guarantees provided by the network
protocols will be useless. This is one of the reasons why
other interaction models such as publish–subscribe or pro-
ducer–consumer are advocated as better suited for real-time
communications [24]. This issue will not be addressed in
this paper. Neither will we deal with the presentation and
the session layers, as they are most of the time absent.
The next layer is the transport layer. A number of trans-
port layers dedicated to real-time were proposed. Examples
are Express Transfer Protocol (XTP) [25], and Multistream
Protocol (MSP) [26]. A survey of most of the transport pro-
tocols can be found in [27]. As TCP and UDP are the most
widely used, most efforts concentrate on their improvements
in relationship with the Internet Protocol (IP) [28].
At the network layer, IP is the most widely used solution.
The behavior of the routers has a clear impact on the real-
time guarantees that can be obtained. In particular, without
special routing policy, urgent messages may be unnecessarily
delayed by lower priority messages that arrived earlier and
are still waiting to be forwarded.
Finally, at the data link layer, the medium access control
(MAC) scheme is obviously an integral part of the guaran-
tees. To overcome the random nature of the Ethernet ac-
cess protocol, a number of solutions have been proposed.
Some can coexist with regular Ethernet nodes; some reuse
the same hardware but are incompatible; some are compat-
ible but cannot offer guarantees in presence of nodes that do
not implement the same modifications.
As a final word, guaranteeing time-bounded operations is
also a matter of software implementation. This involves ad-
equate implementation architectures [29], as well as proper
scheduling techniques. In many cases, the bottleneck in terms
of throughput is not the network but the local protocol pro-
cessing capability that is not sufficient or not scheduled ade-
quately to receive all the traffic at network speed.
The paper is organized as follows. Section II introduces
the requirements that an industrial communication system
must fulfill. Section III gives an overview of Ethernet from
its first version to the new evolutions. It explains the limita-
tions of the technology with regards to the requirements of
industrial communication. The following section reviews the
different approaches to improve the real-time behavior and
compares their relative merits. We restrict our scope to the
proposals that reuse existing Ethernet hardware. Section V
deals with two major requirements, action synchronization
and temporal consistency [30]. It shows that they cannot be
satisfied without adding a new layer of protocols dealing
with time synchronization. Finally, Section VI concludes the
paper.
II. B
ACKGROUND
A. Application Model
Control applications operate along two different
paradigms, time triggered or event triggered [31]. Time-trig-
gered applications work periodically. They first wait for the
beginning of the period (or some offset from the beginning),
sample their inputs, and compute some algorithm according
to the inputs and some set point data received from com-
puters higher in the hierarchy. They then make the results
available at the outputs. Inputs and outputs correspond to
sensors and actuators at the lowest level in the hierarchy. At
higher levels, inputs correspond to status and completion
reports from the next lower level. Outputs are set points or
commands to the lower level. Acquisition and distribution
applications are special cases. Acquisition applications have
no outputs to the process but store the output results inter-
nally. Distribution applications have no input and compute
the algorithms from stored information.
Periodicity is not mandatory but often assumed because it
simplifies the algorithms. For instance, most digital control
theory assumes periodicity. Furthermore, it assumes limited
jitter on the period and bounded latency from input instant to
output instant. Acquisition and distribution applications have
similar requirements in terms of periodicity and jitter.
Event-triggered applications are activated upon the occur-
rence of events. An event may be the arrival of a message
with a new command or a completion status or the change
of an input detected by some circuitry. When an event
is received, the application computes some algorithm to
determine the appropriate answer. The answer is then sent
as an event to another application locally or remotely. The
DECOTIGNIE: ETHERNET-BASED REAL-TIME AND INDUSTRIAL COMMUNICATIONS 1103
Fig. 1. Line topology based on switches.
time elapsed between the generation of the input event
and the reception of the corresponding answer must be
bounded. Its value is part of the requirements on the ap-
plication and also the communication system if the events
have to be transported through some network. Furthermore,
applications should be able to assess some ordering in the
event occurrences. This is usually not a problem when the
event is detected and processed on the same computer. It
becomes a problem when the events are detected at different
locations linked by a network which may introduced some
variable delay. This issue will be addressed later in the paper
(Section V).
At some occasion, actions on different nodes need be syn-
chronized. This is the case in a machine tool or a robot where
the movements of the various axes must be coordinated to
follow a precise path. This should be supported by the net-
work. The use of broadcast or multicast is an option. Syn-
chronizing distributed clocks is another (see Section V).
B. Network Model
In this paper, we will assume that all nodes are on the same
subnetwork. Traffic may come from and go to external com-
puters through routers, but this traffic has no real-time con-
straints. Without excluding them, we will not discuss virtual
LANs as defined by IEEE 802.1Q, as they do not add to the
discussion. Besides these restrictions, the network topology
is free. Two extreme cases may occur. In the first one, a single
node is at the root of a tree. In the second, nodes are linked to
each other via three-port hubs or bridges. Each node is con-
nected to one hub or switch. The other two ports are used to
link to the adjacent hubs or bridges [32] (see Fig. 1).
C. Data Model
The network transports different kinds of data: process
data, configurations and parameters, programs. Some tem-
poral and spatial properties have to be preserved such as ab-
solute temporal consistency, relative temporal consistency,
and spatial consistency [33]. Absolute temporal consistency
[34] refers to the difference in time between the current time
and the time at which the information has been acquired. This
is the age of information. Most data are no longer useful
when they are too old. The applications should be able to
decide if the information they handle is too old or not. Con-
sider two state variables
and . Let and
be their internal representations where and indicate the
instants at which the values
and of and have been
acquired. At instant
, is said to be absolutely consistent
if and only if
(1)
where
is the absolute consistency threshold for .
At instant
, is said to be absolutely consistent if and
only if
(2)
where
is the absolute consistency threshold for .
Relative temporal consistency applies when samples from
different signals must be correlated in time. It refers to
the temporal delay between the sampling instants on each
signal. Two samples of two signals are said to be temporally
consistent if their sampling instants differ less than a given
duration, called the relative consistency threshold [30].
Many applications assume temporal consistency of their
input signals. For instance, a robot controller based on
the time-triggered approach will read the positions of the
joints to calculate the absolute position of the extremity of
a robot arm. If the positions are not sampled at the same
time (they correspond the position of the joints measured
at different instants), the calculation will be wrong and
the controller will determine a wrong position for the arm.
Networks should support relative temporal consistency by
indicating when this property is present and when it is not.
Using the definition of the internal representation of state
variables,
and are said to be relatively consistent
if and only if
(3)
where
is the relative consistency threshold.
Spatial consistency [35] applies when the same informa-
tion is copied at different locations. The replicas are said to
be spatially consistent if they correspond to the same sam-
pling instant or more generally are identical. Again, some
applications assume that networks will at least tell them if
consistency is present. In the robot controller example given
above, the position of the arm may be used by different dis-
tributed units: one to control the position of the arm, another
to prevent collisions with another robot. If, at a given time,
the values are different, collision may occur. Optionally, con-
sistency may be guaranteed.
D. Traffic Model
At least three types of traffic have to be handled: real-time
periodic, real-time sporadic, and best effort. Best effort traffic
corresponds to file and configuration transfers. Periodic real-
time traffic is typical of time-triggered applications. It is char-
acterized by the following parameters:
(4)
where
is the period of the transfers, its relative deadline
(the deadline is measured from the beginning of the period),
and
the length of the message.
Sporadic real-time messages often result from the event-
triggered applications. They can be described as
(5)
is the minimal interarrival time between two consecu-
tive messages. The other parameters remain the same as in
1104 PROCEEDINGS OF THE IEEE, VOL. 93, NO. 6, JUNE 2005
Fig. 2. Ethernet and IEEE 802.3 MAC frames (SOF: start of
frame; FCS: frame check sequence).
(4). Both periodic and sporadic traffics correspond to the ex-
change of small messages, down to a few bytes of application
data.
File transfers have a periodically sporadic pattern of arrival
(6)
File transfers take place at most every
. Each time
they occur,
messages of duration are sent every .
This reflects that file transfers come at irregular intervals, but
when they start, they generate a rapid flow of messages.
E. Error Model
As strange as it may be, most of the literature on real-time
communications assumes an absence of errors in frame trans-
missions. As this paper is an overview of the approaches,
we will stay with this implicit convention. The interested
reader may, however, consult one of the few notable excep-
tions [36]–[38].
F. Soft Versus Hard Real-Time Constraints
It is usual to distinguish soft real-time constraints from
hard real-time ones. The latter may not be violated, while
the former may occasionally be. In the rest of this paper, we
shall not make any distinction between these two categories
because we will try to assess which guarantees can be ob-
tained by the various approaches.
III. E
THERNET
A. Conventional (Vintage) Ethernet
Ethernet was developed in the 1970s and emerged in
products in the early 1980s. The first IEEE standard on
802.3 [39] was published in 1985. IEEE 802.3 and Ethernet
have little difference. Both use the same MAC algorithm,
Carrier-Sense Multiple Access With Collision Detection
(CSMA/CD). The main difference is the logical link control
sublayer, which is absent in IEEE 802.3 (this is covered by
IEEE 802.2 [40]) and included in Ethernet. As a result, LLC
protocol data units (PDUs) are transparently encapsulated
in IEEE 802.3. Ethernet on its side defines a type field in
the MAC frame that is used by the LLC sublayer (Fig. 2).
This field is also used to carry the identification of the
encapsulated protocol. The corresponding field in the IEEE
802.3 standard is the length field.
Despite the fact that compatibility between both standards
was guaranteed [41], all new products are now IEEE 802.3
compliant.
1
Hence, in the rest of this paper, we will refer only
to IEEE 802.3. Because, Ethernet is now used as a popular
name for IEEE 802.3, we will also use the term Ethernet to
designate this standard.
“Vintage Ethernet” [42] refers to the 1985 version of IEEE
802.3 and its principles of operations. In this version, all sta-
tions on a link share the same medium. In other words, every
station will hear what another station emits. A station that
wants to transmit first listens to the medium. If it finds the
medium free for minimum duration (interframe gap), it starts
transmitting. While transmitting, it monitors the medium to
check for possible collisions. Collisions may occur if two or
more stations start to transmit at the same time. If the frame is
transmitted without collision, the process ends. If a collision
occurs, the station stops transmitting after emitting a jam-
ming sequence. The duration of this sequence has been se-
lected to ensure that all connected stations effectively detect
the collision. The station then prepares for retransmission.
Instead of retransmitting immediately, a process that would
likely cause another collision if all colliding stations do the
same, the station randomly selects a number in the backoff
range. This range is initially 0
1 but is doubled after each
consecutive collision. In other words, if there is a collision
during the first retry the interval becomes 0
3. If there is
collision during the following retry it becomes 0
7. This
repeats until the 10th consecutive collision after which the
interval remains 0
1023. After 16 successive collisions,
the frame transmission is aborted and an error message is re-
turned to the higher layers. After a successful transmission,
the range returns to its initial value. The randomly selected
integer number in the backoff range is multiplied by the slot
time to obtain the backoff delay. This is the delay the station
will wait before attempting to transmit again the frame. The
slot time is a system parameter calculated as the time neces-
sary to send 512 bits (4096 bits at 1 Gbit/s). The slot time is
the maximum propagation delay on the link and thus gives
physical limits to the link [43].
Under light loads, the protocol yields a minimal waiting
time. When the load increases, the random backoff mecha-
nism adapts smoothly to the load of the link. As collisions are
indicators of a busy medium and thus of the load, increasing
the delay before retries is a good way to adjust the link load.
B. Performance Results of Conventional Ethernet
As explained in [44], obtaining accurate results con-
cerning Ethernet is difficult because even reasonable
configurations are intractable using the existing theory.
The quoted paper is an interesting review of a number of
theoretical and experimental analyses of Ethernet. There
are a few results that have been confirmed by experiments
[45]. An IEEE 802.3 network is able to operate close to
100% utilization of the bandwidth (100% utilization occurs
when no time is spared in collisions and retransmission of
messages lost in the collisions) when packets are long even
1
Some “industrial Ethernet” proposals still use the Ethernet frames.
DECOTIGNIE: ETHERNET-BASED REAL-TIME AND INDUSTRIAL COMMUNICATIONS 1105
Fig. 3. Hub-based Ethernet.
with a large number of nodes. With smaller packets, the
maximum utilization drops but remains much higher than
the theoretical limit of 37% [46]. Note that the theoretical
limit has been obtained under simplified assumptions. This
explains the discrepancies with the measurements and the
simulations [44]. There are some other results. The trans-
mission delay increases nearly linearly with the size of the
packet and the number of nodes. The standard deviation of
the delay, which is a measure of the jitter, also increases
with the number of hosts and the size of the packets but
less rapidly. In summary, small packets are favorable when
considering the delay and its variation. However, they lead
to an inferior utilization of the network.
C. Problems With Conventional Ethernet
The MAC is usually fair and gives a minimum waiting
delay when the medium is not too loaded. It is, however, im-
possible to guarantee a bounded message transmission time
with this scheme. Under heavy loads, the 802.3 MAC may
become unfair through a phenomenon called the capture ef-
fect. Let us assume that two stations
and both have a lot
of traffic to submit. At some point, their emissions collide.
Station
chooses 1 as random number in the backoff range.
Station
chooses 0. Station will hence succeed at the first
retry. As it has more traffic to emit, it will then send a new
frame immediately. This frame will collide with station
’s
retry.
will double its backoff range while starts with the
initial range. It is thus likely that
will succeed again first.
The same process may repeat. The result is that station
captures the medium for some period [47], [48].
This example shows that a transmission may be canceled
after the maximum number of retries has been reached, even
in the absence of errors on the link.
D. New Evolutions
Since 1985, IEEE 802.3 has been improved in a number
of ways. In 1987, the 1 Mb/s 1BASE5 version standardized
a twisted pair hub-based architecture. In this topology [43],
each station is linked by a point-to-point twisted pair cable
to a device called a hub that acts as a
-port repeater. The
hub regenerates the signals received from one port and trans-
mits then to all other ports. At the same time, it detects pos-
sible collisions on each port and, when one occurs, it notifies
the collision on all other ports. With this new specification,
the topology becomes a tree (Fig. 3). A 10-Mb/s version of
the same topology followed in 1990. It was upgraded to 100
Mb/s in 1995 and 1 Gb/s in 1998. Except obvious increase
in speeds, all these versions did not change the conclusion of
the previous paragraph.
In 1997, project IEEE P802.3x released the standard for
the so-called full duplex operations. Basically, the hub is re-
placed by a MAC bridge, usually called an Ethernet switch.
Following IEEE 802.1D [49], the switch regenerates the in-
formation and only forwards it to the port on which the desti-
nation is attached (or a link to this destination). As any MAC
bridge [50], the switch complies with the IEEE 802.3 MAC
protocol when relaying the frames. If a frame is already being
transmitted on the output port, the newly received frame is
queued and will only be transmitted when the medium be-
comes idle. In addition, all cables are point to point, from
one station to a switch and vice versa or from a switch to an-
other. These two facts render a full duplex operation possible
on each cable
2
without any collision. Hub-based solutions are
limited to half-duplex because what a station transmits is sent
to all other stations. By eliminating collisions, this new ver-
sion was a big step toward a better predictability of the pro-
tocol. Full prediction of the transmission bounds was not yet
there because overflows could still occur in the switches.
Consider two nodes A and B that transmit at full speed to a
third node C. The link between A and the switch can handle
the traffic. The same applies to the link between B and the
switch. However, the combined traffic exceeds the capacity
of the link between the switch and node C. The excess traffic
accumulates in the switch until its output buffer overflows.
To avoid possible overflows in the bridge queues or in sta-
tions, a station or a bridge may send a
PAUSE frame to the
other side of the link. This frame notifies the other side to
suspend the emission of new frames for a given duration.
Bridges forward the information based on their knowl-
edge of the network. In particular, they learn which port must
be used to reach a given node. For this, they “sniff” the re-
ceived packets and look at the source address field. However,
switches cannot learn to which port, or which ports, they
should forward a frame sent to a multicast address. This is be-
cause there is no explicit subscription to a multicast address.
By default, they must forward multicast frames to all their
ports (except the ingress port). In a typical network, there
may be more than a single path between any two nodes. As a
consequence, a multicast frame may loop inside the network
creating an unnecessary traffic that may not be marginal. To
avoid loops, MAC bridges implement a spanning tree algo-
rithm that is used to disable redundant links [49]. When the
IP is used, Internet Group Management Protocol (IGMP) as-
sociations may be spied on by the switches and the resulting
information used to avoid retransmitting a multicast message
to ports that have no listener for that multicast group [51].
This process, often referred to as IGMP snooping, is avail-
able in many commercial switches.
E. Temporal Behavior
MAC bridges handle up to eight traffic priorities (as per
IEEE 802.1D and IEEE 802.1Q), although in practice only
four can be used. However, most of the time, switches have
2
The cable has one twisted pair in each direction.
1106 PROCEEDINGS OF THE IEEE, VOL. 93, NO. 6, JUNE 2005
