![Fig. 3 The basic RACE architecture; modified from [20].](/figures/fig-3-the-basic-race-architecture-modified-from-20-2w72j56h.webp)
This is a repository copy of The RACE Project: Robustness by Autonomous Competence
Enhancement.
White Rose Research Online URL for this paper:
http://eprints.whiterose.ac.uk/96959/
Version: Accepted Version
Article:
Hertzberg, J, Zhang, J, Zhang, L et al. (18 more authors) (2014) The RACE Project:
Robustness by Autonomous Competence Enhancement. KI - Künstliche Intelligenz, 28 (4).
pp. 297-304. ISSN 0933-1875
https://doi.org/10.1007/s13218-014-0327-y
eprints@whiterose.ac.uk
https://eprints.whiterose.ac.uk/
Reuse
Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright
exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy
solely for the purpose of non-commercial research or private study within the limits of fair dealing. The
publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White
Rose Research Online record for this item. Where records identify the publisher as the copyright holder,
users can verify any specific terms of use on the publisher’s website.
Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request.
Noname manuscript No.
(will be inserted by the editor)
The RACE Project
Robustness by Autonomous Competence Enhancement
Joachim Hertzberg · Jianwei Zhang · Liwei Zhang · Sebastian Rockel ·
Bernd Neumann · Jos Lehmann · Krishna S R Dubba · Anthony G.
Cohn · Alessandro Saffiotti · Federico Pecora · Masoumeh Mansouri ·
ˇ
Stefan Koneˇcn´y · Martin G¨unther · Sebastian Stock · Luis Seabra
Lopes · Miguel Oliveira · Gi Hyun Lim · Hamidreza Kasaei · Vahid
Mokhtari · Lothar Hotz · Wilfried Bohlken
Received: date / Accepted: date
Abstract This paper reports on the aims, the approach,
and the results of the European project RACE. The
project aim was to enhance the behavior of an au-
tonomous robot by having the robot learn from c on-
ceptualized exper i en ce s of previous perfor man ce, based
on initial models of the domain and its own actions in it.
This paper intro du ce s the general system architecture;
it then sketches some results in detail regarding hybrid
reasoning and planning used in RACE, and instances of
learning from the experiences of real robot task execu-
tion. Enhanc eme nt of robot competence is operational-
ized in terms of performance quality and descrip ti on
length of the robot instructions, and such enhancement
is shown to result from the RACE system.
1 Project Aim and Demonstration Domain
RACE (Robustness by Autonomous Compete nc e En-
hancement) is a project funded by the European Com-
mission under the 7th Framework Programme and run-
ning from 12/2011 to 11/2014. The partners are those
J. Zhang, L. Zhang, S. Rockel, B. Neumann, J. Lehmann
Hamburg University
E-mail: zhang@ inf orm atik .u ni-ha mburg.de
K. S. R. Dubba, A. G. Cohn,
University of Leeds
A. Saffiotti, F. Pecora, M. Mansouri,
ˇ
S. Koneˇcn´y
¨
Orebro University
J. Hertzberg, M. G¨unther, S. Stock
Osnabr ¨uck University
L.Seabra Lopes, M.Oliveira, G.H.Lim, H.Kasaei, V.Mokhtari
University of Aveiro
L. Hotz, W. Bohlken
HITeC Hamburger Informatik Technologie-Center e. V.
institutes from which this paper is author e d. This short
project report summarizes the RACE methodology of
working towards achieving these aims, and it sketches
main project results, as visible about half a year before
the end of the project .
The overall aim of RACE as set out in the descrip -
tion of work was
to develop an artificial cognitive system, embod-
ied by a service robot, able to build a high-level
understanding of the world it inhabits by stor-
ing and exploiting appropriate memories of its
experiences. Experiences will be recorded inter-
nally at multiple levels: high-level descriptions in
terms of goals, t ask s and behaviours, connected
to constituting subtasks, and finally to sensory
and actuator skills at the lowest level. In this
way, experiences provide a detailed account of
how the robot has achieved past goals or how it
has failed, and what sensory events have accom-
panied the activities.
Contributions were foreseen in the description of
work to advance the state of the art along three lines:
1. robots capable of storing experiences in their
memory in terms of multi-level representa-
tions con ne ct i ng actuator and sensory ex pe-
riences with meaningful high-level structures,
2. methods for learning and generalising from
experiences obtained from behaviour in r e al-
istically scaled real-world environments,
3. robots demonstrating superior robustness and
effectiveness in new sit uat i on s and unknown
environments using experience-based planning
and behaviour ad apt at i on.
2 Joachim Hertzb erg et al.
Fig. 1 The PR2 robot Trixi grasping mugs from the counter.
So the thrust of the project was clearly of a con-
ceptual nature. Yet, to demonstrate an integrated sys-
tem and to have it learn f rom experiences, a physical
robot and a demonstration domain are clearly needed.
An “academi c” demonstration domain was used to fo-
cus on the conceptual issues rather than application re-
quirements, and to keep low the overhead for providing
permanently th e real-life demonstrat i on scenario and
for modeling it in a go od simulation environment that
would al low project partners to work independently at
their sites between code integration events.
The demonstration domain is an AI and Robotics
classic: a (mockup) restaurant wi t h a robot waiter. The
robot, Trixi, Fig. 2, is a PR2 with an additional RGB-
D camera on top of its head. The task spectrum of
Trix i is to serve guests in the mockup restaurant. Fig. 2
schematically shows one of a number of several scenar-
ios defined for the rest au rant domain; these s ce nar i os
are available both physically in a lab room and in sim-
ulation (Gazebo). Having such fixed scenarios allows
tasks to be executed under somewhat controlled en-
vironment conditions to compare robot performances
over different degrees of experience in the domain on
Trix i ’ s side.
In the sparsely populated scenario in Fig. 2, i t would
make sense to give Trixi the order “Serve mug1 to guest1!”
or “Se r ve coffee to guest1!”, both yielding the same s er -
vice. It is also conceivable to teach Trixi how to ser ve:
“Pick up the mug at the counter, bring it to th e guest
at table1 – thi s is how to serve a coffee”.
It is assumed that basic robot behavior (such as nav-
igation, object handling, object recognition) is available
on Trixi – actually, RACE has started from standard
capabilities available for a PR2 in ROS [17], cf. Sec. 2.1
for an explanation of the control architecture. Standard
restaurant action schemata for a waiter, such as serving
something to s ome guest, are available in a pre-d efi n ed
Fig. 2 Schema of an instance of the RACE demo scenarios in
the restaurant do mai n. The counter of Fig. 2 is the counter1
on the left. See text for more explanations.
form as Hierarchical Task Network (HTN) methods,
cf. Sec. 3.1 for more on pl ann i ng in RACE. Trixi was
able to physically perform such restaurant standard ac-
tions in closed-loop plan-based c ontrol from early on
in the project, based on the control architecture ex-
plained later (Sec. 2.1). This state-of-the-art approach
was t aken as the ground level of perform anc e compared
to which competence could be enhanced from experi-
ence by met h ods to be developed in the project.
Now what are reasons and opportunities for com-
petence enhancement here? In a mundane domain like
a restaurant, there is an infinite set of possibilities for
variations of tasks to be executed in the light of ac-
tual conditions, even though the domain itself and the
actions for a waiter (human or robot) to perform are
highly schematized. These variations are the sources of
possible disturbances for Trixi’s execution – actually,
they are the sources of the brittleness of autonomous
robot performance in real-world settings that is so often
deplored. They would in general result in non-nominal
execution of the planned behavior, or in needed varia-
tions of the planned behavior at execution time. For ex-
ample, unknown at planning time, paths may be blocked
for the robot, the guest may have changed his seat on
the table, standard placing areas on the tab l e may be
occupied by belongings of the guest, standard manip-
ulation areas for the robot to stand while serving the
table may be blocked, a newly arriving guest may inter-
rupt plan execution, and so on. Conditions on all levels
of descr i pt i on of robot performance (temporal, spatial,
causal, perceptional, kinematic, dynamic) may actually
deviate from the standard – no matter how the s t and ard
is formulated in detail. The RACE id ea is that actually
experiencing such deviations and learning ways how to
deal with them (cf. Sec. 3.4) should lead to more ro-
bust performance in the domain. Moreover, being able
to conceptualize such experiences and thereby to gen-
eralize them and make them amenable to the robot’s
The RACE Project 3
own reasoning would result in a transfer from concr et e
experiences to classes of situations in which to change
or adapt the st and ar d behavior.
2 Approach
It is apparent from the overall aim of RACE that the
project would face at least three methodological issues
(which it shared with quit e a few companion projects).
First, a bootstrapping problem: to generate robot ex-
periences to learn from, the project had to rely on a
fully integrated and functional robot system in a suit-
able environment from the project start. Second, an
architecture problem: to learn from conceptualized ex-
periences of its own past behavior based not only on
external features (“sensor streams”) , but also on the
internal control knowledge that led to generating the
past behavior, all that d at a and knowledge has to be ex-
plicit and available for learning. More over, to be able to
change its own behavior as a result of learning, the con-
trol knowledge yielding the behavior has to be explicit
for the control. Third, an evaluation problem: to demon-
strate competence enhancement after learning from ex-
perience, some performance metrics need to be used
that would allow a sensible before-after comparison.
This section sketch es the RACE solutions to these
three issues.
The central point to solving the bootstrapping prob-
lem for RACE was early integration. The project has
generated in it s first year a fully integrated and func-
tional robot in the restaurant domain with an ini t i al
instance of its target c ontrol architecture (cf. Sec. 2.1)
in place. This was made possible by
– committing to a partic ul ar version of the above-
described demo domain;
– using a PR2 robot and ROS as readily available
hardware and software frameworks, res pect i vely;
– using prior existing standard processing and reason-
ing modules as b ase syst ems wherever poss i bl e, e.g. ,
for planning and sensor d at a interpretation;
– defining the internal knowledge-interchange language
based on a standar d, namely, Description Logics;
– and committing early to the basic robot control ar-
chitecture, i.e., to a solution of the second problem
addressed above.
Of these items, we will only detail the architecture is-
sue, treated next; but we want to emphasiz e that the
cross-topic and cross-wor k package results achieved in
the pr oject are to a large d egr ee due to this early inte-
gration made p os si bl e in a joint effort by the partners.
The approaches to the control architecture and eval-
uation problems are described next in some det ai l .
Ontology
OWL
Reasoning
and
Interpretation
Experience
Extractor/
Annotator
HTN
Planner
Blackboard
User
Interface
ROBOT
Capabilities
Memory
Perceptual
Perception
Execution
Monitor
Conceptualizer
concepts
OWL
new concepts
fluents
plan
initial state,
continuous
ex−
ex−
instructions
data
fluentsfluents
periences
instructions
periences
data
sensor
actions
ROSplan
fluents
goal
plan,
goal
fluents,
schedule
action
results
OWL
concepts
Fig. 3 The b asic RACE architecture; modified from [20].
2.1 Control Architecture Approach
The corner s ton e of the RACE architecture (Fig. 3) is
the Blackboard. It mainly contains fluents, i.e., ground
facts of the Description Logic (DL) ontology (ex ec ut e d
actions, world state propositions, etc.), with begin and
end timestamps. It is implemented as an RDF database.
We decided to use a classical, “flat” blackboard in the
project to allow for maximal flex i b il i ty of information
flow between mod ule s , including reasoning and learning
modules, and for freely adding and exchanging versions
of modules. This strategic advantage clearly comes at
the cost of hard-wiring a bottlen eck into the architec-
ture; yet, the benefit has outweighed the cost in RACE.
The other modules for perception, reasoning, plan-
ning and execution communicate by reading selected
types of information from the Blackboard, processi ng
this information and writing back their outp ut s. So the
Blackboard serves two roles: from the fluents on it, the
current state as well as past state information can be
derived; and it contains complete experience records,
which can be conceptual i ze d later.
When a new planning goal is entered by the user,
an HTN Planner queries the Blackboar d to build its
initial planning state, then writes the generated plan
back into the Blackboard. Initially, SHOP2 [13] was
used, later replaced by the planner sketched in Sec. 3.1.
The stored plan includes operators ’ preconditi on s and
effects as well as the hierarchy of expanded HTN meth-
ods. The plan is picked up by the Execution Monitor,
which dispatches th e planned actions to the robot plat-
form, mapping them to its c l osed -l oop control module s.
During execution, the monitor l ogs the executed ac-
tions, as well as success or failure information, in the
Blackboard.
ROS [17], as used on Trixi, already provides many
capabilities (e.g., for manipulation or navigation) as
ROS actions; others were added. The robot provides
continuous d at a about its own status (such as joint an-
4 Joachim Hertzb erg et al.
Specification of
ideal behavior
Environment
RACE
System
trace
execution
compare
Distance to Ideal Model
Fig. 4 Principle of evaluation: the system’s behavior is com-
pared to a model of the ideal behavior for the specific scenario.
gles) as well as data from its sensors. The Perception
module di sc re t iz es thi s inform at ion into symbolic, time-
stamped fluents.
The OWL ontology stores the robot’s conceptual
knowledge. It provides a common representation for-
mat, from which the knowledge used by all other rea-
soners is generated. Spatial, temporal, resource and on-
tological reasoners as well as a high-level scene inter-
pretation module contribute higher-level se mantic in-
formation to the experiences via the Blackboar d .
Background processes responsible for experience ex -
traction (grouped in an Experience Extractor module)
and concept ual i zat i on (Conceptualizer) support a long-
term learning loop, resulting in more robust and flexible
future plans. The architecture is detailed in [20].
2.2 Evaluation Approach
To eval u at e success for a gi ven task in a given scenario,
we measure the compliance of the actual robot behav-
ior to the intended ideal behavior for that task in that
scenario. Fig. 4 illustrates this principle: the trace of
a given execution of Trixi is compared to a specifica-
tion of what the ideal behavior should be, resulting in
a “Distance to Id eal Model” (DIM) measure.
Discrepancies between the observed and the ide al
behavior can originate from errors of four different types:
Conceptual, Perceptual, Navigation and/or Lo-
calization, and Manipulation er ror s. The latter three
types of errors are, to some degree, platform specific.
Our metrics focus on qu antifying conceptual errors.
Conceptual errors arise from discrepancies betwee n
the knowledge used by the robot and the one encoded
in the specification of the ideal behavior. We call these
discrepancies inconsistencies. Again, they can be of four
types: (1) Temporal, (2) Spatial, (3) Taxonomical,
and (4) Compositional. The DIM metric chosen in
RACE i s the weighted sum of the numbers of the in -
consistencies (1–4), respectively, lower DIM values si g-
naling better behavior.
In ad di t i on to es ti m ati n g the effec t iveness of le ar ne d
knowledge by DIM, the Description Length (DLen, [19])
of the instructions given to the robot to achieve a goal
DIM
DLen
Fig. 5 RACE’s aim: use as few as possible instructions (low
DLen) to achieve correct behavior (low D IM ). The enhance-
ment of competence is indicated by the transition from the
solid line to the dashed line
matters. Normally, longer desc rip t ion s coul d yie ld bet-
ter D IM as sugge st ed by the solid l i ne in F i g. 5. Afte r
learning fr om experiences, still successful or even more
successful (even lower DIM) behavior following shorter
instructions would be indicative of the effectiveness of
the learned knowledge. This may indirectly provide a
measure of how general the knowledge is, too, if applied
to a wid e range of scenarios and initial conditions.
So, the general RACE aim of designing learning and
reasoning tools for a robot to autonom ous ly and effec-
tively increase its competence was operationalized as:
make it pos si b le for a robot to collect experiences al-
lowing it to perform at lower DIM and sh or t er DLen.
3 Results
In addition to the overall system behavior, RACE has
yielded a number of results in the individual modules
shown in Fig. 3. They are sketched next. Details are in
the references and on t h e website [18].
3.1 Hybrid Reasoning and Pl an ni n g
To enable early integration as mentioned in Sec. 2, off-
the-shelf planners were used in the beginning of the
project. The goal was to analyze the limitations of the
state of the art and develop an integrated planning sys-
tem to overcome them. For task planning, HTN plan-
ning [5] proved to be useful for improving the robot’s
performance based on experience: th e plan generation
itself is fast, and the plans are robust and have a st r uc -
ture that can be used for learning.
While employing the off-the-shelf SHOP2 HTN plan-
ner was good for early integration, it was evident that
state-of-the-art planning techniques were inadequate for
the pur poses of RACE: none of them could leverage the
full knowledge that the project set out to learn from
experience. The key issue is that this knowledge is hy-
brid addressing diverse semantics. For example, Trixi