ARQuake: An Outdo or/Indo or Augmented Reality First Person
Application
Bruce Thomas, Ben Close, John Donoghue, John Squires,
Phillip De Bondi, Michael Morris and Wayne Piekarski
Sc hool of Computerand Information Science
Universit y of South Australia
Bruce.Thomas@UniSA.Edu.Au
Abstract
This pap er presents an outdoor/indoor augmentedre-
ality rst p erson applic ation ARQuake we have devel-
op ed. ARQuake is an extension of the desktop game
Quake, and as such we are investigating how to convert
a desktop rst person application into an outdoor/indoor
mobile augmentedreality application. We present an ar-
chite cture for a low cost, moderately accurate six degr ees
of freedom tracking system based on GPS, digital com-
p ass, and ducial vision-basedtracking. Usability issues
such as monster selection, c olour, and input devices ar e
discussed. Asec ond application for AR architectural de-
sign visualisation is presented.
1 Intro duction
Many current applications place the user in a rst-
person p erspective view of a virtual world [6], suc h as
games, arc hitectural design viewers [2], geographic in-
formation systems and medical applications [12 ]. In
this pap er w e describe a pro ject to mov e these forms
of applications outdo ors, displaying their relevantinfor-
mation b y augmenting realit y . In particular w e con-
sider the game Quake [4] and the viewing of arc hitec-
tural designs [13]. As with other researchers [3], w e
wish to place these applications in a spatial context with
the physical world, whichweachieveby employing our
w earable computer system Tinmith-4 [9, 10]. Tinmith-
4 is a con text-a w are wearable computer system, allow-
ing applications to sense the p osition of the user's b ody
and the orien tation of the user's head. The tec hnique
we are developing will genuinely take computers out of
the lab oratory and into the eld, with geographically-
a w are applications designed to in teract with users
in
the physical world
, not just in the connes of the com-
puter's articial reality . The key to this exciting prac-
tical tec hnology is
augmented reality
(AR). Users wear
see-through head-mounted displays through whichthey
see not only the w orld around them, but also ov erlaid
computer-generated information that enriches the user's
perception. Unlike virtual reality , where the computer
generates the en tire user en vironment, augmented re-
ality places the computer in a relatively unobtrusive,
assistiverole.
In the ARQuake application, the physical w orld is
modelled as a Quake 3D graphical mo del. The aug-
mented reality information (monsters, weapons, ob jects
of interest) is display ed in spatial context with the physi-
cal world. The Quake mo del of the physical world (walls,
ceiling, oors) is not shown to the user: the see-through
displayallows the user to see the actual wall, ceilings and
oors whichARQuake need only mo del internally. Co-
incidence of the actual structures and virtual structures
is k ey to the inv estigation; the AR application mo dels
the existing physical outdo or structures, and so omis-
sion of their rendered image from the display b ecomes
in eect one of our rendering techniques.
1.1 Aims
Our
aim
is to construct rst-person p erspective ap-
plications with the following attributes: 1) The applica-
tions are situated in the physical world. 2) The p oint
of view which the application shows to the user is com-
pletely determined b y the position and orientation of
the user's head. 3) Relevan t information is displayed
as augmented realit y via a head-mounted see-through
display . 4) The user is mobile and able to walk through
the information space. 5) The application is op erational
in both outdo or and indo or environments. 6) The user
interface additionally requires only a simple hand-held
button device.
1.2 Research issues
T o achieve these aims, w e inv estigate a n umber of
research issues in the areas of user interfaces, tracking,
and con v ersion ofexisting desktop applications to AR
en vironments.
User in terfaces for augmented realit y applications
which simultaneously display both the ph ysical w orld
and computer generated images require sp ecial care.
The c hoice of screen colours for the purely virtual im-
ages which the application must display requires atten-
tion to the lighting conditions and background colours
of the outdo ors. The keyboard and mouse interactions
Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00)
0-7695-0795-6/00 $10.00 @ 2000 IEEE
In ISWC2000 - 4th International Symposium on Wearable Computers
October, 2000 - Atlanta, Ga, USA - Copyright (C) 2000 IEEE
Please visit http://www.tinmith.net for more information
must b e replaced with head/b ody mov ement and simple
buttons. The lay out of the user interface must accom-
modate the AR nature of the application.
The six degrees of freedom (6DOF) tracking require-
ments for these forms of applications must b e addressed.
We require a low cost, mo derately accurate 6DOF track-
ing system. T rac king is required for indoor and outdo or
environments o ver large areas, for example our usual
testing environment is our campus [7]. GPS p ositional
error has a less noticeable eect for the registration of
augmented realit y information at distance, but we need
to address p ositional error when registering augmented
information at close distances (
<
50
m
). Suchatrack-
ing system could b e used for other applications, suchas
tourism information, visualisation of GIS information,
and as described in this pap er arc hitectural visualisa-
tion.
It is also necessary to mo dify the Quake game to ac-
commodate the AR nature of the new application. The
user's movement c hanges from a k eystroke-based rela-
tivemov ement mode to a tracking-based absolute mo de.
The game's co ordinate system must b e calibrated to the
physical world. Finally, the eld of view of the display
must b e calibrated to the physical world.
1.3 Progress to date
This is an ongoing pro ject, and it is not complete at
the time this pap er was submitted. We wish to clarify
the status of the dierent p ortions of system presented
in this paper. The ARQuake game has been ported
to our our w earable computer platform and op erates
with GPS and digital compass as a means of tracking.
The keyboard and mouse interaction with the game has
been completely replaced with user mov ements and a
tw o-button input device, and is fully functional within
the accuracy of this tracking system. Wehav e modelled
in our Quake w orld an outdo or section of our campus
and the interior of our Wearable Computer Lab oratory
(WCL). The graphics of the game runs at 30 frames per
second, but GPS up dates once p er second and the com-
pass updates at 15 times p er second. The colours of the
graphics inthe game hav e been optimised for the user
with a see-through displayinanoutdoorenvironment.
The ma jor hurdle left is the vision based tracking sys-
tem. Wehav e the vision based tracking system export-
ing a common co ordinate system as to the GPS/compass
system. There are issues of accuracy and sp eed which
are currently being inv estigating, which preven ts us from
stating this portion of system is functional. The pap er
presents the current state w e ha veachieved in this re-
gard.
2 Background
There are key tec hnologies we are employing in our
in v estigations. A brief review of tracking as applied to
this pro ject, the Quake game, and our w earable com-
puter platform are supplied.
2.1 Tracking
Previous research has established that outdo or track-
ing with inexp ensive dierential GPS and commercial
grade magnetic compasses are inaccurate for augmented
realit y applications [1]. T raditional hybrid approaches
combine a numb er of dierent systems such as inertial,
optical, electro-magnetic and GPS. We combine vision-
based optical tracking with GPS and a magnetic com-
pass.
A number of researchers are inv estigating
ducial
vision-based trac king [8, 11]. We based our optical
tracking system on the ducial marker tracking sys-
tem ART o olKit developed by Kato and Billinghurst [5].
The ART o olKit is a set of computer vision tracking li-
braries that can be used to calculate camera position
and orientation relativeto physical markers in real time.
ARTo olKit features include the use of a single camera
for p osition/orientation tracking, ducial tracking from
simple black squares, pattern matching software that al-
lows an y marker patterns to be used, calibration code
for video and optical see-through applications, and suÆ-
cien tly fast p erformance for real-time augmented reality
applications.
The ducial markers are kno wn-sized squares with
high contrast patterns in their centres. Figure 4 shows
an example marker. The ART o olKit determines the rel-
ative distances and orientation of the marker from the
camera. In addition, the ARTo olKit incorp orates a cal-
ibration application to determine the placementofthe
camera relative to the user's line of sight; thus the AR-
To olKit can determine proper placement of graphical
ob jects for AR applications.
2.2 The original Quake game
We chose Quake as the primary application for a
number of reasons. Quake ts the general model of AR
whichwe are studying, as it is a rst-person 3D applica-
tion with autonomous agents to interact with the user.
The application itself is public domain, with open source
code. Finally, the Quake graphics engine is very quick
and runs on a wide range of computing platforms and
operating systems.
Quake is a rst-p erson
shoot 'em up
game. Quake
has t w o stated goals:\First, stay aliv e. Second, get out
of the place you're in"[4]. Theuserin terface is based
around a single, rst-person persp ective screen. The
large top part of the screen is the view area, showing
monsters and architecture. Status information is imme-
diately b elow at the b ottom of the screen.
One moves around Quake in one of four mo des: walk-
ing, running, jumping or swimming, andp erformsone
of three actions: sho oting a weapon, using an ob ject, or
pic king up an ob ject. Weap ons are aimed bychanging
the view direction of the user, and red b y pressing a
key. T o push a button or op en a do or, the user walks
up to the door or button. A user pic ks up items b y
w alking over them. P art of the challenge of the game is
Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00)
0-7695-0795-6/00 $10.00 @ 2000 IEEE
nding sp ecial ob jects like buttons, o or-plates doors,
secret doors, platforms, pressure plates and motion de-
tectors. Quake incorporates platforms that moveup and
down, or followtracks around ro oms or levels. Pressure
plates and motion detectors maybe invisible or visible,
and there are sensors which open do ors, unleash traps,
or warn monsters.
2.3 Wearable computer platform
The Tinmith-4 wearable computer system hardware
is all mounted on a rigid bac kpack so that the items
can be attached rmly. Pro cessing is p erformed b y a
T oshiba 320CDS notebo ok (Pentium-233, 64 Mb RAM)
running the freely a v ailable Lin uxOS and asso ciated
programs and dev elopment tools. The laptop is v ery
generic, and not ev en the latest in a v ailable CPUs, so
another computing unit could be substituted. The lim-
ited I/O capabilities of the single serial p ort are aug-
mented with the use of a four serial p ort Quatech QSP-
100 communications card. Connected to the laptop are
a Precision Navigation TCM2-80 digital compass for ori-
entation information, a Garmin 12XL GPS receiver for
positioning, and a DGPS receiver for improv ed accuracy.
For the head mounted display (HMD), w e use alter-
nately the i-Glasses unit from I-O Display Systems, and
the Sony Glasstron PLM-S700E. Various other devices
are present as w ell, such as a small forearm k eyb oard
for data en try , pow er con v erters forthe dierent com-
ponents, and necessary connection cabling and adaptors.
The construction of the backpackwas directed with ease
of modications in mind, at the sacrice of wearability
and miniaturisation.
The Tinmith system [10] supports outdo or aug-
mented reality research. The system is comprised of
anumber of interacting libraries and mo dules. Firstly,
anumber of soft w are libraries form a supp ort basefor
writing co de in the system: libGfx - a graphics inter-
face on top of X windows; libConv ert - co ordinate and
datum transformations, n umeric conv ersions; libProto-
col - enco de/decode libraries for transmitting structures
o v er a netw ork; libSystem - net w orkcommunications
and high level I/O; libCodeBase - low level interfaces to
Unix system calls, asynchronous I/O co de, string han-
dling, event generation, error checking. These libraries
are used in turn to implement soft w aremo dules (im-
plemented as individual Unix pro cesses) that perform
the actual tasks in the system. These software mo dules
process input from hardware devices and other mo dules,
and then pass this output on to other modules which
are interested in these v alues. The communication be-
tw een modules is p erformed using TCP/IP, and allows
the system to b e distributed ov er a net work of wearable
processors, and also for other machines to collab orate
and share information.
The original Tinmith-1 system was an outdo or aug-
mented realit y na vigation system, supporting 2D top-
down maps, and a 3D immersive wire frame ren-
derer [10]. This system was then extended as Tinmith-2
to share information with standard VR entity proto-
cols [9]. Simulated and real entities app ear on the HMD,
and outdoor wearable machines app ear back on the in-
door displays.
3 Using ARQuake
The goal of ARQuakewas to bring the intuitivena-
ture of VR/AR interfaces into an indo or/outdoor game.
A user rst dons the wearable computer on their back,
places the HMD on their head, and holds a simple two-
button input device. The user then p erforms a simple
calibration exercise to align the HMD with their eyes,
and then they start playing the game. All of the key-
board and mouse controls hav e been replaced with p o-
sition/orientation information and a two-button input
device. As mov ement aspects of the game hav e b een en-
gineered to t the physical world, there is no concept of
commands to walk, run, jump, swim, or of moving plat-
forms. The user's o wn mov ement determines the rate
and direction of movement. The remainder of this sec-
tion describes the Quake lev el w e dev elop ed and its user
interaction.
3.1 Monsters
There are sixteen dierent typ es of monster in the
Quake world. Some hav e attributes that make them
unsuitable for inclusion in this type of level. Because of
the limitations on movement imposed b y the tracking
hardware, the b est monsters were those that walked or
leaped and those that were relativ ely easy to destroy
and did not inict extreme damage on the user with
their rst attack.
We c hose seven t yp es of monsters to be included in
this lev el. These monsters types are all land-based crea-
tures whichuseweap ons from a distance, and all seem
w ell suited to the system. The monsters'
skin
colour and
texture were c hanged to make them easier to see and dis-
tinguish from the physical world. The choice of colours
used in the texture maps or skins of the monsters are
based on the user testing described later in Section 5.
3.2 Campus level
We created a Quake level representing a p ortion of
the Levels campus of the University of South Australia.
The walls in Quake are the walls of the external and in-
terior of the WCL. The walls are rendered in two fash-
ions, black for game mode and a grid patterned for test-
ing mode. In both these modes, the w alls o cclude the
graphic ob jects in Quake that may be lo cated behind
the walls. As describ ed earlier, in the game mo de black
walls are invisible to the users during the game. The
Quake graphics engine renders only monsters, items on
the ground, and regions of interest. This Quake level
w as derived from architectural drawings of the campus
provided by the university; where the architect's draw-
ings had become incorrect, wesurvey ed those p ortions
ourselves.
Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00)
0-7695-0795-6/00 $10.00 @ 2000 IEEE
Figure 1: Quakecampuslevel
The size of the outside mo delled area is 94 metres
(East/West) b y 156 metres (North/South). Figure 1
depicts a top-down view of the level we created. Wehav e
placed 16 monsters in the levels as follo ws:tw o enforcers
on top of D Building, tw o ogres on the 2nd o or of F
building, and the rest spread around the ground level.
There are 51 items placed on the ground for the user
to pickup: six pieces of armour, 22 ro ck ets, four rock et
launchers, nine shotgun shells, and ten health b oxes.
The system of tracking used in this system tends to
make the user less agile than the \sup er-human" agility
found in the normal game. Therefore wehav e included
more supp ort equipment than w ould be found in the
normal game, armour, weapons and ammunition.
3.3 Walking around
Once the system is up and running, the user mov es
through the level bywalking, and changes view bylook-
ing around. The user views the game and the physical
w orld through the HMD, an example is shown in Fig-
ure 2. The bottom portion of the screen is a status bar
con taining information ab out armor, health, ammo and
w eap on type. The ma jority of the screen is reserved for
the AR images of monsters and game ob jects.
In the original Quake, certain actions are performed
b y the user being in a certain proximity to a lo cation in
aQuakelevel. Wehav e retained most of those actions.
Doors op en when the user attempts to w alk through
them. Users pic k up ob jects as in the original Quake
bywalking ov er them. T raps are triggered by standing
in or moving through predetermined lo cations. Actions
which are not easily reected in the physical world are
remov ed from the game, such as secret and lo cked do ors.
The tracking of the user's p osition and orientation of
the user's head handles the ma jority of the interaction
Figure 2: User's Heads Up Display
for the user. The only other interactions for the user
to perform are to shoot or change the currentweapon.
We employatw o-button (thumb button and index n-
ger button) hand-held device as a physical input device
for these actions. The thumb button is used to change
weapons, and the index nger button res the current
weapon. The direction the weapon res is the center of
the current view of the HMD.
3.4 Field of view
Even if alignment of the Quakeworld with the phys-
ical world is exact, an incorrect p erspective or eld of
view will be highlighted as inconsistencies in the virtual
w orld. The default eld of view for the game is 90 de-
grees (45 degrees each side), allowing a reasonable cov er-
age of the world to t onto a computer screen. This eld
of view unfortunately suers from the
sh eye
distortion
eect when comparing the ob jects in the Quakeworld
with real ob jects. The HMD we are using, I-Glasses, has
approximately a 25 degree horizontal eld of view. The
only calibration adjustment for the HMD with Quakeis
c hanging the game's eld of view setting and scaling of
the graphical ob jects. We are currently using a eld of
view v alue of 25 degrees, but there are artifacts intro-
duced as in the user is p ositioned farther forward. We
are inv estigating the graphics mo del of Quake to deter-
mine how it diers from traditional graphics mo dels.
4 Tracking
As previously stated, one of the goals of the sys-
tem is to provide contin uous indo or and outdoor track-
ing. The system trac ks through the combination of a
GPS/compass system with a vision-based system. Our
tracking needs are categorized in to three areas as fol-
lows: outdoors far from buildings, outdo ors near build-
ings, and indoors. Each of these require a dierent ap-
proach, while maintaining p osition and orientation in-
Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00)
0-7695-0795-6/00 $10.00 @ 2000 IEEE
formation in a common format of WGS 84/UTM po-
sitioning information and heading/pitch/roll angles for
orientation information. The use of visual landmarks
can improv e registration in one of tw o w ays, rstly to
allow the system to correct the nal image by aligning
the landmark with a kno wn p osition in the graphical
image, and secondly to use the landmarks to extract a
relativ e position and orientation of the camera from the
landmarks. Wehavechosen the second option to inv es-
tigate as it provides the most general tracking solution.
4.1 Outdo ors a wa y from buildings
GPS p ositional inaccuracies are less of a problem for
our Quake application when a user is at a large distance
(
>
50
m
) from an ob ject which requires registration,
while orientation errors remain constant as to angular
deviations in the user's eld of view. An extreme ex-
ample of how p ositional errors hav e a reduced registra-
tion error eect at distance is the using of the ARQuake
game on a at op en eld, where the system does not
require graphics to b e registered to anyphysical ob ject
except the ground. In this scenario there are no walls
to o cclude the monsters and items of interest. Since the
game is slav ed to the screen, what the user sees on the
display is what the game b elieves is the user's current
view. Therefore the user's actions will perform correctly
in the context of the game.
In the case where a building is visible but the user
is a large distance from the building, the inaccuracies
are lo w and therefore not distracting. The problems
come when monsters are not occluded prop erly bythe
physical buildings. The visual eect of p oor occlusion is
that monsters appear to walk through walls or pop out
of thin air, but at distance these errors donot detract
from the game. Such o cclusion problems exist but they
are visually very minor, b ecause the user is generally
moving their head during the operation of the game..
A t 50 meters a dierence of 2-5 metres (GPS tracking
error) of the user's p osition is approximately a 2-5 degree
error in user's horizontal eld of view, and the compass
itself has an error of +/- 1 degrees.
4.2 Outdo ors near buildings
When using ARQuake with the GPS/compass track-
ing less then 50 metres from a building, the poor oc-
clusion of monsters and ob jects near the physical build-
ings, due to GPS error, becomes more apparent. As the
user mov es closer to buildings, inaccuracies in GPS posi-
tional information b ecome prevalen t. The system is now
required to slav e the Quakeworld to the real world, and
furthermore in real time. As an example when a user is
ten metres from a building their p osition is out by 2-5
metres, this equates to an error of 11-27 degrees; this
is approximately a half to the full size of the horizon-
tal eld of view of the HMD. When the error is greater
than the horizontal eld of view, the virtual ob ject is
not visible on the HMD.
Figure 3: Fiducial marker on a building
T o enhance the accuracy when the user is near build-
ings we use an extended version of ART o olKit. By us-
ing ducial markers sp ecically engineered for outdoor
clarity (approximately 1 metre in size), and with eac h
marker setup on arealworld ob ject with known co or-
dinates, accurate lo cation information can be obtained.
Figure 3 shows a ducial marker on the corner of a build-
ing in our Quakeworld. These markers provide a correc-
tion in the alignmentof the twoworlds. We are in v es-
tigating the use of multiple ducial markers to reduce
uncertainty due to marker mis-detection caused by light-
ing issues. Since the extended ARTo olKit we are devel-
oping supplies positioning and orien tation information
in the same format as the GPS/compass system, AR-
Quake can transparently use either the GPS/compass
or vision-based tracking systems. Our initial approach
for determining when to use the information from the
GPS/compass or the ARTo olKit metho ds is use the AR-
T o olKit's information rst, when the ARToolKit is con-
dent of registering a ducial marker. As ART o olKit
recognises a ducialmark er, the to olkit returns a con-
dence v alue, and the system will ha v ea threshold of
when to switc h o v er to use the the toolkit. When
the condence value go es b elow the the threshold, the
GPS/compass information is used.
4.3 Indo ors
As a user walks in to a building with ducial markers
on the inside walls and/or ceilings, the tracking system
starts using the vision-based comp onent of the tracking
system. This form of tracking is similar to the work of
Ward, et al. [14 ]. Our system is low er-cost and is not
as accurate, but does keep tracking errors within the
accuracy which our application needs, 2-5 degree of error
in user's horizontal eld of view. We are exp erimenting
with placing markers on the walls and/or the ceilings.
Figure 4 shows one conguration of how w e are using
wall mounted markers.
When the markers are placed on the w all, w e point
the vision-based tracking camera forwards. It was nec-
essary to size and p osition the patterns on the walls so
Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00)
0-7695-0795-6/00 $10.00 @ 2000 IEEE
