IEEE TRANSACTIONS ON MEDICAL IMAGING, FINAL MANUSCRIPT #187/98, JUNE 30, 1999 1
Geometrically Correct 3-D Reconstruction of
Intravascular Ultrasound Images by Fusion with
Biplane Angiography — Methods and Validation
Andreas Wahle,
Member, IEEE, Guido P. M. Prause, Steven C. DeJong, and Milan Sonka, Member, IEEE
Abstract— In the rapidly evolving field of intravascular ultrasound
(IVUS), the assessment of vessel morphology still lacks a geometrically
correct 3-D reconstruction. The IVUS frames are usually stacked up to
form a straight vessel, neglecting curvature and the axial twisting of the
catheter during the pullback. Our method combines the information about
vessel cross-sections obtained from IVUS with the information about the
vessel geometry derived from biplane angiography. First, the catheter path
is reconstructed from its biplane projections, resulting in a spatial model.
The locations of the IVUS frames are determined and their orientations
relative to each other are calculated using a discrete approximation of the
Frenet-Serret formulas known from differential geometry. The absolute
orientation of the frame set is established utilizing the imaging catheter it-
self as an artificial landmark. The IVUS images are segmented using our
previously developed algorithm. The fusion approach has been extensively
validated in computer simulations, phantoms, and cadaveric pig hearts.
Keywords—Coronary Artery System, Intravascular Ultrasound, Biplane
Angiography, Multimodal Image Fusion.
I. INTRODUCTION
A
CCURATE ASSESSMENT of vessel lesions like stenoses
or diffusealterationsisindispensableindiagnosisandtreat-
ment of coronaryartery disease. For a number of decades, quan-
titative coronary analysis from selective contrast angiography
(QCA) represented the state of the art in clinical applications.
Several computer-based systems for quantification of local le-
sions have been developed during this time and are still wide-
spread in clinical use [1]–[8]. However, it became apparent that
systems based on a single projection could not provide reliable
data in the common cases of foreshortening or overlapping.
Consequently, there was an extension from the projection-
based 2-D measurements into the 3-D space. Spatial recon-
structions from biplane angiograms evolved as important tools
for morphological analyses of vessel trees in both coronary and
cerebral domains [9]–[19]. From the known imaging geome-
try and based on the epipolar constraint, any point visible in
both projections can be spatially reconstructed by retracing the
projection rays back to the point of their intersection. Since
the reconstructed rays often miss this point due to slight recon-
struction or calibration errors, usually their closest location is
estimated as approximation (Fig. 1). High-level systems allow
This work has been supported in part by grants Pr 507/1-2 and Wa1280/1-1
of the Deutsche Forschungsgemeinschaft [German Research Society], Bonn,
Germany, and by grants IA-94-GS-65 and IA-96-GS-42 of the American Heart
Association, Iowa Affiliate. Asterisk indicates corresponding author.
A. Wahle is with the University of Iowa, Department of Electrical and Com-
puter Engineering, Iowa City, IA 52242, USA (e-mail: a.wahle@ieee.org).
G. P.M. Prause was with the University of Iowa, Department of Electrical and
Computer Engineering, Iowa City, IA 52242, USA. He is now with the MeVis
Institute at the University of Bremen, D-28359 Bremen, Germany.
S. C. DeJong is with the University of Iowa, Department of Internal Medicine,
Iowa City, IA 52242, USA.
M. Sonka is with the University of Iowa, Department of Electrical and Com-
puter Engineering, Iowa City, IA 52242, USA.
accurate volumetric measurements and an indirect assessment
of diffuse alterations from the morphological relationships be-
tween the vessels within the arterial tree [20]–[23]. A major
drawback of many of these systems is their assumption of el-
liptical cross-sections. Binary reconstruction methods, which
allow the modeling of free-shaped contours from densitometric
profiles, are well-established in ventricle analyses [24]. How-
ever, due to the limited resolution of angiographythey are rarely
used to assess smaller vessels like coronary or cerebral arter-
ies [25]–[27]. As proposed in [28], the geometrically derived
elliptical shape may serve as a basis for binary reconstruction
methods to refine the cross-sectional contour.
Another modality, intravascular ultrasound (IVUS), was in-
troduced carrying the promise to overcome the shortcomings of
angiography. By inserting into the vessel a catheter with an ul-
trasonic transducer in its tip, the exact cross-sectional shape of
the lumen can be visualized and quantified, and the thickness
and composition of the vessel wall and plaque can be deter-
mined [29]–[36]. Acquisitions in 3-D are possible by pulling
the catheter tip back during imaging, thus generating a series of
images at different locations. A new problem arose: IVUS itself
does not provideany information about the location of a specific
image or about its spatial orientation. Common systems simply
perform a straight stacking of adjacent frames, completely ne-
glecting the influence of the vessel curvature (Fig. 2, see also
[37],[38]).
This paper describes a comprehensive system for fusion of
both modalities, i.e. combining the geometrical information as
obtained from biplane angiography with the volumetric data
derived from intravascular ultrasound. The catheter path is ex-
tracted and reconstructed from the biplane angiograms and used
to map the IVUS images to their locations (Figs. 3, 4). Aside
from the localization of the individual IVUS frames in 3-D, the
estimation of their spatial orientations is of major importance.
Examplesof previous workperformed in this areainclude Laban
et al. [39], Evans et al. [40], Pellot et al. [41], and Shekhar et
al. [42].
Our approach incorporates several well-established algo-
rithms for vessel detection in angiograms, for geometrical 3-D
reconstruction, and for IVUS segmentation. It includes a novel
method for the estimation of the frame orientation, which first
determines the relative relations between adjacent IVUS frames
and then optimizes the absolute orientation of the entire frame
set. The methods have been validated in computer and phantom
models, as well as in in-vitro studies.
Figure 5 shows the processing steps for the angiographic im-
ages (described in Section II) and the IVUS data (Section III),
IEEE TRANSACTIONS ON MEDICAL IMAGING, FINAL MANUSCRIPT #187/98, JUNE 30, 1999 2
which are performed in parallel. Their output is then used for
the actual fusion process (Section IV).
II. BIPLANE ANGIOGRAPHY
A. Acquisition and Preprocessing
The first step in angiographicprocessingconsists of the acqui-
sition and digitization of the images. Despite the fact that many
angiographic devices provide digital output in standardized for-
mats, the 35 mm cine film is still a major archivingmedium. This
implies that the images may have to be digitized after selection.
It is well-knownfromconventionalQCAthat the imagingpro-
cessintroducesnumerousgeometricaldistortionsonangiograms
which need to be rectified [6],[43]. In addition, the elimination
of distorting axial rotations and shifts is required for 3-D recon-
structionpurposes[44],[45]. Formoresophisticateddiagnostics,
e.g. volumetric measurements on the vessel lumen, the highest
accuracy in image rectification is required. For the fusion ap-
proach,weonly needa 3-Ddescriptionof thecatheterpath,while
the cross-sectional data are obtained from IVUS. Since calibra-
tion procedures are mostly unacceptable in clinical routine, we
are using an 8-point dewarping method that provides sufficient
accuracy without the need of separately imaging a rectification
grid [46]. The eight lead markers can simply be mounted on the
image intensifiers and are visible in all angiograms (Fig. 6).
B. Geometry Estimation
For the reconstruction process, it is necessary to have an ac-
curate description of the imaging geometry. Figure 7 shows
the five degrees of freedom that may be chosen on a common
Type-II [9] device. The angulation is defined as a combination
of two rotations. The gantry shifts result in a vector
~
I
perpendic-
ular to the projection axis; their point of intersection defines the
distances
D
S
to the X-ray source and
D
I
to the corresponding
image intensifier.
Our approach has been adapted from the reconstruction sys-
tempreviously developed at theGermanHeart InstituteofBerlin,
which is described in detail elsewhere [22],[23]. An initial ge-
ometry is obtained from the parameters as read from the device,
i.e. the angulations and the total distances
D
S
+
D
I
.Sofar,the
geometrical model correspondsto the definition of Wollschl
¨
ager
et al. [9]. Orthogonal projections are desirable, due to a higher
amount of spatial information as compared to low inclination
angles, but are not required. The initial geometry is refined
afterwards from a set of given reference points (at least two).
The reconstructionerrors are analyzed for specific patterns, thus
correction coefficients can be calculated and applied iteratively.
For absolute length measurements, a known reference is
needed. In our phantom studies, we use a wooden ball of known
size (83.5mm in diameter) with two nails as markers. The outer
tips of the nails provide a known distance in 3-D space and can
thus be used for absolute calibration of the scene. They serve
as a reference distance for refinement of the
D
S
:
D
I
ratios,
which are related to the respective magnifications, and the deter-
mination of the shift vectors
~
I
(Figs. 6,7). Of course, any other
markers of reasonable distance can be used for this purpose as
well, e.g. markers on the imaging catheter for in-vivo studies.
C. Extraction of the Catheter Path
After the geometry is known, the transducer in its most distal
location is interactively marked in biplane angiograms acquired
before the pullback is started, as well as in a location at or
proximal to the end of the pullback. To find the correct corre-
spondences,the user issupportedby inserting the projectionrays
as auxiliary lines into the images. Arbitrary intermediate points
following the catheter between its endpoints are set for further
guidance. The catheter is usually visible as a local maximum
along the vessel profiles (Fig. 3) and can thus be extracted us-
ing common approaches of dynamic programming or heuristic
graph search [47],[48].
Our algorithm is based on dynamic programming and allows
free manipulation of the regions of interest (ROIs), which are
derivedfromCatmull-Romsplines[49]throughtheguidepoints.
In the same process, the two edges of the vessel lumen outline
are extracted as well, which serve as reference for establishment
of the absolute frame orientation as presented in Section IV-G.
D. 3-D Reconstruction of Trajectory
The initial reconstruction algorithm as described in [46] has
been replaced by the well-established and validated 3-D recon-
struction algorithm for which details are given in [22],[23]. In
contrast to the reconstruction of a single point, the reconstruc-
tion of two ray bundles resultingfrom theprojected catheter path
requires a correspondence assignment of ray pairs between the
projections.
We are using a cost matrix approach, which is an enhanced
version of the algorithm presented by Parker et al. [50]. As
shown earlier, our algorithm works well even in difficult orienta-
tions of the vessel [23],[51]. However,while the initial algorithm
required a number of corresponding guide points uniquely iden-
tified in both angiograms (e.g. vessel branches), the input for
this modified approach consists only of the pullback start and
end points. The algorithm is further optimized to handle large
vessel segments.
III. INTRAVASCULAR ULTRASOUND
A. Acquisition and Preprocessing
Currently, there are two major kinds of IVUS devices avail-
able [35], mechanically driven catheters and solid-state devices.
Mechanically driven catheters consist of a flexible sheath, a core
which contains the transducer in its tip, and an external motor
which rotates the core. Solid-state devices generate images from
a transducer array and contain no moving parts. The sheathed
design of mechanicallydriven catheters has the major advantage
of a stable pullback path, since only the core is moving in the
direction of the pullback and the sheath remains in its position.
There are common artifacts associated with the mechanical de-
vices, mostly caused by bending of the catheter or other types
of friction [37],[52],[53]. One way to avoid these distortions is
to place the motor close to the tip, thus reducing influences of
the path between the imaging machine and the transducer [31].
Solid-state transducers can avoid distortions from friction ef-
fects almost completely. To ensure a constant-speed pullback,
automated devices are recommended. For in-vivo studies, ECG-
gating and/or respiratory control is required as well [35],[54].
IEEE TRANSACTIONS ON MEDICAL IMAGING, FINAL MANUSCRIPT #187/98, JUNE 30, 1999 3
B. Segmentation
A detailed overview of the common 2-D and 3-D segmenta-
tion and reconstruction systems was recently presented in [35].
Some ‘classic’ approaches include the segmentation method of
Herrington et al. [29] based on simulated annealing, and the
3-D segmentation of Li et al. [30], which performs longitudi-
nal as well as cross-sectional semi-automatic contour detection
based on a minimum cost algorithm. A validation of volumet-
ric quantifications on straight vessels and a feasibility study of
the latter algorithm in clinical applications were performed by
von Birgelen et al. [33],[34].
Weare usingthe well-established segmentationalgorithmpre-
viously described in [32],[36], which is based on a graph-search
approach within a given elliptical region of interest (ROI). This
ROI is automatically adapted from frame to frame. Within each
image, the contours indicating the inner and outer wall borders
are detected, as well as the lumen/plaque interface if applicable
(Fig. 8). In addition, the plaque composition may be determined
for extended analysis. The entire process is automated, except
for the specification of the ROI in the first image, but may be
interrupted and manually corrected at any stage. The segmented
contour lists are kept in 2-D polar coordinates
(
v;
)
.
IV. FUSION OF ANGIOGRAPHY AND IVUS
A. Outline of the Fusion Process
Itisawell-knownproblemthattheconventionalstraightstack-
ing of the IVUS pixel and segmentation data does not deliver
correct results [37],[38]. Figure 2 shows the typical effects of
the stacking: Due to neglecting the vessel curvature, portions
of the vessel volume may be either over- or underestimated,
depending on which side of the vessel they are located; since
the vessel torsion is not considered, the axial rotation (twist)
of the catheter leads to a wrong relation of radial segments be-
tween different images. As shown in Section IV-C, curvature
and torsion of an idealized catheter can be expressed using the
Frenet-Serret formulas, and are a function of the pullback path
of the transducer.
The vessel course is determinedfrom the biplane angiograms,
thusfor eachIVUS frame, its location canbe determined. It must
be kept in mind that the course of the catheter is not identical
with the course of the vessel [55], but the correct assignment of
the IVUS data in 3-D yields correct results even if the frames are
oblique to the vessel. Since a constant angiographic supervision
of the pullbackas proposedin [40]–[42] is notapplicable in clin-
ical cases, the location of a specific frame is directly determined
from its time-stamp and the pullback speed.
The estimation of the absolute orientation of the IVUS frames
in 3-D is a challenging problem. While Evans et al. [40] ne-
glected axial catheter rotations completely, the iterative methods
of Pellot et al. [41] and Shekhar et al. [42] both used a local
match of each individual IVUS frame with the angiographicout-
line. Pellot, however, refined the preliminary contour by using
the densitometric profiles and a-priori information as input for
a combination of Markov random fields and adapted simulated
annealing [28], [41]. In contrast to determining the orientation
individually for each frame, constraints from the Frenet-Serret
formulas can be used to determine the twist between the frames.
Laban et al. [39] implemented the Frenet-Serret formulas di-
rectly, using a Fourier function to approximate the catheter path.
This function satisfied the requirement of a third-order deriva-
tive as implied by the rules. However, the Frenet-Serret formulas
can only deliver the relations between adjacent images relative
to each other. What remains to be determined is the absolute
orientation of the set of frames, which Laban performed itera-
tively by backprojection of the segmented contours and visual
matching of the IVUS data in the angiograms.
Wehavedevelopeda non-iterativeapproach forthedetermina-
tion of the absolute orientation, which combines the analytical
calculation of the catheter twist based upon the Frenet-Serret
formulas with a global optimization of the absolute frame orien-
tation over the entire frame set.
B. Localization of IVUS Frames on 3-D Path
Our approach uses the length determination as described in
[22],[23] for assigning the IVUS frames along the catheter path.
Incontrast toa vessel, which is of variablediameterand stiffness,
the catheter has constant physical properties, which simplify the
length measurement. An arbitrary downsampling rate can be
specified, resulting in the assignment of a fixed number of im-
ages per millimeter pullback. In the current version, a constant
pullback speed is assumed, since landmarks for synchronization
are frequently not available or may be unreliable [46]. If such
landmarks (e.g. clips) are available, the speed between adjacent
synchronization points is assumed to be constant. For the in-
vivo case, ECG-gated image acquisition in combination with a
constant pullback have to be synchronized with the heart phase,
which is subject to certain variabilities. Thus, the acquired
frames would no longer be equidistant. To satisfy this condi-
tion, the algorithm is easily extendible to consider non-uniform
distances between adjacent frames.
C. Catheter Curvature and Torsion
According to a fundamentaltheorem of differential geometry,
a space curve with non-zero curvature is determined up to an
Euclidean transformation by its curvature and torsion [56],[57].
Let
c
:[0
;
1]
!R
3
,
c
(
s
)=[
x
(
s
)
;y
(
s
)
;z
(
s
)]
be the pullback
path of the transducer as a function of the arc length
s
.The
local behavior of the space curve
c
may be described by the
Frenet frame, a righthanded trihedron of three orthonormalvec-
tors
~
t
(tangent),
~n
(normal), and
~
b
(binormal). The Frenet-Serret
formulas express the local change of the Frenet frame in terms
of the frame itself:
~
t
0
(
s
) = +
(
s
)
~n
(
s
) (1)
~n
0
(
s
) =
,
(
s
)
~
t
(
s
)+
(
s
)
~
b
(
s
) (2)
~
b
0
(
s
) =
,
(
s
)
~n
(
s
) (3)
where the prime denotes the derivative with respect to arc
length
s
. The curvature
and the torsion
are the angular
velocities of the vectors
~
t
and
~
b
, respectively, as the Frenet frame
is moved along
c
according to
s
. They calculate to:
(
s
) =
k
c
00
(
s
)
k
(4)
(
s
) = det[
c
0
(
s
)
;c
00
(
s
)
;c
000
(
s
)]
2
(
s
) (5)
IEEE TRANSACTIONS ON MEDICAL IMAGING, FINAL MANUSCRIPT #187/98, JUNE 30, 1999 4
Based upon this theory, we havedevelopedan analyticalcatheter
model as well as a sequential triangulation method for determi-
nation of catheter curvature and torsion during the pullback.
D. Analytical Model of the Catheter
The behavior of an imaging catheter was estimated analyti-
cally, resulting in the following geometrical model: We assume
that the catheter consists of a chain of generalized joints, which
are torsion-free (i.e. do not introduce any other axial rotation
on bending than specified by the Frenet-Serret formulas). Each
joint performs a relative transformation, resulting in an absolute
position and orientation after applying all transformations on an
initial position of the transducer. Furthermore, some constraints
are given. The imaged frame is always perpendicular to the
catheter axis, and the initial axial orientation is fixed (i.e. no
axial twist is applied). Thus, three parameters are given for each
of these joints. The length
t
describes the distance of the end-
points when the joint is straight, an angle
#
gives the amount of
bending at this joint, and a second angle
!
specifies the axis of
rotation. A non-discrete version of this model can be derived for
t
!
0
, yielding an infinite number of joints within the chain.
As shown in Fig. 9, the catheter segment between two ad-
jacent points
C
i
,
1
and
C
i
can be described by the following
transformations, considering that each point is associated with
both location and orientation:
1. the previous location is translated by
t
i
=
2
;
2. the orientation is axially rotated by
+
!
i
;
3. the orientation is rotated by
#
i
around a fixed axis local to
the current orientation (i.e. the rotation axis was affected by
the previous axial rotation);
4. the axial rotation is compensated by applying
,
!
i
;
5. the location is translated by another
t
i
=
2
.
Equations (6)–(8) show the resulting transformation matrices
using homogeneous coordinates.
C
0
= [
x; y ; z ; h
] =
h
x
h
;
y
h
;
z
h
i
(6)
C
i
=
C
i
,
1
M
(
#
i
;!
i
;t
i
)
=
C
0
i
Y
j
=1
M
(
#
j
;!
j
;t
j
) (7)
The local curvature
i
in joint
i
is directly related to
#
i
,
while the local torsion
i
dependson the difference
!
i
between
bending directions
!
i
and
!
i
,
1
of adjacent joints.
E. Estimation of the Relative Twist
In accordance with the catheter model described in the previ-
ous section, a sequential triangulation method is used to deter-
mine the relative twist between adjacent IVUS frames (Fig. 10).
This can be considered as a discrete approximationof the Frenet-
Serret formulas where the local torsion
i
is calculated as the
angle between the normal vectors of two adjacent triangles in
the reconstructed pullback polygon.
Each frame is defined by the location of its center and a local
2-D coordinate system. For time instances
i
and
i
+1
, both
frames are located halfway between three consecutive points
P
i
;P
i
+1
;P
i
+2
so that:
S
i
= (
P
i
+
P
i
+1
)
=
2 (9)
S
i
+1
= (
P
i
+1
+
P
i
+2
)
=
2 (10)
The frames are perpendicular to the tangent vectors:
~
t
i
=
P
i
+1
,
P
i
(11)
~
t
i
+1
=
P
i
+2
,
P
i
+1
(12)
The center of the circumscribed circle of the triangle as defined
by:
T
i
= (
P
i
;P
i
+1
;P
i
+2
) (13)
is determined as the intersection of the perpendicularbisectorsof
the tangent vectors
~
t
i
and
~
t
i
+1
. The orientation of frame
i
+1
is
determined by rotating frame
i
by the enclosed angle
i
(which
reflects the local curvature
i
) around the normal vector:
~n
i
=
~
t
i
~
t
i
+1
(14)
of the triangle
T
i
. Finally, the center of frame
i
+1
is shifted
to point
S
i
+1
. In the special case of the collinearity of
(
P
i
;P
i
+1
;P
i
+2
)
, the circle has an infinite radius, i.e.
i
=0
,
and thus no twisting takes place.
F. Quantification of the Relative Twist
Following the calculation of the catheter twist and the changes
in IVUS frame orientation, all frames have a fixed orientation
relative to the first frame in terms of a scalar twisting angle, even
though the absolute orientation is yet unknown. To estimate the
amount of twisting, i.e. the presumed error if the torsion is not
considered during the reconstruction, we are using a reference
plane (Fig. 11).
This reference plane is generated by bilinear regression from
the catheter path data and is thus the plane of best approximation
in a least square sense. The torsion-loadedpath is then projected
onto this plane, creating another path that retains the curvature
of the catheter, but which is torsion-free. The frames for the
torsion-free copy are generated in the same way as performed
for its torsion-loaded original. Afterwards, the 2-D orienta-
tion vectors of the torsion-loaded path are projected onto the
corresponding torsion-free frame, and their difference angle de-
termined. Finally, the difference angles are normalized to be
zero for the first frame of the pullback.
G. Estimation of the Absolute Orientation
After establishing the relativeorientation changes between the
frames, the absoluteorientationin3-D remains ambiguous. This
problem is comparable to fitting a sock on a leg [39]: While the
leg (catheter path) is stable, the sock (axial orientation of the
frame set) can be freely rotated around the leg, but fits optimally
only in one orientation. Our method uses the bending behavior
of the imaging catheter as a reference, which is expected to fall
in the position of minimum energy within the vessel [55]. This
results in an out-of-center position of the catheter relative to the
innerlumen,whichis visiblein bothangiographicandIVUSdata
(Figs. 3,8). Based on this out-of-center position, a correction
angle can be calculated.
IEEE TRANSACTIONS ON MEDICAL IMAGING, FINAL MANUSCRIPT #187/98, JUNE 30, 1999 5
M
(
#; ! ; t
) =
2
6
6
4
cos
#
,
sin
#
sin
!
,
sin
#
cos
!
0
sin
#
sin
!
cos
#
sin
2
!
+ cos
2
!
(cos
#
,
1) sin
!
cos
!
0
sin
#
cos
!
(cos
#
,
1) sin
!
cos
!
cos
#
cos
2
!
+ sin
2
!
0
t
2
(cos
#
+1)
,
t
2
sin
#
sin
!
,
t
2
sin
#
cos
!
1
3
7
7
5
(8)
First, the segmented IVUS lumen contour is mapped into
3-D using an initial orientation. For each frame location, the
angiographic outline is reconstructed as an elliptical contour as
described in [22],[23], and mapped onto the IVUS frame at the
respective transducer location (Fig. 12). For both angiographic
and IVUS outlines, 3-D out-of-centervectors are generated from
the contour center to the catheter position:
~
a
=
p
cath
,
p
center
(15)
~
v
=
m
cath
,
m
centroid
(16)
where
~
a
is derived from the angiographic reconstruction of
the catheter point
p
cath
and the ellipse center
p
center
,and
~
v
is
derived from the mapped IVUS frame center
m
cath
indicating
the catheter location and the centroid
m
centroid
of the contour.
For each frame
i
, the out-of-centerstrength
i
andasignederror
angle
'
i
with respect to the initial orientation can be calculated:
i
=
k
~
v
i
k
(17)
'
i
=
6
(
~
a
i
;
~
v
i
) (18)
Within a moving window of arbitrarybut fixed width
w
, a statisti-
calanalysis is performed. For
n
f
frames,
n
w
=
n
f
,
(
w
,
1)
win-
dow locations exist. For each window location
k
, the summed
out-of-center strength
k
, the weighted mean
'
k
,andthe
weighted standard deviation
(
'
k
)
of the difference angle are
calculated:
k
=
k
+(
w
,
1)
X
i
=
k
i
(19)
'
k
=
1
k
k
+(
w
,
1)
X
i
=
k
i
'
i
(20)
(
'
k
)
2
=
1
k
k
+(
w
,
1)
X
i
=
k
i
(
'
i
,
'
k
)
2
(21)
From these values, a reliability weight is calculated for each lo-
cation of the moving window, giving higher weight to locations
with high out-of-center strength (i.e. locations with increased
significance for the estimation of the correction angle), and lim-
iting those with a high standard deviation of the difference angle
function (i.e. locationsshowingdistortionsin eitherangiographic
or IVUS lumen outlines):
r
k
=
k
(
'
k
)
(22)
A single correction angle
'
corr
is determined and applied to all
IVUS frames relative to the initial orientation:
r
=
n
w
,
1
X
k
=0
r
k
(23)
'
corr
=
1
r
n
w
,
1
X
k
=0
r
k
'
k
(24)
H. 3-D Mapping of the IVUS Data
The frames generated during the previous steps may be used
in many ways: For a fast visualization, which does not even re-
quirea segmentationof the IVUSdata, thepixeldata are mapped
into the frames; furthermore, the segmented contour and plaque
composition information can be mapped into 3-D space as well
by locating their corresponding pixels. The points of the seg-
mented and mapped IVUScontourscanbe connectedto asurface
model by triangulation and then be displayed, e.g. by using the
standardized Virtual Reality Modeling Language (VRML) with
a respective viewer.
A voxel cube of pre-defined size can be generated and used
as input for common volume-oriented systems. Therefore, the
3-D space is divided into cubicvoxels. After assigning a specific
IVUS pixel to its 2-D coordinates, and mapping into 3-D space,
the corresponding voxel can be determined. Depending on the
size of the cube,it may occurthat two or moreIVUS pixels share
the same voxel, or that a single IVUS pixel extends over more
than one voxel.
The resolution of the voxel cube depends on several factors.
The basic limitation is the resolution of the image data (both
angiographic and IVUS), as well as the distance between the
frames. Furthermore, the dimensions of the voxel cube are re-
stricted by computerperformanceandmemory. Typical sizes are
380
380
380 voxels, e.g. allowing a resolution around 150–
180
m for a curved vessel segment of 100–120mm length, or
of 50
m for assessing a local lesion of 12–15mm length.
V. VALIDATION
A. Computer Simulations
For validation in an idealized environment, computer simula-
tions were performed. Since the algorithm was tested on regular
structures like helices before [55], we focused on models of
known, but irregular bending. The first model consisted of five
generalized joints as describedin SectionIV-D. To check its cor-
rectness, a physicalmodel with five joints was built and supplied
with markers indicating the orientation of the local 2-D coor-
dinate systems. Both computer and physical models were bent
with the same sets of angles
(
!
i
;#
i
)
. Figure 13a shows some
examples. In the following step, the sequential triangulation al-
gorithm was applied to the 3-D path as indicated by points
P
i
,
and thematched 2-Dcoordinatesystems
(
U
i
;V
i
)
were compared
with the originally computed ones.
To simulate the non-discretecase, an irregular path was gener-
ated. Twosine-functionsof differentamplitudesand frequencies
were applied to the angles
!
and
#
, thus changingboth local cur-
vatureand torsionineveryjoint. A totalpath of10 cm lengthwas
