scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Virtual fixture control of a hybrid parallel-serial robot for assisting ophthalmic surgery: An experimental study

02 Oct 2014-Vol. 1, pp 732-738
TL;DR: The virtual fixture control methods for a hybrid parallel-serial micromanipulator, which is designed for assisting ophthalmic surgeons, and the virtual fixture using this mechanism enables the user to adjust them even during the procedure, are presented.
Abstract: This paper presents the virtual fixture control methods for a hybrid parallel-serial micromanipulator, which is designed for assisting ophthalmic surgeons. Virtual fixtures are features of surgical robotic setups to improve quality of the surgery and reduces the operation risk. In the domain of ophthalmic surgery lack of virtual fixtures in manual operations has limited, and sometimes even blocked, the treatment options. The contribution of this paper is concept analysis and implementation of flexible virtual fixture for the novel hybrid parallel-serial mechanism and experimentally evaluation of this concept. The virtual fixture using this mechanism enables the user to adjust them even during the procedure. Pivoting around a Remote Center of Motion (RCM), which in retinal surgery is the incision point, is the most famous virtual fixture of ophthalmic surgery. Autonomous RCM adjustment for Vitreo-Retinal surgery, implying retinal reachability study, is the secondary contribution which is investigated in this paper.

Summary (3 min read)

Introduction

  • Since the 1970s when the first pars plana vitrectomy was performed [1], there has been an important change in trends in ophthalmic operations, particularly in VitreoRetinal surgery.
  • For instance Remote Center of Motion (RCM) guarantees that during the Vitreo-Retinal surgery there won’t be a damage to sclera.
  • In this method the kinematics of the robot is designed in a way that physically limits the motions.
  • 2- Semi Mechanically constrained virtual fixtures (see e.g. [4]): Flexibility of the virtual fixtures plays an important role for Vitreo-Retinal surgery.

B. Model of the Eye

  • The cannula for inserting the tool into the eye is placed about 3.5mm away from the cornea [9] and defines the location of the RCM.
  • Additionally, another constraint is taken into account: because the surgeon uses the microscope during intra-ocular operations, it is essential that the areas which are accessible overlap with the field of view of the microscope.
  • The radius of the circular hole is set to the average radius of the cornea.
  • Fig. 3 shows the relevant parameters of the model.
  • By tilting the eye about its center the RCM is relocated and the line of sight transforms accordingly as displayed in Fig.

III. METHODOLOGY

  • The restriction of motions through the entry point of the patient’s body is one of the most important characteristics of minimally invasive surgery assisted by robots.
  • More specifically, the link penetrating the tissue is only allowed to translate along its axis and rotate about the entry point.
  • As the surgeon uses the microscope to look at the retina through the widened pupil the position of the eye should be maintained while moving the tooltip.
  • The first is based on the concept of taskpriority [10] as implemented in [8] and the second on the alternative kinematics [11] (also referred to as extended Jacobian footnote distinguishing [12]) approach as shown in [7] [14].
  • With the augmented kinematics applied in the control design further singularities can be introduced due to rank deficiency or linear dependencies in the sub matrix JRCM [12].

IV. DISCUSSION

  • Considering only translational movement, that is parallel movement of the first four piezo positioners with travel ranges of ±15mm, the workspace is limited by a cube of dimensions 30× 30× 30 mm, also known as Reachability analysis.
  • It is desirable that most of the points within the workspace are also accessible when enforcing an RCM constraint.
  • In their analysis the authors assume that by rotating the eye about its center the RCM location can be automatically determined.
  • For determining the best location of the eye with respect to the robot base as well as the necessary tilt of the robot the authors apply the following method: First, the location is determined experimentally until it yields satisfactory results within a certain predefined range of rotations of the eye.
  • Green dots would indicate areas that are visible and reachable.

V. RESULTS

  • The candidates obtained for each tilt with the best overlap of visible and reachable areas, which the authors will refer to as coverage or performance, are chosen as the center of a sphere with diameter of 10mm.
  • Candidates within this neighbourhood are considered by their analysis and are illustrated in Fig. 7 within the workspace.
  • The exact locations of the candidates with the best performance are listed in Table I. Fig. 8 illustrates the average coverage and the standard deviation for each configuration within the region of interest.
  • The deviation is also very high and at worst only a coverage of around 45% is obtained.
  • Thus the regions of interest obtained in the analysis should be considered as where to roughly place the eye.

VI. EXPERIMENTAL EVALUATION

  • The elaborated model and its results are evaluated experimentally using the simulation environment and the area that can be reached experimentally is examined.
  • The RCM is set manually, which inhibits to place it exactly at the position where the maximum coverage could be obtained.
  • Fig. 10 shows the top view on the determined areas on the retina for the best position of the eye with no tilt of the robot for each considered rotation.
  • Looking at the percentage of reachable areas within the visible area vM and vE , the model yields better results than those obtained during the evaluation run.
  • By pushing the button on the 3D mouse, the coordinates of the marker are recorded and after the fourth registration procedure the parameters of a sphere can be determined as explained in the following.

A. Clinical Experiments

  • Ex-vivo clinical experiments was performed at ophthalmic operation theater of klinikum rechts der isar, Munich (See Fig. 11).
  • Using the standard 23G trocar system the cannula was docked into the porcine eye.
  • The RCM point (incision point) was detected at the beginning of insertion phase and it was chosen by pressing a bottom on input device.
  • There is also the ability of changing the RCM point by pressing the bottom, moving it to the new position (e.g. for eye rotation) and selecting of the new point by pressing the bottom once more.
  • Fig. 12 shows the error from the raw position data in x, y and zdirection, which was captured during clinical experiments.

VII. CONCLUSION

  • The virtual fixture control of the hybrid parallel-serial micromanipulator has been investigated in this paper.
  • The feasibility of this feature, the method of implementation, modeling and experimental validation are the elements of this investigation.
  • It has also been shown that the distribution of visible-accessible intraocular points is highly dependent on the location of the incision point and the robot.
  • This dependency and the optimum locations of the robot, with respect to the eye, have been also discussed in this work.
  • The virtualfixture control methods was simulated and experimentally evaluated.

Did you find this useful? Give us your feedback

Figures (15)

Content maybe subject to copyright    Report

Virtual Fixture Control of a Hybrid Parallel-Serial Robot for Assisting
Ophthalmic Surgery: an Experimental Study
M. A. Nasseri
1
, P. Gschirr
1
, M. Eder
1
, S. Nair
1
, K. Kobuch
2
, M. Maier
2
, D. Zapp
2
, C. Lohmann
3
and A. Knoll
2
Abstract This paper presents the virtual fixture control
methods for a hybrid parallel-serial micromanipulator, which
is designed for assisting ophthalmic surgeons. Virtual fixtures
are features of surgical robotic setups to improve quality of
the surgery and reduces the operation risk. In the domain of
ophthalmic surgery lack of virtual fixtures in manual opera-
tions has limited, and sometimes even blocked, the treatment
options. The contribution of this paper is concept analysis and
implementation of flexible virtual fixture for the novel hybrid
parallel-serial mechanism and experimentally evaluation of this
concept. The virtual fixture using this mechanism enables the
user to adjust them even during the procedure. Pivoting around
a Remote Center of Motion (RCM), which in retinal surgery
is the incision point, is the most famous virtual fixture of
ophthalmic surgery. Autonomous RCM adjustment for Vitreo-
Retinal surgery, implying retinal reachability study, is the
secondary contribution which is investigated in this paper.
I. INTRODUCTION
Since the 1970s when the first pars plana vitrectomy
was performed [1], there has been an important change
in trends in ophthalmic operations, particularly in Vitreo-
Retinal surgery. Not only have the outcomes of these surg-
eries been improved, but nowadays, it is also possible to
find cure to ocular conditions that were untreatable before.
The increasing positive results in the ophthalmic surgery are
mostly due to new and better developed surgical techniques,
improved low-gauge instrumentation, high-speed cutters and
upgraded and enhanced visualization tools. Nevertheless, the
success of these procedures is still limited by the surgeons’
precision and dexterity. In this line, it is the employment
of assisting robots what sets a break through the barrier
of human abilities. The abilities barrier which is discussed
in this paper is virtual fixture control. To maximize the
quality and safety of operations the virtual fixtures should
be realized. For instance Remote Center of Motion (RCM)
guarantees that during the Vitreo-Retinal surgery there won’t
be a damage to sclera. Or the collision avoidance of lens
and retina helps the surgeons to avoid unwanted touching
of sensitive regions (See Fig.1). A clinically acceptable
*This work was supported by TUM Graduate School of Information
Science in Health
1
M. Ali Nasseri, P. Gschirr, S. Nair, M. Eder and A. Knoll are
with the Department of Robotics and Embedded Systems, Institut f
¨
ur
Informatik, Technische Universit
¨
at M
¨
unchen nasseri, gschirr,
nair, ederma, knoll at in.tum.de
2
K. Kobuch, M. Maier, D. Zapp and C. P. Lohmann are with
the Augenklinik rechts der Isar, Technische Universit
¨
at M
¨
unchen
karin.kobuch, mathias.maier, daniel.zapp,
c.lohmann at mri.tum.de
Fig. 1. Ophthalmic tool introduced into the eye ball using micro cannula
(Vitreo-Retinal surgery). The most famous virtual fixtures are marked
surgical robot should be able to satisfy virtual fixtures. This
experimental study investigates the virtual fixture capabilities
of the ophthalmic micromanipulator from Technische Uni-
versit
¨
at M
¨
unchen. This tiny robot which is introduced in [2]
has a hybrid parallel-serial configuration and is specifically
designed for Vitreo-Retinal surgery.
Related Works: Virtual fixture is a common problem in
Minimally Invasive Surgery(MIS). There are three methods
to solve virtual fixture problems in the robotics literature;
1- Mechanically constrained virtual fixtures (see e.g. [3]):
In this method the kinematics of the robot is designed in
a way that physically limits the motions. This method, due
to hardware constraints, is considered to have the highest
reliability and safety. However, it has no flexibility and con-
sequently is not intuitive. 2- Semi Mechanically constrained
virtual fixtures (see e.g. [4]): In this method the kinematics
of the robot provides dependencies which together with
controller satisfies virtual fixture constraints. This method is
more flexible than the first one but there still are degrees of
freedom reductions which limits the free motion. 3- Control
based virtual fixtures (see e.g. [5]): Normally in this method
the robot’s kinematics is capable of 6DOF motion. The
control algorithms are used to define virtual fixtures when
it is needed. It provides the maximum flexibility but the
reliability of the controller should be carefully verified for
2014 5th IEEE RAS & EMBS International Conference on
Biomedical Robotics and Biomechatronics (BioRob)
August 12-15, 2014. São Paulo, Brazil
978-1-4799-3128-6/6/14/$31.00 ©2014 IEEE 732

Fig. 2. TUM Ophthalmic Micromanipulator (right) and Simulator (left):
Serial configuration of parallel joints
safety reasons.
Flexibility of the virtual fixtures plays an important role
for Vitreo-Retinal surgery. During the operation the surgeon
needs to change the virtual fixtures. For instance for the
needle approach and insertion phase the surgeon needs at
least ve degrees of freedom to locate the needle. When
the needle is inserted into the microcannula the RCM needs
to be set at the incision point. The RCM position needs to
be changed several times to keep the line of sight of the
microscope and meanwhile access the points of interest on
the surface of the retina. The surgeon should also be able
to enable/disable other virtual fixtures (Tool-retina distance,
Lens collision avoidance, ...). The contribution of this paper
is investigating the capability of the robot from [2] for
realizing controlled based virtual fixtures. The investigation
includes the mathematical modeling as well as the exper-
imental evaluation. Furthermore, the reachability analysis
result, which is discussed in this paper, is used as a reference
for automatic adjustment of the RCM.
Organization of this Paper: The remainder of this paper
is structured as follows: In section 2 the models of the Eye
and the Robot are explained. The methods for solving virtual
fixture problems is described in section 3. The implemented
methods are discussed in section 4 which is followed by
results in section 5. Section 6 describes the experimental
evaluation and clinical experiments and this paper is going
to be concluded in section 7.
II. MODELS
A. Model of the Robot
Fig.2 shows the robot which is used in this study. It
consists of a novel serial configuration of parallel coupled
joint mechanisms. The detailed kinematics and mechanical
models of the robot was analyzed in [2], in this paper the
feasibility of the adjustable RCM is discussed. Five sub mi-
cron precision piezo actuators are used to drive the robot and
it is controlled using a middle-ware based architecture [6].
B. Model of the Eye
The Eye ball in this work is modeled as a simple sphere
with a diameter of 24.2mm [13]. The cannula for inserting
the tool into the eye is placed about 3.5mm away from the
cornea [9] and defines the location of the RCM. Additionally,
another constraint is taken into account: because the surgeon
uses the microscope during intra-ocular operations, it is
essential that the areas which are accessible overlap with
the field of view of the microscope. For this it is assumed
that the inner eye is visible through a circular hole defined
by the pupil without refractions of the lens. The radius of the
circular hole is set to the average radius of the cornea. Fig. 3
shows the relevant parameters of the model. The diameter of
the eyeball and the cornea are taken from [13].
By tilting the eye about its center the RCM is relocated
and the line of sight transforms accordingly as displayed in
Fig. 5. In our analysis we assume a maximum rotation of
the eye 30 degrees about the z-axis, maximum rotation
in the direction of the cannula (around the x-axis) 15
degrees and the maximum rotation around the y-axis ±10
degrees.
III. METHODOLOGY
The restriction of motions through the entry point of the
patient’s body is one of the most important characteristics
of minimally invasive surgery assisted by robots. More
specifically, the link penetrating the tissue is only allowed
to translate along its axis and rotate about the entry point.
This reduces stress on the tissue and thus accelerates the
healing process after the surgery. For ophthalmic-Vitreo-
Retinal surgery the RCM is especially useful to maintain
the eyeball in a certain position to perform operations on
the retina. As the surgeon uses the microscope to look at
the retina through the widened pupil the position of the
eye should be maintained while moving the tooltip. This
is only possible by restricting the motion of the last link
with respect to the entry point on the sclera. We follow the
general approach in [7] and adapt it to our setting. One major
advantage of this approach in comparison to [8] is that it does
not require the definition of a tangent plane at the entry point.
For this setup the RCM is, but not necessarily, located on
the axis of the tool shaft (See Fig.1) and its position can be
written as:
p
rcm
= p
5
+ λ(p
tool
p
5
) (1)
Deriving the equation with respect to time yields:
˙
p
rcm
=
˙
p
5
+ λ(
˙
p
tool
˙
p
5
) +
˙
λ (p
tool
p
5
) (2)
Given the Jacobians J
rcm
and J
5
at the respective points p
rcm
and p
5
, the equation can be reformulated by making use of
24.2
17.4
11.8
Z
Y
X
Y
Fig. 3. The eye model used in our analysis.
733

Fig. 4. Evaluation run in the simulation environment. The grey dot indicates
the point on the retina the tool is aimed at.
X
Y
p
5
p
rcm
p
tool
X
Y
p
5
p
rcm
p
tool
Fig. 5. Different rotations of the eye, Left: 10
about z-axis, Right: 30
about z-axis.
the fact that
˙
p
i
= J
i
˙
q:
˙
p
rcm
= J
5
˙
q +λ (J
tool
˙
q J
5
˙
q) +
˙
λ (p
tool
p
5
) (3)
In matrix form this can be written as
˙
p
rcm
=
J
5
+ λ(J
tool
J
5
)
p
tool
p
5
T
˙
q
˙
λ
= J
rcm
˙
q
˙
λ
(4)
With this formulation it is possible to restrict the motion of
the RCM by forcing its velocity to zero i.e.
˙
p
rcm
= J
rcm
˙
q
˙
λ
!
= 0
3×1
(5)
with J
rcm
R
3×6
. By fulfilling the RCM constraint the
degrees of freedom of the robot are reduced by two, i.e. for
fulfilling a task in an n
t
-dimensional space the robot must
have at least n n
t
+ 2 degrees of freedom [7].
Control Design: To satisfy the RCM constraint while
moving the tooltip within the eye:
˙
q = J
Ke (6)
For incorporating the RCM constraint two methods are
usually used. The first is based on the concept of task-
priority [10] as implemented in [8] and the second on the
alternative kinematics [11] (also referred to as extended
Jacobian footnote distinguishing [12]) approach as shown in
[7] [14].
In this work the second approach was followed; using the
alternative kinematics where the RCM constraint can be
directly incorporated into the robot task. The robot has
five degrees of freedom from which two are used to fulfill
the RCM constraint. As a result there are three degrees of
freedom left, which match the three-dimensional workspace
necessary to place the tooltip within the eyeball.
The robot task with coordinates x
t
=
x φ ψ
T
is ex-
pressed. The task space is defined by T = R × S
2
. Hence
the robot task is defined as the Cartesian x position and the
two angles φ and ψ. The joint velocities are related to the
task velocities through:
˙
x
t
= J
t
˙
q (7)
where J
t
R
3×6
is the Jacobian derived from the kinematic
mapping. Taking the RCM constraint into account the ex-
tended task can be defined as:
x
ext
=
x
T
t
p
T
rcm
T
=
x φ ψ x
rcm
y
rcm
z
rcm
T
R × S
2
× R
3
(8)
And the kinematics of the extended task are then given by:
˙
x
ext
=
J
t
0
3×1
J
rcm
˙
q
˙
λ
= J
ext
˙
q
˙
λ
(9)
Assuming a desired robot task x
d
and the position of the
RCM, which is defined by the location of the trocar p
trocar
the extended task error is given by:
e
ext
=
x
ext
x
d
p
trocar
p
rcm
(10)
Similar to the unrestricted movement the kinematic control
is written as:
˙
q
˙
λ
= J
ext
K
t
0
3×3
0
3×3
K
rcm
e
ext
(11)
where K
t
and K
rcm
are both postive definite diagonal
matrices in R
3×3
. The control law guarantees decoupled
exponential convergence of the task to the desired value [7].
Algorithmic Singularities In the case of unconstrained
movement it was already shown in [2] that the robot does
not have any singularities within its workspace. However,
with the augmented kinematics applied in the control design
further singularities can be introduced due to rank deficiency
or linear dependencies in the sub matrix J
RCM
[12]. These
are called algorithmic singularities and can be avoided by
choosing the kinematic functions. In our control design we
chose the angles ψ and φ instead of y and z coordinates to
avoid algorithmic singularities within the robot’s workspace.
IV. DISCUSSION
Reachability analysis: For the robot to be used in practice
it is essential that its workspace covers the regions of the eye
that have to be accessible during a surgical procedure.
Considering only translational movement, that is parallel
movement of the first four piezo positioners with travel
ranges of ±15mm, the workspace is limited by a cube
734

of dimensions 30 × 30 × 30 mm. By assuming an average
diameter of the eye of 24.2mm this seems to be sufficient.
Effect of the RCM constraint: It is desirable that most
Fig. 6. Reachability Analysis, Blue, Red and green dots are showing
Visible, Accessible and Visible-Accessible points respectively
of the points within the workspace are also accessible when
enforcing an RCM constraint. However, enforcing an RCM
constraint limits the workspace considerably depending on
where it is placed. This is obviously due to the fact that
two degrees of freedom are lost. Fig. 6 shows the effect
of enforcing the RCM constraint at different locations. As
a result, in order to access different regions of the retina
the RCM location needs to be changed during a surgery
accordingly. Another property that is connected to this issue
is that in which configurations more visible-accessible points
can be reached.
Reachability within the eye: To further assess the prac-
ticality of the robotic setup the reachability within the
eye is investigated under different RCM constraints. More
specifically the location of the eye with respect to the robot
base is investigated, which then automatically defines the
location of the RCM on the eye. Moreover, we also include
the tilt of the robot about the z-axis. In our analysis we
assume that by rotating the eye about its center the RCM
location can be automatically determined.
Best location of the eye with respect to the robot base:
For determining the best location of the eye with respect to
the robot base as well as the necessary tilt of the robot we
apply the following method:
First, the location is determined experimentally until it
yields satisfactory results within a certain predefined
range of rotations of the eye.
Second, the workspace of the robot is sampled. The
sampled points are taken as an input for possible RCM
locations and compared to the previously experimentally
determined location.
In doing so the eye model is rotated by 10 to 30 degrees
about the z-axis and 10 to 10 degrees about the y-axis.
Intuitively the best flexibility of the robot is within the central
30×30×30 mm cube and thus an offset of the eye that covers
Fig. 7. Locations with the best performance for every tilt within the
workspace. The marked angles are the tilt of the robot base with respect to
the eye
this part of the workspace should yield good results. More
specifically, the center of the eye is set to the location of
the tooltip of the robot where all its positioners are in zero
position. From this the RCM location is calculated and the
analysis of reachable as well as visible areas within the eye
is conducted. Fig. 6 shows the result of the analysis. The
red dots represent areas that can be reached by the tool tip
of the robot. Blue dots are those which are visible through
the simplified circular hole. Green dots would indicate areas
that are visible and reachable.
Comparison to other RCM candidates: To compare
the reference with other candidates we sample the robot’s
workspace by positioning the ve actuators in 3mm steps in
their working range of ±15mm and calculate the position of
the end effector via forward kinematics. This yields 11
5
=
161051 points. To get rid of configurations that yield much
worse results than our reference we first filter the workspace
by analysing the reachability of the robot without rotating
the eye but by tilting the robot about the z-axis within the
range of 0 30 degrees with 10 degree steps. The filtered
candidates are then more closely analysed within the full
rotation range of the eye.
V. RESULTS
The candidates obtained for each tilt with the best overlap
of visible and reachable areas, which we will refer to as
coverage or performance, are chosen as the center of a sphere
with diameter of 10mm. Candidates within this neighbour-
hood are considered by our analysis and are illustrated in
Fig. 7 within the workspace. The exact locations of the
candidates with the best performance are listed in Table
I. Fig. 8 illustrates the average coverage and the standard
deviation for each configuration within the region of interest.
The results are summarized in Table II. All values correspond
735

TABLE I
POINTS WITH THE BEST PERFORMANCE IN EACH TILT CONFIGURATION.
ref 0
10
20
30
x -54.19 -22.19 -31.15 -39.84 -46.96
y 82.50 104.68 98.62 90.03 78.66
z 54.50 55.90 55.83 55.90 56.10
to the percentage of overlap of visible and reachable points.
As the results show, the best coverage of our reference
is similar to those obtained by the analysis. However, the
deviation is also very high and at worst only a coverage of
around 45% is obtained. i.e. the region around the reference
is very volatile. In a practical setting were the location of the
eye cannot be positioned exactly at the desired coordinates
this is crucial. Thus the regions of interest obtained in the
analysis should be considered as where to roughly place the
eye.
TABLE II
QUANTITATIVE RESULTS OF THE ANALYSIS.
ref 0
10
20
30
Mean 0.65 0.88 0.91 0.90 0.80
Deviation 0.11 0.03 0.02 0.03 0.03
Max 0.89 0.84 0.92 0.95 0.95
Min 0.42 0.70 0.80 0.84 0.84
VI. EXPERIMENTAL EVALUATION
The elaborated model and its results are evaluated experi-
mentally using the simulation environment and the area that
can be reached experimentally is examined. The RCM is set
manually, which inhibits to place it exactly at the position
where the maximum coverage could be obtained. After fixing
the RCM the needle is inserted into the eye and moved
into all directions along the retina as far as possible. The
positions are recorded and it is indicated whether the retina
can be touched with the current orientation of the robot. One
evaluation run for the best position without tilting the robot
and no rotation of the eye is shown in Figure 9. The red
area indicates the positions on the retina that can be reached.
cove r age
reference
0
10
20
30
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig. 8. Average performance of candidates and their standard deviation.
95 100 105 110 115
46
48
50
52
54
56
58
60
62
64
66
Y
Z
Fig. 9. Recorded trajectory of reachable points on the retina from one
evaluation run. The shaded region indicates the reachable area derived from
the trajectory.
This procedure is repeated for all configurations listed above,
including rotations of the eye about the z-axis with 10, 0
and 30 degrees. Then the area which was reached during the
experiment is compared to the prediction from the model as
presented in the former section. For simplicity, the areas are
projected onto a plane.
The following indicators are used to quantitatively asses the
results:
Overlap of reachable areas r:
o
r
=
M
r
E
r
M
r
E
r
(12)
where M
r
defines the reachable area predicted by the
model M and E
r
the area reached during the evaluation
run E.
95 100 105 110 115
46
48
50
52
54
56
58
60
62
64
66
Y
Z
Fig. 10. The visible and reachable areas predicted by the model and
determined during the evaluation run for the best location of the eye without
tilting the robot(Green: Visible area, Blue: Reachable area from model and
Red: reachable area from evaluation.
736

Citations
More filters
Journal ArticleDOI
TL;DR: This data indicates that intraocular surgical steps continue to be manually performed because existing technologies are unable to perform complete, multi‐step procedures from start to finish.
Abstract: Background: Since the advent of robotic-assisted surgery, the value of using robotic systems to assist in surgical procedures has been repeatedly demonstrated. However, existing technologies are unable to perform complete, multi-step procedures from start to finish. Many intraocular surgical steps continue to be manually performed. Methods: An intraocular robotic interventional surgical system (IRISS) capable of performing various intraocular surgical procedures was designed, fabricated, and evaluated. Methods were developed to evaluate the performance of the remote centers of motion (RCMs) using a stereo-camera setup and to assess the accuracy and precision of positioning the tool tip using an optical coherence tomography (OCT) system. Results: The IRISS can simultaneously manipulate multiple surgical instruments, change between mounted tools using an onboard tool-change mechanism, and visualize the otherwise invisible RCMs to facilitate alignment of the RCM to the surgical incision. The accuracy of positioning the tool tip was measured to be 0.205±0.003 mm. The IRISS was evaluated by trained surgeons in a remote surgical theatre using post-mortem pig eyes and shown to be effective in completing many key steps in a variety of intraocular surgical procedures as well as being capable of performing an entire cataract extraction from start to finish. Conclusions: The IRISS represents a necessary step towards fully automated intraocular surgery and demonstrated accurate and precise master-slave manipulation for cataract removal and—through visual feedback—retinal vein cannulation.

73 citations

Journal ArticleDOI
TL;DR: A systematic review of the state-of-the-art systems, picturing a detailed landscape of the system configurations, actuation schemes, and control approaches of the existing surgical robotic systems for keyhole and endoscopic procedures is presented.
Abstract: Minimally invasive surgery, including laparoscopic and thoracoscopic procedures, benefits patients in terms of improved postoperative outcomes and short recovery time. The challenges in hand-eye coordination and manipulation dexterity during the aforementioned procedures have inspired an enormous wave of developments on surgical robotic systems to assist keyhole and endoscopic procedures in the past decades. This paper presents a systematic review of the state-of-the-art systems, picturing a detailed landscape of the system configurations, actuation schemes, and control approaches of the existing surgical robotic systems for keyhole and endoscopic procedures. The development challenges and future perspectives are discussed in depth to point out the need for new enabling technologies and inspire future researches.

40 citations


Cites background from "Virtual fixture control of a hybrid..."

  • ...The manipulators can have either serial [28–31], parallel [32–34], or hybrid [35,36] structures....

    [...]

Journal ArticleDOI
TL;DR: These results present the key steps towards achieving an integrated system for OCT feedback control using a miniature intraocular B-mode probe and three-dimensional assistive telemanipulation virtual fixtures based on microscope and OCT feedback.
Abstract: During retinal microsurgery, surgeons cannot adequately visualize subsurface anatomical structures. In our previous work, a customized B-mode optical coherence tomography (OCT) probe was integrated into an ophthalmic robotic system to provide depth perception. This paper presents new approaches for implementing and achieving real-time feedback and assistive robotic control based on B-mode OCT imaging. The robotic system was comprised of a parallel robot, a micro-injection tool, and a telemanipulation master interface. A method for calibrating the B-mode OCT image scaling and distortion was presented using thin plate splines. Determining the OCT scanning plane relative to the robot base frame is presented through experiments and analyzed for sensitivity. A dual-rate controller using low frequency OCT feedback and high frequency position servoing was presented and tested for accuracy and latency. Three-dimensional assistive telemanipulation virtual fixtures based on microscope and OCT feedback are presented. The experimental evaluation demonstrated following target anatomy and semi-automated micro-injection. These results present the key steps towards achieving an integrated system for OCT feedback control using a miniature intraocular B-mode probe.

39 citations


Cites result from "Virtual fixture control of a hybrid..."

  • ...Previous works [28]–[30] have demonstrated computer vision VF...

    [...]

Proceedings ArticleDOI
01 Sep 2017
TL;DR: A new generalized kinematic formulation of the Remote Center of Motion (RCM) constraint for serial robotic arms to be useful for implementation in torque-controlled robots and can be applied not only in a kinematics control but also in a dynamic control approach.
Abstract: In this paper, we provide a new generalized kinematic formulation of the Remote Center of Motion (RCM) constraint for serial robotic arms to be useful for implementation in torque-controlled robots. A minimally invasive surgery assisted by a serial robot could be used for inserting a surgical tool through a desired insertion position, i.e. trocar position. The restricted movement caused by the trocar, i.e. the RCM constraint, must be guaranteed by the robotic system in order to avoid injury to the patient. The generalized formulation, presented in this paper, associates the task-space with the RCM constraint coordinates, which are described by the minimum distance between the surgical tool and the trocar position. In order to simultaneously achieve the surgical tool-tip trajectory and the RCM constraint, two control methods are proposed and compared, one using a projection in the null-space, the other by combining both tasks using a task-space augmentation method. Moreover, when additional lower priority tasks have to be implemented in the null-space of the two higher priority tasks, the kinematic formulation simplifies the calculation of the null-space projectors. Furthermore, unlike the previous studies, the formulation represented can be applied not only in a kinematic control but also in a dynamic control approach. Simulations were conducted to validate effectiveness of the proposed formulation, using a planar 4-DOF system as well as the dynamic model of a Kuka LBR 7 iiwa R800 robot arm.

30 citations


Cites methods from "Virtual fixture control of a hybrid..."

  • ...This approach was applied for a robotic-assisted intraocular surgery, as presented in [11]....

    [...]

Journal ArticleDOI
23 Sep 2017-Sensors
TL;DR: An assistive system combining a handheld micromanipulator, called “Micron”, with a force-sensing microneedle with a new control method to actively compensate unintended movements of the operator, and to keep the cannulation device securely inside the vein following cannulation is developed.
Abstract: Retinal vein cannulation is a technically demanding surgical procedure where therapeutic agents are injected into the retinal veins to treat occlusions. The clinical feasibility of this approach has been largely limited by the technical challenges associated with performing the procedure. Among the challenges to successful vein cannulation are identifying the moment of venous puncture, achieving cannulation of the micro-vessel, and maintaining cannulation throughout drug delivery. Recent advances in medical robotics and sensing of tool-tissue interaction forces have the potential to address each of these challenges as well as to prevent tissue trauma, minimize complications, diminish surgeon effort, and ultimately promote successful retinal vein cannulation. In this paper, we develop an assistive system combining a handheld micromanipulator, called “Micron”, with a force-sensing microneedle. Using this system, we examine two distinct methods of precisely detecting the instant of venous puncture. This is based on measured tool-tissue interaction forces and also the tracked position of the needle tip. In addition to the existing tremor canceling function of Micron, a new control method is implemented to actively compensate unintended movements of the operator, and to keep the cannulation device securely inside the vein following cannulation. To demonstrate the capabilities and performance of our uniquely upgraded system, we present a multi-user artificial phantom study with subjects from three different surgical skill levels. Results show that our puncture detection algorithm, when combined with the active positive holding feature enables sustained cannulation which is most evident in smaller veins. Notable is that the active holding function significantly attenuates tool motion in the vein, thereby reduces the trauma during cannulation.

25 citations

References
More filters
Book
01 Jan 2000
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Abstract: From the Publisher: A basic problem in computer vision is to understand the structure of a real world scene given several images of it. Recent major developments in the theory and practice of scene reconstruction are described in detail in a unified framework. The book covers the geometric principles and how to represent objects algebraically so they can be computed and applied. The authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly.

15,558 citations

01 Jan 2001
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
Abstract: Downloading the book in this website lists can give you more advantages. It will show you the best book collections and completed collections. So many books can be found in this website. So, this is not only this multiple view geometry in computer vision. However, this book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts. This is simple, read the soft file of the book and you get it.

14,282 citations

Journal ArticleDOI
TL;DR: The concept of task priority in relation to the inverse kinematic problem of redundant robot manipulators is introduced and the effectiveness of the proposed redundancy control scheme is shown.
Abstract: In this paper, we describe a new scheme for redundancy control of robot manipulators. We introduce the concept of task priority in relation to the inverse kinematic problem of redundant robot manipulators. A required task is divided into subtasks according to the order of priority. We propose to determine the joint motions of robot manipulators so that subtasks with lower priority can be performed utilizing re dundancy on subtasks with higher priority. This procedure is formulated using the pseudoinverses of Jacobian matrices. Most problems of redundancy utilization can be formulated in the framework of tasks with the order of priority. The results of numerical simulations and experiments show the effectiveness of the proposed redundancy control scheme.

933 citations

Proceedings ArticleDOI
John Baillieul1
25 Mar 1985
TL;DR: It is argued that because this technique may be expected to lift closed end effector paths to closed joint angle paths, it provides a promising approach for the control of kinematically redundant industrial manipulators.
Abstract: In the growing literature on redundant manipulator control, a number of techniques have been proposed for solving the inverse kinemetics problem. Some of these techniques are surveyed with a discussion of strengths and weaknesses of each. A new approach, called the extended Jacobian technique, is also presented. It is argued that because this technique may be expected to lift closed end effector paths to closed joint angle paths, it provides a promising approach for the control of kinematically redundant industrial manipulators. It is further shown that this technique may be implemented as a suitably parameterized generalized inverse method.

629 citations

Journal ArticleDOI
01 Aug 1989
TL;DR: The configuration control scheme is implemented for real-time control of three links of a PUMA 560 industrial robot, and experimental results are presented and discussed, and its capabilities for performing various realistic tasks are demonstrated.
Abstract: A simple approach for controlling the manipulator configuration over the entire motion, which is based on augmentation of the manipulator forward kinematics, is presented. A set of kinematic functions is defined in Cartesian or joint space to reflect the desirable configuration. The user-defined kinematic functions and the end-effector Cartesian coordinates are combined to form a set of task-related configuration variables as generalized coordinates for the manipulator. A task-based adaptive scheme is then utilized to control directly the configuration variables so as to achieve tracking of some desired reference trajectories throughout the robot motion. This accomplishes the basic task of desired end-effector motion, while utilizing the redundancy to achieve any additional task through the desired time variation of the kinematic functions. Simulation results for a direct-drive two-link arm are given to illustrate the proposed control scheme. The scheme has also been implemented for real-time control of three links of a PUMA 560 industrial robot, and experimental results are presented and discussed. The simulation and experimental results validate the configuration control scheme, and demonstrate its capabilities for performing various realistic tasks. >

414 citations

Frequently Asked Questions (9)
Q1. What are the contributions mentioned in the paper "Virtual fixture control of a hybrid parallel-serial robot for assisting ophthalmic surgery: an experimental study" ?

This paper presents the virtual fixture control methods for a hybrid parallel-serial micromanipulator, which is designed for assisting ophthalmic surgeons. The contribution of this paper is concept analysis and implementation of flexible virtual fixture for the novel hybrid parallel-serial mechanism and experimentally evaluation of this concept. Autonomous RCM adjustment for VitreoRetinal surgery, implying retinal reachability study, is the secondary contribution which is investigated in this paper. 

The restriction of motions through the entry point of the patient’s body is one of the most important characteristics of minimally invasive surgery assisted by robots. 

To get rid of configurations that yield much worse results than their reference the authors first filter the workspace by analysing the reachability of the robot without rotating the eye but by tilting the robot about the z-axis within the range of 0− 30 degrees with 10 degree steps. 

By fulfilling the RCM constraint the degrees of freedom of the robot are reduced by two, i.e. for fulfilling a task in an nt -dimensional space the robot must have at least n≥ nt +2 degrees of freedom [7]. 

Five sub micron precision piezo actuators are used to drive the robot and it is controlled using a middle-ware based architecture [6]. 

5. In their analysis the authors assume a maximum rotation of the eye → 30 degrees about the z-axis, maximum rotation in the direction of the cannula (around the x-axis) → −15 degrees and the maximum rotation around the y-axis→ ±10 degrees. 

Taking the RCM constraint into account the extended task can be defined as:xext = ( xTt pTrcm )T = ( x φ ψ xrcm yrcm zrcm)T ∈ R×S2×R3 (8) And the kinematics of the extended task are then given by:ẋext = (Jt 03×1 Jrcm )( q̇ λ̇ ) = Jext ( q̇ λ̇ ) (9)Assuming a desired robot task xd and the position of the RCM, which is defined by the location of the trocar ptrocar the extended task error is given by:eext = (xext −xd ptrocar−prcm) (10)Similar to the unrestricted movement the kinematic control is written as: (q̇ λ̇) = J†ext ( Kt 03×303×3 Krcm) eext (11)where Kt and Krcm are both postive definite diagonal matrices in R3×3. 

The most general way of describing the geometry of a sphere is implicitly in the projective space P3 with the equationxT Qx = 0with x representing a point in P3 and Q the quadric surface, which is given by the diagonal matrix Q = diag(1,1,1,−1) (see e.g.[15]). 

In this work the second approach was followed; using thealternative kinematics where the RCM constraint can be directly incorporated into the robot task.