scispace - formally typeset
Search or ask a question

Computational Modeling Approaches For Task Analysis In Robotic-Assisted Surgery

TL;DR: A new framework, namely DTW-kNN, is introduced, to recognize and classify three important surgical tasks including suturing, needle passing and knot tying based on kinematic data captured using da Vinci robotic surgery system.
Abstract: COMPUTATIONAL MODELING APPROACHES FOR TASK ANALYSIS IN ROBOTIC-ASSISTED SURGERY by MAHTAB JAHANBANI FARD May 2016 Advisor: Dr. R. Darin Ellis Major: Industrial Engineering Degree: Doctor of Philosophy Surgery is continuously subject to technological innovations including the introduction of robotic surgical devices. The ultimate goal is to program the surgical robot to perform certain difficult or complex surgical tasks in an autonomous manner. The feasibility of current robotic surgery systems to record quantitative motion and video data motivates developing descriptive mathematical models to recognize, classify and analyze surgical tasks. Recent advances in machine learning research for uncovering concealed patterns in huge data sets, like kinematic and video data, offer a possibility to better understand surgical procedures from a system point of view. This dissertation focuses on bridging the gap between these two lines of the research by developing computational models for task analysis in robotic-assisted surgery. The key step for advance study in robotic-assisted surgery and autonomous skill assess- ment is to develop techniques that are capable of recognizing fundamental surgical tasks intelligently. Surgical tasks and at a more granular level, surgical gestures, need to be quan- tified to make them amenable for further study. To answer to this query, we introduce a new framework, namely DTW-kNN, to recognize and classify three important surgical tasks including suturing, needle passing and knot tying based on kinematic data captured using da Vinci robotic surgery system. Our proposed method needs minimum preprocessing that

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: A predictive framework for objective skill assessment based on movement trajectory data is introduced to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise.
Abstract: Background Surgical skill assessment has predominantly been a subjective task. Recently, technological advances such as robot-assisted surgery have created great opportunities for objective surgical evaluation. In this paper, we introduce a predictive framework for objective skill assessment based on movement trajectory data. Our aim is to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise. Methods Eight global movement features are extracted from movement trajectory data captured by a da Vinci robot for surgeons with two levels of expertise – novice and expert. Three classification methods – k-nearest neighbours, logistic regression and support vector machines – are applied. Results The result shows that the proposed framework can classify surgeons' expertise as novice or expert with an accuracy of 82.3% for knot tying and 89.9% for a suturing task. Conclusion This study demonstrates and evaluates the ability of machine learning methods to automatically classify expert and novice surgeons using global movement features.

138 citations


Cites background from "Computational Modeling Approaches F..."

  • ...Surgical tasks have different characteristics, such as smoothness, straightness or response orientation, which determine competence while relying only on instrument motion.(24) For instance, studies have shown that the tool motion of an experienced surgeon has more clearly defined features than that of a less experienced surgeon while performing the same task....

    [...]

  • ...Although these methods have the ability to find the structure underlying MIS or RMIS tasks, they are context‐ based and suffer from requiring a large number of training samples and complex parameter tuning, causing a lack of robustness in the results.(24) While the first approach focuses on skill evaluation at a more granular level, the second approach, in contrast, evaluates overall surgical skill using global movement features (GMFs), and this is easier to implement and interpret....

    [...]

Journal Article
TL;DR: The proposed method is applicable to any gesture represented by a multi- dimensional signal, and will be a valuable tool in telerobotics and human computer interfaces.
Abstract: This paper proposes a novel hidden Markov model (HMM)-based gesture recognition method and applies it to an HCI to control a computer game. The novelty of the proposed method is two-fold: 1) the proposed method uses a continuous streaming of human motion as the input to the HMM instead of isolated data sequences or pre-segmented sequences of data and 2) the gesture segmentation and recognition are performed simultaneously. The proposed method consists of a single HMM composed of thirteen gesture-specific HMMs that independently recognize certain gestures. It takes a continuous stream of pose symbols as an input, where a pose is composed of coordinates that indicate the face, left hand, and right hand. Whenever a new input Pose arrives, the HMM continuously updates its state probabilities, then recognizes a gesture if the probability of a distinctive state exceeds a predefined threshold. To assess the validity of the proposed method, it was applied to a real game, Quake II, and the results demonstrated that the proposed HMM could provide very useful information to enhance the discrimination between different classes and reduce the computational cost.

115 citations

Journal ArticleDOI
TL;DR: The problem space for OCase-T is defined and 45 publications representing recent research in this domain are summarized, finding that most studies on OCASE-T are simulation based; very few are in the operating room; and the algorithms and validation methodologies used are highly varied.
Abstract: Training skillful and competent surgeons is critical to ensure high quality of care and to minimize disparities in access to effective care. Traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. Simultaneously, technological developments are enabling capture and analysis of large amounts of complex surgical data. These developments are motivating a "surgical data science" approach to objective computer-aided technical skill evaluation (OCASE-T) for scalable, accurate assessment; individualized feedback; and automated coaching. We define the problem space for OCASE-T and summarize 45 publications representing recent research in this domain. We find that most studies on OCASE-T are simulation based; very few are in the operating room. The algorithms and validation methodologies used for OCASE-T are highly varied; there is no uniform consensus. Future research should emphasize competency assessment in the operating room, validation against patient outcomes, and effectiveness for surgical training.

106 citations


Cites methods from "Computational Modeling Approaches F..."

  • ...Reported measures of accuracy in classification were similar across two studies for a suturing task (64–79%), with minimal differences among linear, nonlinear, and prototype-based classifiers (78, 84, 85)....

    [...]

  • ...Generic descriptors of tool motion efficiency, such as time, path length, or movements, which may be easily computed on any surgical platform, were used for OCASE-T in several other studies (78, 79, 81, 82, 84, 85, 93, 97)....

    [...]

Journal ArticleDOI
01 Jan 2017
TL;DR: A new segmentation algorithm, namely soft-boundary unsupervised gesture segmentation (Soft-UGS), to segment the temporal sequence of surgical gestures and model gradual transitions between them using fuzzy membership scores is developed.
Abstract: Robotic-assisted surgery holds significant promise to improve patient treatment by allowing surgeons to perform many types of complex operations with greater precision and flexibility than before. In order to facilitate automation of robotic surgery and more practical training for surgeons, more detailed comprehension of the surgical procedures is needed. In this regard, a key step is to develop techniques that segment and recognize surgical tasks intelligently. Surgeries involve complex continuous activities that may contain superfluous, repeated actions, and temporal variation. Therefore, any segmentation approach that has the capability to account for all these characteristics is of increased interest. Toward this goal, we develop a new segmentation algorithm, namely soft-boundary unsupervised gesture segmentation (Soft-UGS), to segment the temporal sequence of surgical gestures and model gradual transitions between them using fuzzy membership scores. The proposed framework is evaluated using a real robotic surgery dataset. Our extensive set of experiments and evaluation metrics show that the proposed Soft-UGS method is able to match manual annotations with upto 83% sensitivity, 81% precision, and 73% segmentation score. The results show that the proposed soft boundary approach can provide more insight into the surgical activities and can contribute to the automation of robotic surgeries.

43 citations


Cites background from "Computational Modeling Approaches F..."

  • ...There are few shortcomings in these existing works [20]....

    [...]

Journal ArticleDOI
TL;DR: To allow the robot to adjust parameters such as camera position, the system needs to know automatically what task the surgeon is performing, and to allow the system to know Automated Peripheral Integration (AI) of the surgeon and the robot.
Abstract: Background Robotic-assisted surgery allows surgeons to perform many types of complex operations with greater precision than is possible with conventional surgery. Despite these advantages, in current systems, a surgeon should communicate with the device directly and manually. To allow the robot to adjust parameters such as camera position, the system needs to know automatically what task the surgeon is performing. Methods A distance-based time series classification framework has been developed which measures dynamic time warping distance between temporal trajectory data of robot arms and classifies surgical tasks and gestures using a k-nearest neighbor algorithm. Results Results on real robotic surgery data show that the proposed framework outperformed state-of-the-art methods by up to 9% across three tasks and by 8% across gestures. Conclusion The proposed framework is robust and accurate. Therefore, it can be used to develop adaptive control systems that will be more responsive to surgeons' needs by identifying next movements of the surgeon. Copyright © 2016 John Wiley & Sons, Ltd.

29 citations


Cites background from "Computational Modeling Approaches F..."

  • ...Surgical tasks and at a more granular level, surgical gestures need to be quantified to make them amenable to further study in autonomous surgical system (8)....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

40,826 citations


"Computational Modeling Approaches F..." refers background in this paper

  • ...It is suggested [109] to choose γ as inverse of number...

    [...]

01 Jan 1998
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Abstract: A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

26,531 citations


"Computational Modeling Approaches F..." refers methods in this paper

  • ...In this study we applied radial basis function (RBF) which is one of the most popular kernel functions used in SVM [108], defined as K(xi, xj) = e (−γ||xi−xj ||(2)) (4....

    [...]

  • ...59 4.9 Illustration of kernel function in support vector machine (SVM) method for two features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.10 Box plots of Exp (experts) and Nov (novices) in knot tying for L (left hand) and R (right hand). . . . . . . . . . . . . . . . . . . . . . . . . ....

    [...]

  • ...56 4.7 Example for influence of the divider step size on the path length for a suturing task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.8 Illustration of support vector machine (SVM) method for two features....

    [...]

Reference EntryDOI
15 Oct 2005
TL;DR: Principal component analysis (PCA) as discussed by the authors replaces the p original variables by a smaller number, q, of derived variables, the principal components, which are linear combinations of the original variables.
Abstract: When large multivariate datasets are analyzed, it is often desirable to reduce their dimensionality. Principal component analysis is one technique for doing this. It replaces the p original variables by a smaller number, q, of derived variables, the principal components, which are linear combinations of the original variables. Often, it is possible to retain most of the variability in the original variables with q very much smaller than p. Despite its apparent simplicity, principal component analysis has a number of subtleties, and it has many uses and extensions. A number of choices associated with the technique are briefly discussed, namely, covariance or correlation, how many components, and different normalization constraints, as well as confusion with factor analysis. Various uses and extensions are outlined. Keywords: dimension reduction; factor analysis; multivariate analysis; variance maximization

14,773 citations

Journal ArticleDOI
TL;DR: The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Abstract: The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points. This rule is independent of the underlying joint distribution on the sample points and their classifications, and hence the probability of error R of such a rule must be at least as great as the Bayes probability of error R^{\ast} --the minimum probability of error over all decision rules taking underlying probability structure into account. However, in a large sample analysis, we will show in the M -category case that R^{\ast} \leq R \leq R^{\ast}(2 --MR^{\ast}/(M-1)) , where these bounds are the tightest possible, for all suitably smooth underlying distributions. Thus for any number of categories, the probability of error of the nearest neighbor rule is bounded above by twice the Bayes probability of error. In this sense, it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.

12,243 citations


"Computational Modeling Approaches F..." refers methods in this paper

  • ...In particular, we compare three frequently used machine learning techniques, k-nearest neighbor (kNN) [44], Logistic regression [106] and Support Vector Machine [107]....

    [...]

  • ...It classifies robotic-assistant surgical tasks and recognize surgical gestures by integrating temporal sequence similarity measure techniques such as Dynamic Time Warping (DTW) [43] with well-known k-nearest neighbor (kNN) classification method [44]....

    [...]

Book
24 Aug 2012
TL;DR: This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach, and is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.
Abstract: Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

8,059 citations


"Computational Modeling Approaches F..." refers methods in this paper

  • ...While machine learning techniques have been used extensively in other fields because of their advantages over traditional statistical methods such as robustness, better prediction ability and higher tolerance to violations of assumptions [11], but it is only recently that these methods have been considered to analyze RMIS tasks....

    [...]

  • ...normality or undependability of data) [11], but it is only recently that these methods have been considered to analyze RMIS tasks [10]....

    [...]