scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Integrated vision-based system for efficient, semi-automated control of a robotic manipulator

10 Aug 2014-International Journal of Intelligent Computing and Cybernetics (Emerald Group Publishing Limited)-Vol. 7, Iss: 3, pp 253-266
TL;DR: An integrated, computer vision-based system to operate a commercial wheelchair-mounted robotic manipulator (WMRM) and a gesture recognition interface system was developed specially for individuals with upper-level spinal cord injuries including object tracking and face recognition to function as an efficient, hands-free WMRM controller.
Abstract: Purpose – The purpose of this paper is to develop an integrated, computer vision-based system to operate a commercial wheelchair-mounted robotic manipulator (WMRM). In addition, a gesture recognition interface system was developed specially for individuals with upper-level spinal cord injuries including object tracking and face recognition to function as an efficient, hands-free WMRM controller. Design/methodology/approach – Two Kinect® cameras were used synergistically to perform a variety of simple object retrieval tasks. One camera was used to interpret the hand gestures and locate the operator's face for object positioning, and then send those as commands to control the WMRM. The other sensor was used to automatically recognize different daily living objects selected by the subjects. An object recognition module employing the Speeded Up Robust Features algorithm was implemented and recognition results were sent as a commands for “coarse positioning” of the robotic arm near the selected object. Automat...
Citations
More filters
Journal ArticleDOI
TL;DR: The experimental results indicated that hybrid (gesture and speech) interaction outperforms any single modal interaction in a multiple-sensors, 3D vision-based, autonomous wheelchair-mounted robotic manipulator (WMRM).

32 citations

Journal ArticleDOI
01 Sep 2016
TL;DR: A benchmark and validation framework is provided for the comparison of touch-based and touchless interfaces selected to teleoperate a highly dexterous surgical robot and discusses their potential for current and future adoption in the surgical setting.
Abstract: This research presents an exploratory study among touch-based and touchless interfaces selected to teleoperate a highly dexterous surgical robot. The possibility of incorporating touchless interfaces into the surgical arena may provide surgeons with the ability to engage in telerobotic surgery similarly as if they were operating with their bare hands. On the other hand, precision and sensibility may be lost. To explore the advantages and drawbacks of these modalities, five interfaces were selected to send navigational commands to the Taurus robot in the system: Omega, Hydra, and a keyboard. The first represented touch-based, while Leap Motion and Kinect were selected as touchless interfaces. Three experimental designs were selected to test the system, based on standardized surgically related tasks and clinically relevant performance metrics measured to evaluate the user's performance, learning rates, control stability, and interaction naturalness. The current work provides a benchmark and validation framework for the comparison of these two groups of interfaces and discusses their potential for current and future adoption in the surgical setting.

13 citations

Journal ArticleDOI
TL;DR: Simulation experiments show that the intelligent recognition algorithm based on convolutional neural network can effectively improve the recognition accuracy of small-scale motion in medical moving images and improve the speed of motion.
Abstract: For the small-scale motion in medical motion images, the traditional medical motion image intelligent recognition algorithm has low recognition accuracy, and requires a large amount of calculation statistics. There is no self-learning function, which seriously affects the accuracy and speed of medical motion image recognition. Therefore, in order to improve the accuracy of human body small-scale motion recognition in medical motion images and the computational efficiency of large-scale data sets, an intelligent recognition algorithm based on convolutional neural network for medical motion images is proposed. The algorithm first learns the dense trajectory features and depth features, and then further fuses the dense trajectory features with the deep learning features. Finally, the extreme learning machine is applied to the convolutional neural network, and the fused features are further trained as input information of the convolutional neural network, and the features from the bottom layer to the upper layer can be extracted step by step from the raw data of the pixel level. Simulation experiments show that the algorithm can effectively improve the recognition accuracy of small-scale motion in medical moving images and improve the speed of motion.

12 citations


Cites background from "Integrated vision-based system for ..."

  • ...And it will have a great influence on the calculation amount of the subsequent detection and recognition algorithm [27], [28]....

    [...]

Journal ArticleDOI
TL;DR: A natural human–robot teleoperation system, which capitalizes on the latest advancements of monocular human pose estimation to simplify scenario requirements on heterogeneous robot arm teleoperation and enhances the controllability and flexibility of robot arms by releasing human operator from motion constraints, paving a new way for effective robot teleoperation.
Abstract: This paper aims to present a natural human–robot teleoperation system, which capitalizes on the latest advancements of monocular human pose estimation to simplify scenario requirements on heterogeneous robot arm teleoperation.,Several optimizations in the joint extraction process are carried on to better balance the performance of the pose estimation network. To bridge the gap between human joint pose in Cartesian space and heterogeneous robot joint angle pose in Radian space, a routinized mapping procedure is proposed.,The effectiveness of the developed methods on joint extraction is verified via qualitative and quantitative experiments. The teleoperation experiments on different robots validate the feasibility of the system controlling.,The proposed system provides an intuitive and efficient human–robot teleoperation method with low-cost devices. It also enhances the controllability and flexibility of robot arms by releasing human operator from motion constraints, paving a new way for effective robot teleoperation.

7 citations

Proceedings ArticleDOI
Chen Naijian1, Han Xiangdong1, Wang Yantao1, Cai Xinglai1, Chen Hui1 
05 Jun 2016
TL;DR: Experimental results show that the coordination control strategy between human and wheelchair manipulator based on BCI is effectiveness and validity in assisting activities of daily living.
Abstract: To assist activities of daily living, a wheelchair and its mounted manipulator is developed using a coordination control strategy in this paper. The coordination control strategy is composed of a operating intention express and identifying with EEG, locating objective based on EOG and head gesture and a human-robot interface. EEG is used to be acquired and identified the operating intention through user's motion imagery such as left and right legs, left and right hands and so on to express control instructions for the wheelchair and its mounted manipulator. EOG signals and user head gesture are used to locate the operating objective. The EEG and EOG identification systems are integrated in the human-robot interface effectively. Experimental results show that the coordination control strategy between human and wheelchair manipulator based on BCI is effectiveness and validity in assisting activities of daily living.

5 citations


Additional excerpts

  • ...Jiang [9] developed an integrated, computer vision- based system to operate a commercial wheelchairmouted robotic manipulator....

    [...]

References
More filters
Proceedings ArticleDOI
01 Dec 2001
TL;DR: A machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates and the introduction of a new image representation called the "integral image" which allows the features used by the detector to be computed very quickly.
Abstract: This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.

18,620 citations


"Integrated vision-based system for ..." refers methods in this paper

  • ...A face detector (Viola and Jones, 2001) was used to remove the face region and extract the remaining two largest blobs as the hand regions....

    [...]

  • ...Automatic Face Detection Module A face detector (Viola and Jones, 2001) was employed in this module to perform automatic face detection....

    [...]

  • ...A face detector (Viola and Jones, 2001) was employed in this module to perform automatic face detection....

    [...]

Book ChapterDOI
07 May 2006
TL;DR: A novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features), which approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.
Abstract: In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.

13,011 citations


"Integrated vision-based system for ..." refers methods in this paper

  • ...A Speeded Up Robust Features (SURF) algorithm was employed to recognize these daily living test objects (Bay et al., 2006)....

    [...]

Journal ArticleDOI
TL;DR: It is found that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.
Abstract: Learning visual models of object categories notoriously requires hundreds or thousands of training examples. We show that it is possible to learn much information about a category from just one, or a handful, of images. The key insight is that, rather than learning from scratch, one can take advantage of knowledge coming from previously learned categories, no matter how different these categories might be. We explore a Bayesian implementation of this idea. Object categories are represented by probabilistic models. Prior knowledge is represented as a probability density function on the parameters of these models. The posterior model for an object category is obtained by updating the prior in the light of one or more observations. We test a simple implementation of our algorithm on a database of 101 diverse object categories. We compare category models learned by an implementation of our Bayesian approach to models learned from by maximum likelihood (ML) and maximum a posteriori (MAP) methods. We find that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.

2,976 citations


"Integrated vision-based system for ..." refers methods in this paper

  • ...For recognition of unknown objects, one shot learning techniques can be used (Fei-Fei et al., 2006; Jiang et al., 2013a)....

    [...]

Proceedings ArticleDOI
16 Jun 2012
TL;DR: This work presents a novel dataset and novel algorithms for the problem of detecting activities of daily living in firstperson camera views, and develops novel representations including temporal pyramids and composite object models that exploit the fact that objects look different when being interacted with.
Abstract: We present a novel dataset and novel algorithms for the problem of detecting activities of daily living (ADL) in firstperson camera views. We have collected a dataset of 1 million frames of dozens of people performing unscripted, everyday activities. The dataset is annotated with activities, object tracks, hand positions, and interaction events. ADLs differ from typical actions in that they can involve long-scale temporal structure (making tea can take a few minutes) and complex object interactions (a fridge looks different when its door is open). We develop novel representations including (1) temporal pyramids, which generalize the well-known spatial pyramid to approximate temporal correspondence when scoring a model and (2) composite object models that exploit the fact that objects look different when being interacted with. We perform an extensive empirical evaluation and demonstrate that our novel representations produce a two-fold improvement over traditional approaches. Our analysis suggests that real-world ADL recognition is “all about the objects,” and in particular, “all about the objects being interacted with.”

757 citations


"Integrated vision-based system for ..." refers methods in this paper

  • ...In this paper, we combine this gesture recognition-based interface with face and object recognition modules for subjects to more efficiently retrieve daily living objects (Pirsiavash and Ramanan, 2012) in the environment....

    [...]

Journal ArticleDOI
TL;DR: Smart wheelchairs have been the subject of research since the early 1980s and have been developed on four continents and presented a summary of the current state of the art and directions for future research.
Abstract: — Several studies have shown that both children andadults benefit substantially from access to a means of indepen-dent mobility. While the needs of many individuals with disabil-ities can be satisfied with traditional manual or poweredwheelchairs, a segment of the disabled community finds it diffi-cult or impossible to use wheelchairs independently. To accom-modate this population, researchers have used technologiesoriginally developed for mobile robots to create “smart wheel-chairs.” Smart wheelchairs have been the subject of researchsince the early 1980s and have been developed on four conti-nents. This article presents a summary of the current state of theart and directions for future research. Key words: artificial intelligence, independent mobility, infra-red range finder, laser range finder, machine vision, powerwheelchairs, robotics, sonar, subsumption, voice control. INTRODUCTION Several studies have shown that both children andadults benefit substantially from access to a means ofindependent mobility, including power wheelchairs, man-ual wheelchairs, scooters, a nd walkers [1–2]. Independentmobility increases vocational and educational opportuni-ties, reduces dependence on caregivers and family mem-bers, and promotes feelings of self-reliance. For youngchildren, independent mobility serves as the foundationfor much early learning [1]. Nonambulatory children lackaccess to the wealth of stimuli afforded self-ambulatingchildren. This lack of exploration and control often pro-duces a cycle of deprivation and reduced motivation thatleads to learned helplessness [3].For adults, independent mobility is an importantaspect of self-esteem and plays a pivotal role in “aging inplace.” For example, if older people find it increasinglydifficult to walk or wheel themselves to the commode,they may do so less often or they may drink less fluid toreduce the frequency of urination. If they become unableto walk or wheel themselves to the commode and help isnot routinely available in the home when needed, a moveto a more enabling environment (e.g., assisted living) maybe necessary. Mobility limitati ons are the leading cause offunctional limitations among adults, with an estimatedprevalence of 40 per 1,000 persons age 18 to 44 and 188per 1,000 at age 85 and older [4]. Mobility difficulties arealso strong predictors of activities of daily living (ADL)and instrumental ADL disabi lities because of the need to

531 citations


"Integrated vision-based system for ..." refers background in this paper

  • ...For an exhausted review of the literature in this field, refer to Simpson’s review (Simpson, 2005)....

    [...]