In this paper we provide a broad survey of developments in active vision in robotic applications over the last 15 years. With increasing demand for robotic automation, research in this area has received much attention. Among the many factors that can be attributed to a high-performance robotic system, the planned sensing or acquisition of perceptions on the operating environment is a crucial component. The aim of sensor planning is to determine the pose and settings of vision sensors for undertaking a vision-based task that usually requires obtaining multiple views of the object to be manipulated. Planning for robot vision is a complex problem for an active system due to its sensing uncertainty and environmental uncertainty. This paper describes such problems arising from many applications, e.g. object recognition and modeling, site reconstruction and inspection, surveillance, tracking and search, as well as robotic manipulation and assembly, localization and mapping, navigation and exploration. A bundle of solutions and methods have been proposed to solve these problems in the past. They are summarized in this review while enabling readers to easily refer solution methods for practical applications. Representative contributions, their evaluations, analyses, and future research trends are also addressed in an abstract level.

Active vision in robotic systems: A survey of recent developments

Advancements in nanofibers for wound dressing: A review

Globally, in 2016, 1 out of 11 adults suffered from diabetes mellitus. Diabetic foot ulcers (DFU) are a major complication of this disease, which if not managed properly can lead to amputation. Current clinical approaches to DFU treatment rely on patient and clinician vigilance, which has significant limitations, such as the high cost involved in the diagnosis, treatment, and lengthy care of the DFU. We collected an extensive dataset of foot images, which contain DFU from different patients. In this DFU classification problem, we assessed the two classes as normal skin (healthy skin) and abnormal skin (DFU). In this paper, we have proposed the use of machine learning algorithms to extract the features for DFU and healthy skin patches to understand the differences in the computer vision perspective. This experiment is performed to evaluate the skin conditions of both classes that are at high risk of misclassification by computer vision algorithms. Furthermore, we used convolutional neural networks for the first time in this binary classification. We have proposed a novel convolutional neural network architecture, DFUNet, with better feature extraction to identify the feature differences between healthy skin and the DFU. Using 10-fold cross validation, DFUNet achieved an AUC score of 0.961. This outperformed both the traditional machine learning and deep learning classifiers we have tested. Here, we present the development of a novel and highly sensitive DFUNet for objectively detecting the presence of DFUs. This novel approach has the potential to deliver a paradigm shift in diabetic foot care among diabetic patients, which represent a cost-effective, remote, and convenient healthcare solution.

/pdf/dfunet-convolutional-neural-networks-for-diabetic-foot-ulcer-2zff0c1bbv.pdf

DFUNet: Convolutional Neural Networks for Diabetic Foot Ulcer Classification

A pressure ulcer is a clinical pathology of localized damage to the skin and underlying tissue caused by pressure, shear, or friction. Diagnosis, treatment, and care of pressure ulcers are costly for health services. Accurate wound evaluation is a critical task for optimizing the efficacy of treatment and care. Clinicians usually evaluate each pressure ulcer by visual inspection of the damaged tissues, which is an imprecise manner of assessing the wound state. Current computer vision approaches do not offer a global solution to this particular problem. In this paper, a hybrid approach based on neural networks and Bayesian classifiers is used in the design of a computational system for automatic tissue identification in wound images. A mean shift procedure and a region-growing strategy are implemented for effective region segmentation. Color and texture features are extracted from these segmented regions. A set of k multilayer perceptrons is trained with inputs consisting of color and texture patterns, and outputs consisting of categorical tissue classes which are determined by clinical experts. This training procedure is driven by a k-fold cross-validation method. Finally, a Bayesian committee machine is formed by training a Bayesian classifier to combine the classifications of the k neural networks. Specific heuristics based on the wound topology are designed to significantly improve the results of the classification. We obtain high efficiency rates from a binary cascade approach for tissue identification. Results are compared with other similar machine-learning approaches, including multiclass Bayesian committee machine classifiers and support vector machines. The different techniques analyzed in this paper show high global classification accuracy rates. Our binary cascade approach gives high global performance rates (average sensitivity =78.7% , specificity =94.7% , and accuracy =91.5% ) and shows the highest average sensitivity score ( =86.3%) when detecting necrotic tissue in the wound.

Binary Tissue Classification on Wound Images With Neural Networks and Bayesian Classifiers

One of the main challenges of employing deep learning models in the field of medicine is a lack of training data due to difficulty in collecting and labeling data, which needs to be performed by experts. To overcome this drawback, transfer learning (TL) has been utilized to solve several medical imaging tasks using pre-trained state-of-the-art models from the ImageNet dataset. However, there are primary divergences in data features, sizes, and task characteristics between the natural image classification and the targeted medical imaging tasks. Therefore, TL can slightly improve performance if the source domain is completely different from the target domain. In this paper, we explore the benefit of TL from the same and different domains of the target tasks. To do so, we designed a deep convolutional neural network (DCNN) model that integrates three ideas including traditional and parallel convolutional layers and residual connections along with global average pooling. We trained the proposed model against several scenarios. We utilized the same and different domain TL with the diabetic foot ulcer (DFU) classification task and with the animal classification task. We have empirically shown that the source of TL from the same domain can significantly improve the performance considering a reduced number of images in the same domain of the target dataset. The proposed model with the DFU dataset achieved F1-score value of 86.6% when trained from scratch, 89.4% with TL from a different domain of the targeted dataset, and 97.6% with TL from the same domain of the targeted dataset.

/pdf/towards-a-better-understanding-of-transfer-learning-for-5fbyw0qqwm.pdf

Towards a Better Understanding of Transfer Learning for Medical Imaging: A Case Study

In this paper, after an overview of the literature concerning the imaging technologies applied to skin wounds assessment, we present an original approach to build 3-D models of skin wounds from color images. The method can deal with uncalibrated images acquired with a handheld digital camera with free zooming. Compared with the cumbersome imaging systems already proposed, this novel solution uses a low-cost and user-friendly image acquisition device suitable for widespread application in health care centers. However, this method entails the development of a robust image processing chain. An original iterative matching scheme is used to generate a dense estimation of the surface geometry from two widely separated views. The best configuration for taking photographs lies between 15deg and 30deg for the vergency angle. The metric reconstruction of the skin wound is fully automated through self-calibration. From the 3-D model of the skin wound, accurate volumetric measurements are achieved. The accuracy of the inferred 3-D surface is validated by registration to a ground truth and repetitive tests on volume. The global precision around 3% is in accordance with the clinical requirement of 5% for assessing the healing process.

Three-Dimensional Assessment of Skin Wounds Using a Standard Digital Camera

With the widespread use of digital cameras, freehand wound imaging has become common practice in clinical settings. There is however still a demand for a practical tool for accurate wound healing assessment, combining dimensional measurements and tissue classification in a single user-friendly system. We achieved the first part of this objective by computing a 3-D model for wound measurements using uncalibrated vision techniques. We focus here on tissue classification from color and texture region descriptors computed after unsupervised segmentation. Due to perspective distortions, uncontrolled lighting conditions and view points, wound assessments vary significantly between patient examinations. The main contribution of this paper is to overcome this drawback with a multiview strategy for tissue classification, relying on a 3-D model onto which tissue labels are mapped and classification results merged. The experimental classification tests demonstrate that enhanced repeatability and robustness are obtained and that metric assessment is achieved through real area and volume measurements and wound outline extraction. This innovative tool is intended for use not only in therapeutic follow-up in hospitals but also for telemedicine purposes and clinical research, where repeatability and accuracy of wound assessment are critical.

/pdf/enhanced-assessment-of-the-wound-healing-process-by-accurate-1h0xrlvp55.pdf

Enhanced Assessment of the Wound-Healing Process by Accurate Multiview Tissue Classification

This work is part of the ESCALE project dedicated to the design of a complete 3D and color wound assessment tool using a simple free handled digital camera. The first part was concerned with the computation of a 3D model for wound measurements using uncalibrated vision techniques. This paper presents the second part which deals with color classification of wound tissues, a prior step before to combine shape and color analysis in a single tool for real tissue surface measurements. As direct pixel classification proved to be inefficient for tissue wound labeling, we have adopted an original approach based on unsupervised segmentation prior to classification, to improve the robustness of the labeling step by considering spatial continuity and homogeneity. A ground truth is first provided by merging the images collected and labeled by clinicians. Then, color and texture tissue descriptors are extracted on labeled regions of this learning database to design a SVM region classifier, achieving 88% success overlap score. Finally, we apply unsupervised color region segmentation on test images and classify the regions. Compared to the ground truth, segmentation driven classification and clinician labeling achieve similar performance, around 75 % for granulation and 60 % for slough.

Supervised Tissue Classification from Color Images for a Complete Wound Assessment Tool

In telemedicine environments, a standardized and reproducible assessment of wounds, using a simple free-handled digital camera, is an essential requirement. However, to ensure robust tissue classification, particular attention must be paid to the complete design of the color processing chain. We introduce the key steps including color correction, merging of expert labeling, and segmentation-driven classification based on support vector machines. The tool thus developed ensures stability under lighting condition, viewpoint, and camera changes, to achieve accurate and robust classification of skin tissues. Clinical tests demonstrate that such an advanced tool, which forms part of a complete 3-D and color wound assessment system, significantly improves the monitoring of the healing process. It achieves an overlap score of 79.3 against 69.1% for a single expert, after mapping on the medical reference developed from the image labeling by a college of experts.

/pdf/robust-tissue-classification-for-reproducible-wound-3yr83a4df9.pdf

Robust tissue classification for reproducible wound assessment in telemedicine environments

Hyperspectral imaging is an emerging technology recently introduced in medical applications inasmuch as it provides a powerful tool for noninvasive tissue characterization. In this context, a new system was designed to be easily integrated in the operating room in order to detect anatomical tissues hardly noticed by the surgeon’s naked eye. Our LCTF-based spectral imaging system is operative over visible, near- and middle-infrared spectral ranges (400–1700 nm). It is dedicated to enhance critical biological tissues such as the ureter and the facial nerve. We aim to find the best three relevant bands to create a RGB image to display during the intervention with maximal contrast between the target tissue and its surroundings. A comparative study is carried out between band selection methods and band transformation methods. Combined band selection methods are proposed. All methods are compared using different evaluation criteria. Experimental results show that the proposed combined band selection methods provide the best performance with rich information, high tissue separability and short computational time. These methods yield a significant discrimination between biological tissues. We developed a hyperspectral imaging system in order to enhance some biological tissue visualization. The proposed methods provided an acceptable trade-off between the evaluation criteria especially in SWIR spectral band that outperforms the naked eye’s capacities.

Yves Lucas

Papers

Three-Dimensional Assessment of Skin Wounds Using a Standard Digital Camera

Enhanced Assessment of the Wound-Healing Process by Accurate Multiview Tissue Classification

Supervised Tissue Classification from Color Images for a Complete Wound Assessment Tool

Robust tissue classification for reproducible wound assessment in telemedicine environments

Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods