MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

Video capsule endoscopy (VCE) has revolutionized the diagnostic work-up in the field of small bowel diseases. Furthermore, VCE has the potential to become the leading screening technique for the entire gastrointestinal tract. Computational methods that can be implemented in software can enhance the diagnostic yield of VCE both in terms of efficiency and diagnostic accuracy. Since the appearance of the first capsule endoscope in clinical practice in 2001, information technology (IT) research groups have proposed a variety of such methods, including algorithms for detecting haemorrhage and lesions, reducing the reviewing time, localizing the capsule or lesion, assessing intestinal motility, enhancing the video quality and managing the data. Even though research is prolific (as measured by publication activity), the progress made during the past 5 years can only be considered as marginal with respect to clinically significant outcomes. One thing is clear-parallel pathways of medical and IT scientists exist, each publishing in their own area, but where do these research pathways meet? Could the proposed IT plans have any clinical effect and do clinicians really understand the limitations of VCE software? In this Review, we present an in-depth critical analysis that aims to inspire and align the agendas of the two scientific groups.

Software for enhanced video capsule endoscopy: challenges for essential progress

Today, medical endoscopy is a widely used procedure to inspect the inner cavities of the human body. The advent of endoscopic imaging techniques-allowing the acquisition of images or videos-created the possibility for the development of the whole new branch of computer-aided decision support systems. Such systems aim at helping physicians to identify possibly malignant abnormalities more accurately. At the beginning of this paper, we give a brief introduction to the history of endoscopy, followed by introducing the main types of endoscopes which emerged so far (flexible endoscope, wireless capsule endoscope, and confocal laser endomicroscope). We then give a brief introduction to computer-aided decision support systems specifically targeted at endoscopy in the gastrointestinal tract. Then we present general facts and figures concerning computer-aided decision support systems and summarize work specifically targeted at computer-aided decision support in the gastrointestinal tract. This summary is followed by a discussion of some common issues concerning the approaches reviewed and suggestions of possible ways to resolve them.

/pdf/computer-aided-decision-support-systems-for-endoscopy-in-the-3dfptp4g31.pdf

Computer-Aided Decision Support Systems for Endoscopy in the Gastrointestinal Tract: A Review

Wireless Capsule Endoscopy (WCE) is a standard non-invasive modality for small bowel examination. Recently, the development of computer-aided diagnosis (CAD) systems for gastrointestinal (GI) bleeding detection in WCE image videos has become an active research area with the goal of relieving the workload of physicians. Existing methods based primarily on handcrafted features usually give insufficient accuracy for bleeding detection, due to their limited capability of feature representation. In this paper, we present a new automatic bleeding detection strategy based on a deep convolutional neural network and evaluate our method on an expanded dataset of 10,000 WCE images. Experimental results with an increase of around 2 percentage points in the Fi score demonstrate that our method outperforms the state-of-the-art approaches in WCE bleeding detection. The achieved Fi score is of up to 0.9955.

A deep convolutional neural network for bleeding detection in Wireless Capsule Endoscopy images

In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with real-time assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.

/pdf/content-based-processing-and-analysis-of-endoscopic-images-4v8m37v5pb.pdf

Content-based processing and analysis of endoscopic images and videos: A survey

This paper presents a novel multi-level approach for bleeding detection in Wireless Capsule Endoscopy (WCE) images. In the low-level processing, each cell of K×K pixels is characterized by an adaptive color histogram which optimizes the information representation for WCE images. A Neural Network (NN) cell-classifier is trained to classify cells in an image as bleeding or non-bleeding patches. In the intermediate-level processing, a block which covers 3×3 cells is formed. The intermediate-level representation of the block is generated from the low-level classifications of the cells, which captures the spatial local correlations of the cell classifications. Again, a NN block-classifier is trained to classify the blocks as bleeding or non-bleeding ones. In the high-level processing, the low-level cell-based and intermediate-level block-based classifications are fused for final detection. In this way, our approach can combine the low-level features from pixels and intermediate-level features from local regions to achieve robust bleeding detection. Experiments on real WCE videos have shown that the proposed method of multi-level classification is not only accurate in both detection and localization of potential bleedings in WCE images but also robust to complex local noisy features.

Multi-level local feature classification for bleeding detection in Wireless Capsule Endoscopy images

A video recording of an examination by Wireless Capsule Endoscopy (WCE) may typically contain more than 55,000 video frames, which makes the manual visual screening by an experienced gastroenterologist a highly time-consuming task. In this paper, we propose a novel method of epitomized summarization of WCE videos for efficient visualization to a gastroenterologist. For each short sequence of a WCE video, an epitomized frame is generated. New constraints are introduced into the epitome formulation to achieve the necessary visual quality for manual examination, and an EM algorithm for learning the epitome is derived. First, the local context weights are introduced to generate the epitomized frame. The epitomized frame preserves the appearance of all the input patches from the frames of the short sequence. Furthermore, by introducing spatial distributions for semantic interpretation of image patches in our epitome formulation, we show that it also provides a framework to facilitate the semantic description of visual features to generate organized visual summarization of WCE video, where the patches in different positions correspond to different semantic information. Our experiments on realWCE videos show that, using epitomized summarization, the number of frames have to be examined by the gastroenterologist can be reduced to less than one-tenth of the original frames in the video.

/pdf/epitomized-summarization-of-wireless-capsule-endoscopic-4emzjzq7je.pdf

Weijia Shen

Papers

Multi-level local feature classification for bleeding detection in Wireless Capsule Endoscopy images

Epitomized summarization of wireless capsule endoscopic videos for efficient visualization