Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Training a deep convolutional neural network (CNN) from scratch is difficult because it requires a large amount of labeled training data and a great deal of expertise to ensure proper convergence. A promising alternative is to fine-tune a CNN that has been pre-trained using, for instance, a large set of labeled natural images. However, the substantial differences between natural and medical images may advise against such knowledge transfer. In this paper, we seek to answer the following central question in the context of medical image analysis: Can the use of pre-trained deep CNNs with sufficient fine-tuning eliminate the need for training a deep CNN from scratch? To address this question, we considered four distinct medical imaging applications in three specialties (radiology, cardiology, and gastroenterology) involving classification, detection, and segmentation from three different imaging modalities, and investigated how the performance of deep CNNs trained from scratch compared with the pre-trained CNNs fine-tuned in a layer-wise manner. Our experiments consistently demonstrated that 1) the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch; 2) fine-tuned CNNs were more robust to the size of training sets than CNNs trained from scratch; 3) neither shallow tuning nor deep tuning was the optimal choice for a particular application; and 4) our layer-wise fine-tuning scheme could offer a practical way to reach the best performance for the application at hand based on the amount of available data.

Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?

MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

IEEE Transactions on Pattern Analysis and Machine Intelligence

The WAVES investigation on the WIND spacecraft will provide comprehensive measurements of the radio and plasma wave phenomena which occur in Geospace. Analyses of these measurements, in coordination with the other onboard plasma, energetic particles, and field measurements will help us understand the kinetic processes that are important in the solar wind and in key boundary regions of the Geospace. These processes are then to be interpreted in conjunction with results from the other ISTP spacecraft in order to discern the measurements and parameters for mass, momentum, and energy flow throughout geospace. This investigation will also contribute to observations of radio waves emitted in regions where the solar wind is accelerated. The WAVES investigation comprises several innovations in this kind of instrumentation: among which the first use, to our knowledge, of neural networks in real-time on board a scientific spacecraft to analyze data and command observation modes, and the first use of a wavelet transform-like analysis in real time to perform a spectral analysis of a broad band signal.

/pdf/waves-the-radio-and-plasma-wave-investigation-on-the-wind-ngfb2pv1t7.pdf

Waves: The Radio and Plasma Wave Investigation on the Wind Spacecraft

We propose a novel FPGA-based architecture for the extraction of four texture features using gray level cooccurrence matrix (GLCM) analysis. These features are angular second moment, correlation, inverse difference moment, and entropy. The proposed architecture consists of a hardware and a software module. The hardware module is implemented on Xilinx Virtex-E V2000 FPGA using VHDL. It calculates many GLCMs and GLCM integer features in parallel. The software retrieves the feature vectors calculated in hardware, and performs complementary computations. The architecture was evaluated using standard grayscale images and video clips. The results show that it can be efficiently used in realtime pattern recognition applications.

/pdf/an-fpga-based-architecture-for-real-time-image-feature-1qk4goau45.pdf

An FPGA-based architecture for real time image feature extraction

Background: Complementary DNA (cDNA) microarrays are a well established technology for studying gene expression. A microarray image is obtained by laser scanning a hybridized cDNA microarray, which consists of thousands of spots representing chains of cDNA sequences, arranged in a two-dimensional array. The separation of the spots into distinct cells is widely known as microarray image gridding. Methods: In this paper we propose M 3 G, a novel method for automatic gridding of cDNA microarray images based on the maximization of the margin between the rows and the columns of the spots. Initially the microarray image rotation is estimated and then a pre-processing algorithm is applied for a rough spot detection. In order to diminish the effect of artefacts, only a subset of the detected spots is selected by matching the distribution of the spot sizes to the normal distribution. Then, a set of grid lines is placed on the image in order to separate each pair of consecutive rows and columns of the selected spots. The optimal positioning of the lines is determined by maximizing the margin between these rows and columns by using a maximum margin linear classifier, effectively facilitating the localization of the spots. Results: The experimental evaluation was based on a reference set of microarray images containing more than two million spots in total. The results show that M 3 G outperforms state of the art methods, demonstrating robustness in the presence of noise and artefacts. More than 98% of the spots reside completely inside their respective grid cells, whereas the mean distance between the spot center and the grid cell center is 1.2 pixels. Conclusions: The proposed method performs highly accurate gridding in the presence of noise and artefacts, while taking into account the input image rotation. Thus, it provides the potential of achieving perfect gridding for the vast majority of the spots.

/pdf/m3g-maximum-margin-microarray-gridding-3k5zz984yo.pdf

M3G: Maximum Margin Microarray Gridding

http://rtsimage.di.uoa.gr/publications/Pattern%20Recognition_2012.pdf

Unsupervised 2D gel electrophoresis image segmentation based on active contours

Capsule endoscopy is a non-invasive imaging technique commonly used for screening of the entire small intestine. It is performed by a wireless swallowable endoscopic capsule capable of transmitting thousands of video frames per examination. The visual inspection of the vast amount of images acquired during such an examination is a subjective and highly time consuming task even for experienced gastroenterologists. In this paper we propose a novel approach to the reduction of the number of the video frames to be inspected so as to enable faster inspection of the endoscopic video. It is based on symmetric non-negative matrix factorisation initialised by the fuzzy c-means algorithm and it is supported by non-negative Lagrangian relaxation to extract a subset of video frames containing the most representative scenes from a whole endoscopic examination. The experimental evaluation of the proposed approach was tested on annotated endoscopic videos with frames displaying ulcers, bleedings and normal tissues from various sites in the small intestine. The results demonstrate that the video summary produced consists of representative frames from all the abnormal findings and the normal tissues of the input video.

Dimitris Maroulis

Papers

An FPGA-based architecture for real time image feature extraction

M3G: Maximum Margin Microarray Gridding

Unsupervised 2D gel electrophoresis image segmentation based on active contours

Unsupervised summarisation of capsule endoscopy video

A computer-aided system for malignancy risk assessment of nodules in thyroid US images based on boundary features