Despite significant successes achieved in knowledge discovery, traditional machine learning methods may fail to obtain satisfactory performances when dealing with complex data, such as imbalanced, high-dimensional, noisy data, etc. The reason behind is that it is difficult for these methods to capture multiple characteristics and underlying structure of data. In this context, it becomes an important topic in the data mining field that how to effectively construct an efficient knowledge discovery and mining model. Ensemble learning, as one research hot spot, aims to integrate data fusion, data modeling, and data mining into a unified framework. Specifically, ensemble learning firstly extracts a set of features with a variety of transformations. Based on these learned features, multiple learning algorithms are utilized to produce weak predictive results. Finally, ensemble learning fuses the informative knowledge from the above results obtained to achieve knowledge discovery and better predictive performance via voting schemes in an adaptive way. In this paper, we review the research progress of the mainstream approaches of ensemble learning and classify them based on different characteristics. In addition, we present challenges and possible research directions for each mainstream approach of ensemble learning, and we also give an extra introduction for the combination of ensemble learning with other machine learning hot spots such as deep learning, reinforcement learning, etc.

A survey on ensemble learning

This paper presents a novel computer-aided diagnosis (CAD) technique for the early diagnosis of the Alzheimer's disease (AD) based on nonnegative matrix factorization (NMF) and support vector machines (SVM) with bounds of confidence. The CAD tool is designed for the study and classification of functional brain images. For this purpose, two different brain image databases are selected: a single photon emission computed tomography (SPECT) database and positron emission tomography (PET) images, both of them containing data for both Alzheimer's disease (AD) patients and healthy controls as a reference. These databases are analyzed by applying the Fisher discriminant ratio (FDR) and nonnegative matrix factorization (NMF) for feature selection and extraction of the most relevant features. The resulting NMF-transformed sets of data, which contain a reduced number of features, are classified by means of a SVM-based classifier with bounds of confidence for decision. The proposed NMF-SVM method yields up to 91% classification accuracy with high sensitivity and specificity rates (upper than 90%). This NMF-SVM CAD tool becomes an accurate method for SPECT and PET AD image classification.

NMF-SVM Based CAD Tool Applied to Functional Brain Images for the Diagnosis of Alzheimer's Disease

The ultrasound imaging is one of the most common schemes to detect diseases in the clinical practice. There are many advantages of ultrasound imaging such as safety, convenience, and low cost. However, reading ultrasound imaging is not easy. To support the diagnosis of clinicians and reduce the load of doctors, many ultrasound computer-aided diagnosis (CAD) systems are proposed. In recent years, the success of deep learning in the image classification and segmentation led to more and more scholars realizing the potential of performance improvement brought by utilizing the deep learning in the ultrasound CAD system. This paper summarized the research which focuses on the ultrasound CAD system utilizing machine learning technology in recent years. This study divided the ultrasound CAD system into two categories. One is the traditional ultrasound CAD system which employed the manmade feature and the other is the deep learning ultrasound CAD system. The major feature and the classifier employed by the traditional ultrasound CAD system are introduced. As for the deep learning ultrasound CAD, newest applications are summarized. This paper will be useful for researchers who focus on the ultrasound CAD system.

/pdf/machine-learning-in-ultrasound-computer-aided-diagnostic-58mdlay4eg.pdf

Machine Learning in Ultrasound Computer-Aided Diagnostic Systems: A Survey.

Breast cancer is the most common form of cancer among women worldwide. Ultrasound imaging is one of the most frequently used diagnostic tools to detect and classify abnormalities of the breast. Recently, computer-aided diagnosis (CAD) systems using ultrasound images have been developed to help radiologists to increase diagnosis accuracy. However, accurate ultrasound image segmentation remains a challenging problem due to various ultrasound artifacts. In this paper, we investigate approaches developed for breast ultrasound (BUS) image segmentation. In this paper, we reviewed the literature on the segmentation of BUS images according to the techniques adopted, especially over the past 10 years. By dividing into seven classes (i.e., thresholding-based, clustering-based, watershed-based, graph-based, active contour model, Markov random field and neural network), we have introduced corresponding techniques and representative papers accordingly. We have summarized and compared many techniques on BUS image segmentation and found that all these techniques have their own pros and cons. However, BUS image segmentation is still an open and challenging problem due to various ultrasound artifacts introduced in the process of imaging, including high speckle noise, low contrast, blurry boundaries, low signal-to-noise ratio and intensity inhomogeneity To the best of our knowledge, this is the first comprehensive review of the approaches developed for segmentation of BUS images. With most techniques involved, this paper will be useful and helpful for researchers working on segmentation of ultrasound images, and for BUS CAD system developers.

Breast ultrasound image segmentation: a survey

This research aims to address the problem of discriminating benign cysts from malignant masses in breast ultrasound (BUS) images based on Convolutional Neural Networks (CNNs). The biopsy-proven benchmarking dataset was built from 1422 patient cases containing a total of 2058 breast ultrasound masses, comprising 1370 benign and 688 malignant lesions. Three transferred models, InceptionV3, ResNet50, and Xception, a CNN model with three convolutional layers (CNN3), and traditional machine learning-based model with hand-crafted features were developed for differentiating benign and malignant tumors from BUS data. Cross-validation results have demonstrated that the transfer learning method outperformed the traditional machine learning model and the CNN3 model, where the transferred InceptionV3 achieved the best performance with an accuracy of 85.13% and an AUC of 0.91. Moreover, classification models based on deep features extracted from the transferred models were also built, where the model with combined features extracted from all three transferred models achieved the best performance with an accuracy of 89.44% and an AUC of 0.93 on an independent test set.

/pdf/comparison-of-transferred-deep-neural-networks-in-ultrasonic-5gnexoo4nl.pdf

Comparison of Transferred Deep Neural Networks in Ultrasonic Breast Masses Discrimination

This paper proposes a novel algorithm to estimate a log-compressed K distribution parameter and presents an algorithm to discriminate breast tumors in ultrasonic images. We computed a total of 208 features for discrimination, including those based on a parameter of a log-compressed K-distribution, which quantifies the homogeneity of the echo pattern in the tumor, but is influenced by compression parameters in the ultrasonic device. The proposed algorithm estimates the parameter of the log-compressed K-distribution in a manner free from this influence. To quantify irregularities in tumor shape, pattern-spectrum-based features were newly developed in this paper. The discrimination process uses an ensemble classifier trained by a multiclass AdaBoost learning algorithm (AdaBoost.M2), combined with a sequential feature-selection process. A 10-fold cross-validation test validated the performance, and the results were compared with those of a Mahalanobis distance-based classifier and a multiclass support vector machine. A total of 200 carcinomas, 50 fibroadenomas, and 50 cysts were used in the experiments. This paper demonstrates that the combination of a classifier trained by AdaBoost.M2 and features based on the estimated parameter of a log-compressed K-distribution, as well as those of the pattern spectrum, are useful for the discrimination of tumors.

Discrimination of Breast Tumors in Ultrasonic Images Using an Ensemble Classifier Based on the AdaBoost Algorithm With Feature Selection

Purpose
A cost-sensitive extension of AdaBoost based on Markov random field (MRF) priors was developed to train an ensemble segmentation process which can avoid irregular shape, isolated points and holes, leading to lower error rate. The method was applied to breast tumor segmentation in ultrasonic images.

A cost-sensitive extension of AdaBoost with markov random field priors for automated segmentation of breast tumors in ultrasonic images.

Ultrasonic diagnostic equipment is widely used, but has many problems such as acquiring noisy images and poorly quantifying the results for diagnosis. Therefore, a great deal of research has addressed the image processing of medical ultrasound images because of the need for automation and quantification of the diagnosis. This research began with the problem of region segmentation that automatically identifies living tissue or focus mass and currently relates to the quantification of tissue region information with the objective of tissue property diagnosis and the automatic diagnosis of 3D images. To achieve this, an accurate contour extraction method must be established. In this research, to achieve accurate region segmentation without losing the contour information in the medical ultrasound diagnostic images, we propose a new method based on an adaptive smoothing filter for suppressing speckle patterns and the 2D Gabor wavelet transform. Then we tried to automatically detect a superficial peripheral vein from an image of the forearm obtained by high-frequency ultrasonic diagnostic equipment, which is a difficult problem for conventional image analysis techniques. © 2002 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(1): 36–45, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10026

Segmentation of ultrasonic images by using locally adaptive filter and wavelet analysis: Detection of superficial peripheral vein by a high-frequency ultrasonic equipment

This paper proposes a novel e-learning system for technical experiments involving the construction of practical electronic circuits; this system would meet the various demands of individual experimenters. This mixed mode is beneficial for practical use in that an experimenter who does not have sufficient circuit components for circuit making can still learn to construct practical, large-scale circuits and evaluate their behavior using virtual measurements and a simulation technique involving circuit translation into SPICE. The proposed system is expected to be applicable to various educational media (e.g., e-Learning, mobile learning, virtual laboratories). The proposed system is constructed by combining real and virtual circuit making systems, and performs the improved circuit recognition of images including both real and virtual circuit regions. The usefulness and effectiveness of the proposed system were evaluated by analyzing circuits made by ten university students in an actual class.

/pdf/e-learning-system-for-experiments-involving-construction-of-zp71m2vlf1.pdf

E-Learning System for Experiments Involving Construction of Practical Electronic Circuits.

In this paper, we propose a novel method for accurate automated discrimination of breast tumors (carcinoma, fibroadenoma, and cyst). We defined 199 features related to diagnostic observations noticed when a doctor judges breast tumors, such as internal echo, shape, and boundary echo. These features included novel features based on a parameter of log-compressed K distribution, which reflect physical characteristics of ultrasonic B-mode imaging. Furthermore, we propose a discrimination method of breast tumors by using an ensemble classifier based on the multiclass AdaBoost algorithm with effective features selection. Verification by analyzing 200 carcinomas, 30 fibroadenomas, and 30 cysts showed the usefulness of the newly defined features and the effectiveness of the discrimination by using an ensemble classifier trained by AdaBoost. © 2011 Wiley Periodicals, Inc. Electron Comm Jpn, 94(9): 18–29, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10356

Atsushi Takemura

Papers

Discrimination of Breast Tumors in Ultrasonic Images Using an Ensemble Classifier Based on the AdaBoost Algorithm With Feature Selection

A cost-sensitive extension of AdaBoost with markov random field priors for automated segmentation of breast tumors in ultrasonic images.

Segmentation of ultrasonic images by using locally adaptive filter and wavelet analysis: Detection of superficial peripheral vein by a high-frequency ultrasonic equipment

E-Learning System for Experiments Involving Construction of Practical Electronic Circuits.

Discrimination of breast tumors in ultrasonic images by classifier ensemble trained with AdaBoost