/pdf/il-mcam-an-interactive-learning-and-multi-channel-attention-229cl00v.pdf

IL-MCAM: An interactive learning and multi-channel attention mechanism-based weakly supervised colorectal histopathology image classification approach

Microorganisms play a vital role in human life. Therefore, microorganism detection is of great significance to human beings. However, the traditional manual microscopic detection methods have the disadvantages of long detection cycle, low detection accuracy in large orders, and great difficulty in detecting uncommon microorganisms. Therefore, it is meaningful to apply computer image analysis technology to the field of microorganism detection. Computer image analysis can realize high-precision and high-efficiency detection of microorganisms. In this review, first,we analyse the existing microorganism detection methods in chronological order, from traditional image processing and traditional machine learning to deep learning methods. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. In the end, the future development direction and challenges of microorganism detection are discussed. In general, we have summarized 142 related technical papers from 1985 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of microorganism detection and provide a reference for researchers in other fields. 

/pdf/a-state-of-the-art-survey-of-object-detection-techniques-in-1muv2dnu.pdf

A state-of-the-art survey of object detection techniques in microorganism image analysis: from classical methods to deep learning approaches

The detection of tiny objects in microscopic videos is a problematic point, especially in large-scale experiments. For tiny objects (such as sperms) in microscopic videos, current detection methods face challenges in fuzzy, irregular, and precise positioning of objects. In contrast, we present a convolutional neural network for tiny object detection (TOD-CNN) with an underlying data set of high-quality sperm microscopic videos (111 videos, > 278,000 annotated objects), and a graphical user interface (GUI) is designed to employ and test the proposed model effectively. TOD-CNN is highly accurate, achieving 85.60% AP50 in the task of real-time sperm detection in microscopic videos. To demonstrate the importance of sperm detection technology in sperm quality analysis, we carry out relevant sperm quality evaluation metrics and compare them with the diagnosis results from medical doctors.

TOD-CNN: An Effective Convolutional Neural Network for Tiny Object Detection in Sperm Videos

/pdf/tod-cnn-an-effective-convolutional-neural-network-for-tiny-20s2re38.pdf

TOD-CNN: An effective convolutional neural network for tiny object detection in sperm videos

Background: Breast cancer has the highest prevalence among all cancers in women globally. The classification of histopathological images in the diagnosis of breast cancers is an area of clinical concern. In computer-aided diagnosis, most traditional classification models use a single network to extract features, although this approach has significant limitations. Moreover, many networks are trained and optimized on patient-level datasets, ignoring lower-level data labels. Method: This paper proposes a deep ensemble model based on image-level labels for the binary classification of breast histopathological images of benign and malignant lesions. First, the BreaKHis dataset is randomly divided into training, validation, and test sets. Then, data augmentation techniques are used to balance the numbers of benign and malignant samples. Third, based on their transfer learning performance and the complementarity between networks, VGG16, Xception, ResNet50, and DenseNet201 are selected as base classifiers. Results: In a ensemble network model with accuracy as the weight, the image-level binary classification achieves an accuracy of 98.90%. To verify the capabilities of our method, it is experimentally compared with the latest Transformer and multilayer perception (MLP) models on the same dataset. Our ensemble model shows a a 5%–20% advantage, emphasizing its far-reaching abilities in classification tasks. Conclusion: This research focuses on improving the performance of a classification model with an ensemble algorithm. Transfer learning has an essential role in classification of small datasets, improving training speed and accuracy. Our model outperforms many existing approaches with respect to accuracy and has applications in the field of auxiliary medical diagnosis. 

Application of transfer learning and ensemble learning in image-level classification for breast histopathology

SVIA dataset: A new dataset of microscopic videos and images for computer-aided sperm analysis

Global modeling and integrating over feature relations are beneficial for semantic segmentation. Previous researches excel at modeling features through convolution operations but are inefficient at associating features between distant regions. Consequently, key information within one semantic category can be neglected if the features between distant regions are not bridged. To solve this issue, we propose a novel Attention Enhanced Graph Convolutional Network to explore relational information and preserve details in semantic segmentation tasks. We first build a graph over feature maps, where each node is represented by features in spatial and channel dimensions. Then the interdependent semantic information is obtained through applying the GCN to globally bridge co-occurrence features regardless of their distance. Finally, we introduce the self-attention mechanism in spatial and channel dimensions respectively to project the semantic-aware information into feature maps so as to capture the discriminative information. We examine the merit of our model on three challenging scene segmentation datasets: Cityscapes, PASCAL VOC2012 and COCO Stuff. Experimental results show that our model boosts the performance over other state-of-the-arts. Code will be available.

An Attention Enhanced Graph Convolutional Network for Semantic Segmentation

Currently, the field of transparent image analysis has gradually become a hot topic. However, traditional analysis methods are accompanied by large amounts of carbon emissions, and consumes significant manpower and material resources. The continuous development of computer vision enables the use of computers to analyze images. However, the low contrast between the foreground and background of transparent images makes their segmentation difficult for computers. To address this problem, we first analyzed them with pixel patches, and then classified the patches as foreground and background. Finally, the segmentation of the transparent images was completed through the reconstruction of pixel patches. To understand the performance of different deep learning networks in transparent image segmentation, we conducted a series of comparative experiments using patch-level and pixel-level methods. In two sets of experiments, we compared the segmentation performance of four convolutional neural network (CNN) models and a visual transformer (ViT) model on the transparent environmental microorganism dataset fifth version. The results demonstrated that U-Net++ had the highest accuracy rate of 95.32% in the pixel-level segmentation experiment followed by ViT with an accuracy rate of 95.31%. However, ResNet50 had the highest accuracy rate of 90.00% and ViT had the lowest accuracy of 89.25% in the patch-level segmentation experiments. Hence, we concluded that ViT performed the lowest in patch-level segmentation experiments, but outperformed most CNNs in pixel-level segmentation. Further, we combined patch-level and pixel-level segmentation results to reduce the loss of segmentation details in the EM images. This conclusion was also verified by the environmental microorganism dataset sixth version dataset (EMDS-6).

/pdf/comparative-study-for-patch-level-and-pixel-level-12bsq3rf.pdf

Comparative Study for Patch-Level and Pixel-Level Segmentation of Deep Learning Methods on Transparent Images of Environmental Microorganisms: From Convolutional Neural Networks to Visual Transformers

Background With the gradual increase of infertility in the world nowadays, among which male sperm problems are the main factor of infertility, more and more couples are using Computer Assisted Sperm Analysis to assist in the analysis and treatment of infertility. Meanwhile, the rapid development of deep learning has led to strong results in image classification tasks. However, the classification of sperm images has not been well studied in current deep learning methods, and the sperm images are often disturbed by noise in practical Computer Assisted Sperm Analysis applications. The purpose of this paper is to come to investigate the anti-noise robustness of deep learning classification methods on sperm images. Methods The SVIA dataset is a publicly large-scale sperm dataset containing three subsets. In this paper, we used subset-C, which provides more than 125,000 independent images of sperms and impurities, including 121,401 sperm images and 4,479 impurity images. In order to investigate the anti-noise robustness of deep learning classification methods on sperm images. we conducted a comprehensive comparative study of sperm images using many Convolutional Neural Network and Visual Transformer deep learning methods to find the deep learning model with the most stable anti-noise robustness. Results This study proved that ViT had strong robustness for the classification of the tiny object (sperm and impurity) image datasets under some conventional noises and adversarial attacks. In particular, under the influence of Poisson noise, the accuracy changed from 91.45% to 91.08%, the precision of impurity changes from 92.7% to 91.3%, the recall of impurity changed from 88.8% to 89.5%, and the F1-score of impurity changed 90.7% to 90.4%. Meanwhile, the precision of sperm changed from 90.9% to 90.5%, the recall of sperm changed from 92.5% to 93.8%, and the F1-score of sperm changed from 92.1% to 90.4%. Conclusion We obtain two conclusions: the image classification effects of sperm images are strongly affected by noise in current deep learning methods; the robustness against noise of VT methods based on global information is stronger than that of CNN methods based on local information, indicating that the robustness against noise is mainly reflected in global information. 

A Comprehensive Comparative Study of Deep Learning Methods for Noisy Sperm Image Classification: from Convolutional Neural Network to Visual Transformer

—The accurate detection of sperms and impurities is a very challenging task, facing problems such as the small size of targets, indeﬁnite target morphologies, low contrast and resolution of the video, and similarity of sperms and impurities. So far, the detection of sperms and impurities still largely relies on the traditional image processing and detection techniques which only yield limited performance and often require manual intervention in the detection process, therefore unfavorably escalating the time cost and injecting the subjective bias into the analysis. Encouraged by the successes of deep learning methods in numerous object detection tasks, here we report a deep learning model based on Double Branch Feature Extraction Network (DBFEN) and Cross-conjugate Feature Pyramid Networks (CCFPN). DBFEN is designed to extract visual features from tiny objects with a double branch structure, and CCFPN is further introduced to fuse the features extracted by DBFEN to enhance the description of position and high-level semantic information. Our work is the pioneer of introducing deep learning approaches to the detection of sperms and impurities. Experiments show that the highest AP 50 of the sperm and impurity detection is 91.13% and 59.64%, which lead its competitors by a substantial margin and establish new state-of-the-art results in this problem.

Ao Chen

Papers

SVIA dataset: A new dataset of microscopic videos and images for computer-aided sperm analysis

An Attention Enhanced Graph Convolutional Network for Semantic Segmentation

Comparative Study for Patch-Level and Pixel-Level Segmentation of Deep Learning Methods on Transparent Images of Environmental Microorganisms: From Convolutional Neural Networks to Visual Transformers

A Comprehensive Comparative Study of Deep Learning Methods for Noisy Sperm Image Classification: from Convolutional Neural Network to Visual Transformer

ACTIVE: A Deep Model for Sperm and Impurity Detection in Microscopic Videos