Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples. Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters, so that the model learns how to extract high-quality features. In recent years, due to the rapid development of internet technology, we are in an era of information torrents and we have massive amounts of data. In this way, DL has aroused strong interest of researchers and has been rapidly developed. Compared with DL, researchers have relatively low interest in AL. This is mainly because before the rise of DL, traditional machine learning requires relatively few labeled samples. Therefore, early AL is difficult to reflect the value it deserves. Although DL has made breakthroughs in various fields, most of this success is due to the publicity of the large number of existing annotation datasets. However, the acquisition of a large number of high-quality annotated datasets consumes a lot of manpower, which is not allowed in some fields that require high expertise, especially in the fields of speech recognition, information extraction, medical images, etc. Therefore, AL has gradually received due attention. A natural idea is whether AL can be used to reduce the cost of sample annotations, while retaining the powerful learning capabilities of DL. Therefore, deep active learning (DAL) has emerged. Although the related research has been quite abundant, it lacks a comprehensive survey of DAL. This article is to fill this gap, we provide a formal classification method for the existing work, and a comprehensive and systematic overview. In addition, we also analyzed and summarized the development of DAL from the perspective of application. Finally, we discussed the confusion and problems in DAL, and gave some possible development directions for DAL.

A Survey of Deep Active Learning

The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease. In this Review, we outline the key applications enabled, along with the technical and analytical challenges. We explore opportunities in personalized medicine, digital clinical trials, remote monitoring and care, pandemic surveillance, digital twin technology and virtual health assistants. Further, we survey the data, modeling and privacy challenges that must be overcome to realize the full potential of multimodal artificial intelligence in health. 

/pdf/multimodal-biomedical-ai-1l2w2dny.pdf

Multimodal biomedical AI

/pdf/artificial-intelligence-ai-and-internet-of-medical-things-20rer7my.pdf

Artificial Intelligence (AI) and Internet of Medical Things (IoMT) Assisted Biomedical Systems for Intelligent Healthcare

Literature survey shows that convolutional neural network (CNN)-based pretrained models have been largely used for CoronaVirus Disease 2019 (COVID-19) classification using chest X-ray (CXR) and computed tomography (CT) datasets. However, most of the methods have used a smaller number of data samples for both CT and CXR datasets for training, validation, and testing. As a result, the model might have shown good performance during testing, but this type of model will not be more effective on unseen COVID-19 data samples. Generalization is an important term to be considered while designing a classifier that can perform well on completely unseen datasets. Here, this work proposes a large-scale learning with stacked ensemble meta-classifier and deep learning-based feature fusion approach for COVID-19 classification. The features from the penultimate layer (global average pooling) of EfficientNet-based pretrained models were extracted and the dimensionality of the extracted features reduced using kernel principal component analysis (PCA). Next, a feature fusion approach was employed to merge the features of various extracted features. Finally, a stacked ensemble meta-classifier-based approach was used for classification. It is a two-stage approach. In the first stage, random forest and support vector machine (SVM) were applied for prediction, then aggregated and fed into the second stage. The second stage includes logistic regression classifier that classifies the data sample of CT and CXR into either COVID-19 or Non-COVID-19. The proposed model was tested using large CT and CXR datasets, which are publicly available. The performance of the proposed model was compared with various existing CNN-based pretrained models. The proposed model outperformed the existing methods and can be used as a tool for point-of-care diagnosis by healthcare professionals.

/pdf/deep-learning-based-meta-classifier-approach-for-covid-19-1mc4xmkvya.pdf

Deep learning-based meta-classifier approach for COVID-19 classification using CT scan and chest X-ray images.

Innovative Materials Science via Machine Learning

A decade of unprecedented progress in artificial intelligence (AI) has demonstrated the potential for many fields-including medicine-to benefit from the insights that AI techniques can extract from data. Here we survey recent progress in the development of modern computer vision techniques-powered by deep learning-for medical applications, focusing on medical imaging, medical video, and clinical deployment. We start by briefly summarizing a decade of progress in convolutional neural networks, including the vision tasks they enable, in the context of healthcare. Next, we discuss several example medical imaging applications that stand to benefit-including cardiology, pathology, dermatology, ophthalmology-and propose new avenues for continued work. We then expand into general medical video, highlighting ways in which clinical workflows can integrate computer vision to enhance care. Finally, we discuss the challenges and hurdles required for real-world clinical deployment of these technologies.

/pdf/deep-learning-enabled-medical-computer-vision-288ejsehmj.pdf

Deep learning-enabled medical computer vision.

Active learning aims to develop label-efficient algorithms by querying the most informative samples to be labeled by an oracle. The design of efficient training methods that require fewer labels is an important research direction that allows more effective use of computational and human resources for labeling and training deep neural networks. In this work, we demonstrate how we can use recent advances in deep generative models, to outperform the state-of-the-art in achieving the highest classification accuracy using as few labels as possible. Unlike previous approaches, our approach uses not only labeled images to train the classifier but also unlabeled images and generated images for co-training the whole model. Our experiments show that the proposed method significantly outperforms existing approaches in active learning on a wide range of datasets (MNIST, CIFAR-10, SVHN, CelebA, and ImageNet).

Adversarial Representation Active Learning

We study the problem of medical symptoms recognition from patient text, for the purposes of gathering pertinent information from the patient (known as history-taking). A typical patient text is often descriptive of the symptoms the patient is experiencing and a single instance of such a text can be "labeled" with multiple symptoms. This makes learning a medical symptoms recognizer challenging on account of i) the lack of availability of voluminous annotated data as well as ii) the large unknown universe of multiple symptoms that a single text can map to. Furthermore, patient text is often characterized by a long tail in the data (i.e., some labels/symptoms occur more frequently than others for e.g "fever" vs "hematochezia"). In this paper, we introduce an active learning method that leverages underlying structure of a continually refined, learned latent space to select the most informative examples to label. This enables the selection of the most informative examples that progressively increases the coverage on the universe of symptoms via the learned model, despite the long tail in data distribution.

Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions.

. Automatic surgical activity recognition enables more intelli-gent surgical devices and a more eﬃcient workﬂow. Integration of such technology in new operating rooms has the potential to improve care delivery to patients and decrease costs. Recent works have achieved a promising performance on surgical activity recognition; however, the lack of generalizability of these models is one of the critical barriers to the wide-scale adoption of this technology. In this work, we study the generalizability of surgical activity recognition models across operating rooms. We propose a new domain adaptation method to improve the performance of the surgical activity recognition model in a new operating room for which we only have unlabeled videos. Our approach generates pseudo labels for unlabeled video clips that it is conﬁdent about and trains the model on the augmented version of the clips. We extend our method to a semi-supervised domain adaptation setting where a small portion of the target domain is also labeled. In our experiments, our proposed method consistently outperforms the baselines on a dataset of more than 480 long surgical videos collected from two operating rooms.

Adaptation of Surgical Activity Recognition Models Across Operating Rooms

Activity recognition in surgical videos is a key research area for developing next-generation devices and workflow monitoring systems. Since surgeries are long processes with highly-variable lengths, deep learning models used for surgical videos often consist of a two-stage setup using a backbone and temporal sequence model. In this paper, we investigate many state-of-the-art backbones and temporal models to find architectures that yield the strongest performance for surgical activity recognition. We first benchmark the models performance on a large-scale activity recognition dataset containing over 800 surgery videos captured in multiple clinical operating rooms. We further evaluate the models on the two smaller public datasets, the Cholec80 and Cataract-101 datasets, containing only 80 and 101 videos respectively. We empirically found that Swin-Transformer+BiGRU temporal model yielded strong performance on both datasets. Finally, we investigate the adaptability of the model to new domains by fine-tuning models to a new hospital and experimenting with a recent unsupervised domain adaptation approach.

Ali Mottaghi

Papers

Deep learning-enabled medical computer vision.

Adversarial Representation Active Learning

Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions.

Adaptation of Surgical Activity Recognition Models Across Operating Rooms

An Empirical Study on Activity Recognition in Long Surgical Videos