Abstract Data scarcity is a major challenge when training deep learning (DL) models. DL demands a large amount of data to achieve exceptional performance. Unfortunately, many applications have small or inadequate data to train DL frameworks. Usually, manual labeling is needed to provide labeled data, which typically involves human annotators with a vast background of knowledge. This annotation process is costly, time-consuming, and error-prone. Usually, every DL framework is fed by a significant amount of labeled data to automatically learn representations. Ultimately, a larger amount of data would generate a better DL model and its performance is also application dependent. This issue is the main barrier for many applications dismissing the use of DL. Having sufficient data is the first step toward any successful and trustworthy DL application. This paper presents a holistic survey on state-of-the-art techniques to deal with training DL models to overcome three challenges including small, imbalanced datasets, and lack of generalization. This survey starts by listing the learning techniques. Next, the types of DL architectures are introduced. After that, state-of-the-art solutions to address the issue of lack of training data are listed, such as Transfer Learning (TL), Self-Supervised Learning (SSL), Generative Adversarial Networks (GANs), Model Architecture (MA), Physics-Informed Neural Network (PINN), and Deep Synthetic Minority Oversampling Technique (DeepSMOTE). Then, these solutions were followed by some related tips about data acquisition needed prior to training purposes, as well as recommendations for ensuring the trustworthiness of the training dataset. The survey ends with a list of applications that suffer from data scarcity, several alternatives are proposed in order to generate more data in each application including Electromagnetic Imaging (EMI), Civil Structural Health Monitoring, Medical imaging, Meteorology, Wireless Communications, Fluid Mechanics, Microelectromechanical system, and Cybersecurity. To the best of the authors’ knowledge, this is the first review that offers a comprehensive overview on strategies to tackle data scarcity in DL. 

/pdf/a-survey-on-deep-learning-tools-dealing-with-data-scarcity-3amzf9cp.pdf

A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications

Artificial intelligence (AI) and natural language processing (NLP) have found a highly promising application in automated clinical coding (ACC), an innovation that will have profound impacts on the clinical coding industry, billing and revenue management, and potentially clinical care itself. Dong et al. recently analyzed the technical challenges of ACC and proposed future directions. Primary challenges for ACC exist at the technological and implementation levels; clinical documents are redundant and complex, code sets like the ICD-10 are rapidly evolving, training sets are not comprehensive of codes, and ACC models have yet to fully capture the logic and rules of coding decisions. Next steps include interdisciplinary collaboration with clinical coders, accessibility and transparency of datasets, and tailoring models to specific use cases. 

/pdf/automating-the-overburdened-clinical-coding-system-1heniz4y.pdf

Automating the overburdened clinical coding system: challenges and next steps

/pdf/bottom-up-and-top-down-paradigms-of-artificial-intelligence-2hnv289w.pdf

Bottom-up and top-down paradigms of artificial intelligence research approaches to healthcare data science using growing real-world big data

Antimicrobial resistance (AMR) has become one of the serious global health problems, threatening the effective treatment of a growing number of infections. Machine learning and deep learning show great potential in rapid and accurate AMR predictions. However, a large number of samples for the training of these models is essential. In particular, for novel antibiotics, limited training samples and data imbalance hinder the models’ generalization performance and overall accuracy. We propose a deep transfer learning model that can improve model performance for AMR prediction on small, imbalanced datasets. As our approach relies on transfer learning and secondary mutations, it is also applicable to novel antibiotics and emerging resistances in the future and enables quick diagnostics and personalized treatments.

/pdf/deep-transfer-learning-enables-robust-prediction-of-kdi2w319.pdf

Deep Transfer Learning Enables Robust Prediction of Antimicrobial Resistance for Novel Antibiotics

INTRODUCTION
Deep learning may be able to assist with the prediction of neurosurgical inpatient outcomes. The aims of this study were to investigate deep learning and transfer learning in the prediction of several inpatient outcomes including timing of discharge and discharge destination.


METHOD
Data were collected on consecutive neurosurgical admissions from existing databases over a 15-month period. Following pre-processing artificial neural networks were applied to admission notes and ward round notes to predict four inpatient outcomes. Models were developed on the training dataset, before being tested on a hold-out test dataset and a validation dataset.


RESULTS
1341 individual admissions were included in the study. Using transfer learning and an artificial neural network an area under the receiver operator curve (AUC) of 0.81 and 0.80 on the derivation and validation datasets was able to be achieved for the prediction of discharge within the next 48 hours using daily ward round notes. This result is in comparison to an AUC of 0.71 and 0.68 using an artificial neural network without transfer learning for the same outcome. When the artificial neural network with transfer learning was applied to the other outcomes AUC of 0.72, 0.93 and 0.83 was achieved on the validation datasets for predicting discharge within the next 7 days, survival to discharge and discharge to home as a destination.


CONCLUSIONS
Deep learning may predict inpatient neurosurgery outcomes from free-text medical data. Recurrent predictions with ward round notes enable the use of information obtained throughout hospital admissions in these estimates.

Neurosurgery inpatient outcome prediction for discharge planning with deep learning and transfer learning.

Background Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature. Methods and Findings We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%). Conclusions In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

https://medrxiv.org/content/10.1101/2021.10.01.21264290v1.full.pdf

Mette Østergaard Thunbo

Papers

Transfer learning for non-image data in clinical research: a scoping review