We propose a new deep learning approach for medical imaging that copes with the problem of a small training set, the main bottleneck of deep learning, and apply it for classification of healthy and cancer cells acquired by quantitative phase imaging. The proposed method, called transferring of pre-trained generative adversarial network (TOP-GAN), is a hybridization between transfer learning and generative adversarial networks (GANs). Healthy cells and cancer cells of different metastatic potential have been imaged by low-coherence off-axis holography. After the acquisition, the optical path delay maps of the cells have been extracted and directly used as an input to the deep networks. In order to cope with the small number of classified images, we have used GANs to train a large number of unclassified images from another cell type (sperm cells). After this preliminary training, and after transforming the last layer of the network with new ones, we have designed an automatic classifier for the correct cell type (healthy/primary cancer/metastatic cancer) with 90-99% accuracy, although small training sets of down to several images have been used. These results are better in comparison to other classic methods that aim at coping with the same problem of a small training set. We believe that our approach makes the combination of holographic microscopy and deep learning networks more accessible to the medical field by enabling a rapid, automatic and accurate classification in stain-free imaging flow cytometry. Furthermore, our approach is expected to be applicable to many other medical image classification tasks, suffering from a small training set.

TOP-GAN: Label-Free Cancer Cell Classification Using Deep Learning with a Small Training Set

We present a novel multimodal deep learning framework for cardiac resynchronisation therapy (CRT) response prediction from 2D echocardiography and cardiac magnetic resonance (CMR) data. The proposed method first uses the `nnU-Net' segmentation model to extract segmentations of the heart over the full cardiac cycle from the two modalities. Next, a multimodal deep learning classifier is used for CRT response prediction, which combines the latent spaces of the segmentation models of the two modalities. At inference time, this framework can be used with 2D echocardiography data only, whilst taking advantage of the implicit relationship between CMR and echocardiography features learnt from the model. We evaluate our pipeline on a cohort of 50 CRT patients for whom paired echocardiography/CMR data were available, and results show that the proposed multimodal classifier results in a statistically significant improvement in accuracy compared to the baseline approach that uses only 2D echocardiography data. The combination of multimodal data enables CRT response to be predicted with 77.38% accuracy (83.33% sensitivity and 71.43% specificity), which is comparable with the current state-of-the-art in machine learning-based CRT response prediction. Our work represents the first multimodal deep learning approach for CRT response prediction.

A Multimodal Deep Learning Model for Cardiac Resynchronisation Therapy Response Prediction

A multimodal deep learning model for cardiac resynchronisation therapy response prediction

Predicting the response to cardiac resynchronization therapy (CRT) using the deep learning approach

Unsupervised machine learning algorithm to identify high and low risk patients following crt implantation

Cardiac resynchronization therapy (CRT) improves cardiac function in patients with heart failure (HF), and the result of this treatment is decrease in death rate and improving quality of life for patients. This research is aimed at predicting CRT response for the prognosis of patients with heart failure under CRT. According to international instructions, in the case of approval of QRS prolongation and decrease in ejection fraction (EF), the patient is recognized as a candidate of implanting recognition device. However, regarding many intervening and effective factors, decision making can be done based on more variables. Computer-based decision-making systems especially machine learning (ML) are considered as a promising method regarding their significant background in medical prediction. Collective intelligence approaches such as particles swarm optimization (PSO) algorithm are used for determining the priorities of medical decision-making variables. This investigation was done on 209 patients and the data was collected over 12 months. In HESHMAT CRT center, 17.7% of patients did not respond to treatment. Recognizing the dominant parameters through combining machine recognition and physician’s viewpoint, and introducing back-propagation of error neural network algorithm in order to decrease classification error are the most important achievements of this research. In this research, an analytical set of individual, clinical, and laboratory variables, echocardiography, and electrocardiography (ECG) are proposed with patients’ response to CRT. Prediction of the response after CRT becomes possible by the support of a set of tools, algorithms, and variables.

Mohammad Nejadeh

Papers

Predicting the response to cardiac resynchronization therapy (CRT) using the deep learning approach

Evaluation of Pattern Recognition Techniques in Response to Cardiac Resynchronization Therapy (CRT)