Showing papers by "Miguel Ángel González Ballester published in 2022"

Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge

Abstract: Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, location, and surface largely affect identification, localisation, and characterisation. Moreover, colonoscopic surveillance and removal of polyps (referred to as polypectomy ) are highly operator-dependent procedures. There exist a high missed detection rate and incomplete removal of colonic polyps due to their variable nature, the difficulties to delineate the abnormality, the high recurrence rates, and the anatomical topography of the colon. There have been several developments in realising automated methods for both detection and segmentation of these polyps using machine learning. However, the major drawback in most of these methods is their ability to generalise to out-of-sample unseen datasets that come from different centres, modalities and acquisition systems. To test this hypothesis rigorously we curated a multi-centre and multi-population dataset acquired from multiple colonoscopy systems and challenged teams comprising machine learning experts to develop robust automated detection and segmentation methods as part of our crowd-sourcing Endoscopic computer vision challenge (EndoCV) 2021. In this paper, we analyse the detection results of the four top (among seven) teams and the segmentation results of the five top teams (among 16). Our analyses demonstrate that the top-ranking teams concentrated on accuracy (i.e., accuracy > 80% on overall Dice score on different validation sets) over real-time performance required for clinical applicability. We further dissect the methods and provide an experiment-based hypothesis that reveals the need for improved generalisability to tackle diversity present in multi-centre datasets. Introduction Colorectal cancer (CRC) is the third leading cause of cancer deaths, with reported mortality rate of nearly 51%1. CRC can be characterised by early cancer precursors such as adenomas or serrated polyps that may over time lead to cancer. While polypectomy is a standard technique to remove polyps2 by placing a snare (thin wire loop) around the polyp and closing it to cut though the polyp tissue either with diathermy (heat to seal vessels) or without (cold snare polypectomy), identifying small or flat polyps (e.g. lesion less than 10 mm) can be extremely challenging. This is due to complex organ topology of the colon and rectum that make the navigation and treatment procedures difficult and require expert-level skills. Similarly, the removal of polyps can be very challenging due to constant organ deformations which make it sometimes impossible to keep track of the lesion boundary making the complete resection difficult and subjective to experience of endoscopists. Computer-assisted systems can help to reduce operator subjectivity and enables improved adenoma detection rates (ADR). Thus, computer-aided detection and segmentation methods can assist to localise polyps and guide surgical procedures (e.g. polypectomy) by showing the polyp locations and margins. Some of the major requirements of such system to be utilised in clinic are the real-time performance and algorithmic robustness. Machine learning, in particular deep learning, together with tremendous improvements in hardware have enabled the possibility to design networks that can provide real-time performance despite their computational complexity. However, one major challenge in developing these methods is the lack of comprehensive public datasets that include diverse patient population, imaging modalities and scope manufacturers. Incorporating real-world challenges in the dataset can only be the way forward in building guaranteed robust systems. In the past, there has been several attempts to collect and curate gastrointestinal (GI) datasets that include other GI lesions and polyps. A summary of existing related datasets with polyps are provided in Supplementary Table 1. A major limitation of these publicly available datasets is that they consists of either single center or data cohort representing a single population. Additionally, most widely used public datasets have only single frame and single modality images. Moreover, even though conventional white light endoscopy (WLE) is used in standard colonoscopic procedures, narrow-band imaging (NBI), a type of virtual chromo-endoscopy, is widely used by experts for polyp identification and charaterisation. Most deep learning-based detection3–5 and segmentation6–9 methods are trained and tested on the same center dataset and WLE modality only. These supervised deep learning techniques has a major issue in not being able to generalise to an unseen data from a different center population10 or even different modality from the same center11. Also, the type of endoscope used also adds to the compromise in robustness. Due to selective image samples provided by most of the available datasets for method development, the test dataset also comprise of similarly collected set data samples9, 12–14. Similar to the most endoscopic procedures, colonoscopy is a continuous visualisation of mucosa with a camera and a light source. During this process live videos are acquired which are often corrupted with specularity, floating objects, stool, bubbles and pixel saturation15. The mucosal scene dynamics such as severe deformations, view-point changes and occlusion can be some major limiting factors for algorithm performance as well. It is thus important to cross examine the generalisability of developed algorithms more comprehensively and on variable data settings including modality changes and continuous frame sequences. With the presented Endoscopic Computer Vision (EndoCV) challenge in 2021 we collected and curated a multicenter dataset16 that is aimed at generalisability assessment. For this we took an strategic approach of providing single modality (white light endoscopy modality, WLE) data from five hospitals (both single frame and sequence) for training and validation while the

Machine Learning for Clinical Decision-Making: Challenges and Opportunities in Cardiovascular Imaging

Abstract: The use of machine learning (ML) approaches to target clinical problems is called to revolutionize clinical decision-making in cardiology. The success of these tools is dependent on the understanding of the intrinsic processes being used during the conventional pathway by which clinicians make decisions. In a parallelism with this pathway, ML can have an impact at four levels: for data acquisition, predominantly by extracting standardized, high-quality information with the smallest possible learning curve; for feature extraction, by discharging healthcare practitioners from performing tedious measurements on raw data; for interpretation, by digesting complex, heterogeneous data in order to augment the understanding of the patient status; and for decision support, by leveraging the previous steps to predict clinical outcomes, response to treatment or to recommend a specific intervention. This paper discusses the state-of-the-art, as well as the current clinical status and challenges associated with the two later tasks of interpretation and decision support, together with the challenges related to the learning process, the auditability/traceability, the system infrastructure and the integration within clinical processes in cardiovascular imaging.

Regulatory network-based model to simulate the biochemical regulation of chondrocytes in healthy and osteoarthritic environments

Abstract: In osteoarthritis (OA), chondrocyte metabolism dysregulation increases relative catabolic activity, which leads to cartilage degradation. To enable the semiquantitative interpretation of the intricate mechanisms of OA progression, we propose a network-based model at the chondrocyte level that incorporates the complex ways in which inflammatory factors affect structural protein and protease expression and nociceptive signals. Understanding such interactions will leverage the identification of new potential therapeutic targets that could improve current pharmacological treatments. Our computational model arises from a combination of knowledge-based and data-driven approaches that includes in-depth analyses of evidence reported in the specialized literature and targeted network enrichment. We achieved a mechanistic network of molecular interactions that represent both biosynthetic, inflammatory and degradative chondrocyte activity. The network is calibrated against experimental data through a genetic algorithm, and 81% of the responses tested have a normalized root squared error lower than 0.15. The model captures chondrocyte-reported behaviors with 95% accuracy, and it correctly predicts the main outcomes of OA treatment based on blood-derived biologics. The proposed methodology allows us to model an optimal regulatory network that controls chondrocyte metabolism based on measurable soluble molecules. Further research should target the incorporation of mechanical signals.

Test Time Transform Prediction for Open Set Histopathological Image Recognition

Abstract: Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevant tissue categories, while rejecting in test time Open Set samples, i.e. images that belong to categories not present in the training set. To this end, we introduce a new approach for Open Set histopathological image recognition based on training a model to accurately identify image categories and simultaneously predict which data augmentation transform has been applied. In test time, we measure model confidence in predicting this transform, which we expect to be lower for images in the Open Set. We carry out comprehensive experiments in the context of colorectal cancer assessment from histological images, which provide evidence on the strengths of our approach to automatically identify samples from unknown categories. Code is released at https://github.com/agaldran/t3po .

Prediction of Lung Nodule Progression with an Uncertainty-Aware Hierarchical Probabilistic Network

Abstract: Predicting whether a lung nodule will grow, remain stable or regress over time, especially early in its follow-up, would help doctors prescribe personalized treatments and better surgical planning. However, the multifactorial nature of lung tumour progression hampers the identification of growth patterns. In this work, we propose a deep hierarchical generative and probabilistic network that, given an initial image of the nodule, predicts whether it will grow, quantifies its future size and provides its expected semantic appearance at a future time. Unlike previous solutions, our approach also estimates the uncertainty in the predictions from the intrinsic noise in medical images and the inter-observer variability in the annotations. The evaluation of this method on an independent test set reported a future tumour growth size mean absolute error of 1.74 mm, a nodule segmentation Dice’s coefficient of 78% and a tumour growth accuracy of 84% on predictions made up to 24 months ahead. Due to the lack of similar methods for providing future lung tumour growth predictions, along with their associated uncertainty, we adapted equivalent deterministic and alternative generative networks (i.e., probabilistic U-Net, Bayesian test dropout and Pix2Pix). Our method outperformed all these methods, corroborating the adequacy of our approach.

Identification of Myocardial Insulin Resistance by Using Liver Tests: A Simple Approach for Clinical Practice

Abstract: Background: We report that myocardial insulin resistance (mIR) occurs in around 60% of patients with type 2 diabetes (T2D) and was associated with higher cardiovascular risk in comparison with patients with insulin-sensitive myocardium (mIS). These two phenotypes (mIR vs. mIS) can only be assessed using time-consuming and expensive methods. The aim of the present study is to search a simple and reliable surrogate to identify both phenotypes. Methods: Forty-seven patients with T2D underwent myocardial [18F]FDG PET/CT at baseline and after a hyperinsulinemic–euglycemic clamp (HEC) to determine mIR were prospectively recruited. Biochemical assessments were performed before and after the HEC. Baseline hepatic steatosis index and index of hepatic fibrosis (FIB-4) were calculated. Furthermore, liver stiffness measurement was performed using transient elastography. Results: The best model to predict the presence of mIR was the combination of transaminases, protein levels, FIB-4 score and HOMA (AUC = 0.95; sensibility: 0.81; specificity: 0.95). We observed significantly higher levels of fibrosis in patients with mIR than in those with mIS (p = 0.034). In addition, we found that patients with mIR presented a reduced glucose uptake by the liver in comparison with patients with mIS. Conclusions: The combination of HOMA, protein, transaminases and FIB-4 is a simple and reliable tool for identifying mIR in patients with T2D. This information will be useful to improve the stratification of cardiovascular risk in T2D.

The PNt-Methodology: a top-down network modelling approach to estimate dose- and time-dependent cell responses to complex multifactorial environments

Abstract: Current methodologies to estimate cell responses usually focus on bottom-up intracellular network modeling, accompanied by limitations regarding the network topology and regarding the consideration of transient changes in cell responses due to chronic, dose-dependent stimulus exposure. Here we present a novel high-level top-down network modelling approach to simulate dose- and time-dependent cell responses within heterogeneous multifactorial, multicellular environments. Through the representation of cell activities as time-dependent parallel networks (PNt), the PNt-Methodology provides relative, interrelated cell responses for heterogeneous stimulus environments. The methodology is based on a systematic use of experimental findings, both data- and knowledge-based. Applied to intervertebral disc multicellular systems, the PNt-Methodology allows for the first time to obtain qualitatively validated cell responses for daily human moving activities and exposure to microgravity. The PNt-Methodology is not tissue specific per-se and is designed to be easily scalable. Eventually, it provides a one-of-a-kind network modeling approach to approximate complex multicellular systems.

Attri-VAE: attribute-based, disentangled and interpretable representations of medical images with variational autoencoders

Abstract: Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in reasoning medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attributes. Attribute-based methods enforcing attribute disentanglement have been proposed in the literature for classical computer vision tasks in benchmark data. In this paper, we propose a VAE approach, the Attri-VAE, that includes an attribute regularization term to associate clinical and medical imaging attributes with di ff erent regularized dimensions in the generated latent space, enabling a better disentangled interpretation of the attributes. Furthermore, the generated attention maps explained the attribute encoding in the regularized latent space dimensions. The Attri-VAE approach analyzed healthy and myocardial infarction patients with clinical, cardiac morphology, and radiomics attributes. The proposed model provided an excellent trade-o ff between reconstruction fidelity, disentanglement, and interpretability, outperforming state-of-the-art VAE approaches according to several quantitative metrics. The resulting latent space allowed the generation of realistic synthetic data in the trajectory between two distinct input samples or along a specific attribute dimension to better interpret changes between di ff erent cardiac conditions.

PIRET—A Platform for Treatment Planning in Electroporation-Based Therapies

Abstract: Tissue electroporation is the basis of several therapies. Electroporation is performed by briefly exposing tissues to high electric fields. It is generally accepted that electroporation is effective where an electric field magnitude threshold is overreached. However, it is difficult to preoperatively estimate the field distribution because it is highly dependent on anatomy and treatment parameters. Objective: We developed PIRET, a platform to predict the treatment volume in electroporation-based therapies. Methods: The platform seamlessly integrates tools to build patient-specific models where the electric field is simulated to predict the treatment volume. Patient anatomy is segmented from medical images and 3D reconstruction aids in placing the electrodes and setting up treatment parameters. Results: Four canine patients that had been treated with high-frequency irreversible electroporation were retrospectively planned with PIRET and with a workflow commonly used in previous studies, which uses different general-purpose segmentation (3D Slicer) and modeling software (3Matic and COMSOL Multiphysics). PIRET outperformed the other workflow by 65 minutes (× 1.7 faster), thanks to the improved user experience during treatment setup and model building. Both approaches computed similarly accurate electric field distributions, with average Dice scores higher than 0.93. Conclusion: A platform which integrates all the required tools for electroporation treatment planning is presented. Treatment plan can be performed rapidly with minimal user interaction in a stand-alone platform. Significance: This platform is, to the best of our knowledge, the most complete software for treatment planning of irreversible electroporation. It can potentially be used for other electroporation applications.

An automatic pipeline for atlas-based fetal and neonatal brain segmentation and analysis

Abstract: BACKGROUND AND OBJECTIVE The automatic segmentation of perinatal brain structures in magnetic resonance imaging (MRI) is of utmost importance for the study of brain growth and related complications. While different methods exist for adult and pediatric MRI data, there is a lack for automatic tools for the analysis of perinatal imaging. METHODS In this work, a new pipeline for fetal and neonatal segmentation has been developed. We also report the creation of two new fetal atlases, and their use within the pipeline for atlas-based segmentation, based on novel registration methods. The pipeline is also able to extract cortical and pial surfaces and compute features, such as curvature, local gyrification index, sulcal depth, and thickness. RESULTS Results show that the introduction of the new templates together with our segmentation strategy leads to accurate results when compared to expert annotations, as well as better performances when compared to a reference pipeline (developing Human Connectome Project (dHCP)), for both early and late-onset fetal brains. CONCLUSIONS These findings show the potential of the presented atlases and the whole pipeline for application in both fetal, neonatal, and longitudinal studies, which could lead to dramatic improvements in the understanding of perinatal brain development.

Relationship Between the Choice of Clinical Treatment, Gait Functionality and Kinetics in Patients With Comparable Knee Osteoarthritis

Abstract: Objective: The objective of this study was to investigate the relationship between the choice of clinical treatment, gait functionality, and kinetics in patients with comparable knee osteoarthritis. Design: This was an observational case-control study. Setting: The study was conducted in a university biomechanics laboratory. Participants: Knee osteoarthritis patients were stratified into the following groups: clinical treatment (conservative/total knee replacement (TKR) planned), sex (male/female), age (60–67/68–75), and body mass index (BMI) (<30/≥30). All patients had a Kellgren–Lawrence score of 2 or 3 (N = 87). Main Outcome Measures: All patients underwent gait analysis, and two groups of dependent variables were extracted: • Spatiotemporal gait variables: gait velocity, stride time, and double-support time, which are associated with patient functionality. • Kinetic gait variables: vertical, anterior–posterior, and mediolateral ground reaction forces, vertical free moment, joint forces, and moments at the ankle, knee, and hip. Multifactorial and multivariate analyses of variance were performed. Results: Functionality relates to treatment decisions, with patients in the conservative group walking 25% faster and spending 24% less time in the double-support phase. However, these differences vary with age and are reduced in older subjects. Patients who planned to undergo TKR did not present higher knee forces, and different joint moments between clinical treatments depended on the age and BMI of the subjects. Conclusions: Knee osteoarthritis is a multifactorial disease, with age and BMI being confounding factors. The differences in gait between the two groups were mitigated by confounding factors and risk factors, such as being a woman, elderly, and obese, reducing the variability of the gait compression loads. These factors should always be considered in gait studies of patients with knee osteoarthritis to control for confounding effects.

On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness

Abstract: We study the impact of different loss functions on lesion segmentation from medical images. Although the Cross-Entropy (CE) loss is the most popular option when dealing with natural images, for biomedical image segmentation the soft Dice loss is often preferred due to its ability to handle imbalanced scenarios. On the other hand, the combination of both functions has also been successfully applied in this kind of tasks. A much less studied problem is the generalization ability of all these losses in the presence of Out-of-Distribution (OoD) data. This refers to samples appearing in test time that are drawn from a different distribution than training images. In our case, we train our models on images that always contain lesions, but in test time we also have lesion-free samples. We analyze the impact of the minimization of different loss functions on in-distribution performance, but also its ability to generalize to OoD data, via comprehensive experiments on polyp segmentation from endoscopic images and ulcer segmentation from diabetic feet images. Our findings are surprising: CE-Dice loss combinations that excel in segmenting in-distribution images have a poor performance when dealing with OoD data, which leads us to recommend the adoption of the CE loss for this kind of problems, due to its robustness and ability to generalize to OoD samples. Code associated to our experiments can be found at https://github.com/agaldran/lesion_losses_ood .

Unique cardiac remodeling in young adults born Small for Gestational Age with subsequent central obesity

Abstract: Being born small-for-gestational age (SGA, 10 percent of all births) is associated with increased risk of cardiovascular mortality (1,2) in adulthood together with lower exercise tolerance (3), but mechanistic pathways are unclear. Central obesity is known to worsen cardiovascular outcomes, but it is uncertain how it affects the heart in adults born SGA. We aimed to assess whether central obesity makes young adults born SGA more susceptible to cardiac remodelling and dysfunction. A perinatal cohort study including 80 young adults born SGA (birth weight below 10th centile) and 75 adults with normal birth weight (controls). Current waist-to-hip ratio was used as a surrogate of central obesity. Cardiac structure and function were assessed by cardiac magnetic resonance. Statistical shape analysis was used to study the regional geometric variability of the biventricular surfaces produced by central obesity and SGA, and synthetic surfaces representative of obese and non obese were generated for both SGA and controls. Figure 1 shows the superimposed representative surfaces of obese and non-obese according to our model, for controls (right column) and SGA (left column). Both SGA and waist-to-hip were highly associated to cardiac shape (F=3.94 p<0.001; F=5.18 p<0.001 respectively) with a statistically significant interaction (F=2.29, p=0.02), indicating a different cardiac remodelling due to obesity in SGA. While controls tend to increase left ventricular end-diastolic volumes, mass and stroke volume with increasing waist-to-hip ratio, young adults born SGA showed unique response with inability to increase cardiac dimensions or mass resulting in reduced stroke volume (both in absolute values and indexed by body surface area) and increased heart rate. SGA young adults show unique cardiac adaptation to central obesity, which is associated with a decrease in stroke volume. Preventive strategies aiming to reduce cardiometabolic risk in SGA population may be warranted. Type of funding sources: Public grant(s) – EU funding. Main funding source(s): European Union Horizon 2020 Programme