Showing papers by "Mert R. Sabuncu published in 2021"

Heritability of individualized cortical network topography.

Abstract: Human cortex is patterned by a complex and interdigitated web of large-scale functional networks. Recent methodological breakthroughs reveal variation in the size, shape, and spatial topography of cortical networks across individuals. While spatial network organization emerges across development, is stable over time, and is predictive of behavior, it is not yet clear to what extent genetic factors underlie interindividual differences in network topography. Here, leveraging a nonlinear multidimensional estimation of heritability, we provide evidence that individual variability in the size and topographic organization of cortical networks are under genetic control. Using twin and family data from the Human Connectome Project (n = 1,023), we find increased variability and reduced heritability in the size of heteromodal association networks (h 2 : M = 0.34, SD = 0.070), relative to unimodal sensory/motor cortex (h 2 : M = 0.40, SD = 0.097). We then demonstrate that the spatial layout of cortical networks is influenced by genetics, using our multidimensional estimation of heritability (h 2 -multi; M = 0.14, SD = 0.015). However, topographic heritability did not differ between heteromodal and unimodal networks. Genetic factors had a regionally variable influence on brain organization, such that the heritability of network topography was greatest in prefrontal, precuneus, and posterior parietal cortex. Taken together, these data are consistent with relaxed genetic control of association cortices relative to primary sensory/motor regions and have implications for understanding population-level variability in brain functioning, guiding both individualized prediction and the interpretation of analyses that integrate genetics and neuroimaging.

Heritability and interindividual variability of regional structure-function coupling.

Abstract: White matter structural connections are likely to support flow of functional activation or functional connectivity. While the relationship between structural and functional connectivity profiles, here called SC-FC coupling, has been studied on a whole-brain, global level, few studies have investigated this relationship at a regional scale. Here we quantify regional SC-FC coupling in healthy young adults using diffusion-weighted MRI and resting-state functional MRI data from the Human Connectome Project and study how SC-FC coupling may be heritable and varies between individuals. We show that regional SC-FC coupling strength varies widely across brain regions, but was strongest in highly structurally connected visual and subcortical areas. We also show interindividual regional differences based on age, sex and composite cognitive scores, and that SC-FC coupling was highly heritable within certain networks. These results suggest regional structure-function coupling is an idiosyncratic feature of brain organisation that may be influenced by genetic factors. The relationship between the human structural and functional connectome is still not well established. Here the authors show the interindividual variability that exists in regional coupling of structural and functional connectivity across the brain, and that this is heritable.

Cortical response to naturalistic stimuli is largely predictable with deep neural networks

Abstract: Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation

Abstract: Calibrated estimates of uncertainty are critical for many real-world computer vision applications of deep learning. While there are several widely-used uncertainty estimation methods, dropout inference [11] stands out for its simplicity and efficacy. This technique, however, requires multiple forward passes through the network during inference and therefore can be too resource-intensive to be deployed in real-time applications. We propose a simple, easy-to-optimize distillation method for learning the conditional predictive distribution of a pre-trained dropout model for fast, sample-free uncertainty estimation in computer vision tasks. We empirically test the effectiveness of the proposed method on both semantic segmentation and depth estimation tasks, and demonstrate our method can significantly reduce the inference time, enabling real-time uncertainty quantification, while achieving improved quality of both the uncertainty estimates and predictive performance over the regular dropout model.

Magnetic Resonance Imaging Radiomics-Based Machine Learning Prediction of Clinically Significant Prostate Cancer in Equivocal PI-RADS 3 Lesions

Abstract: Background While Prostate Imaging Reporting and Data System (PI-RADS) 4 and 5 lesions typically warrant prostate biopsy and PI-RADS 1 and 2 lesions may be safely observed, PI-RADS 3 lesions are equivocal. Purpose To construct and cross-validate a machine learning model based on radiomics features from T2 -weighted imaging (T2 WI) of PI-RADS 3 lesions to identify clinically significant prostate cancer (csPCa), that is, pathological Grade Group ≥ 2. Study type Single-center retrospective study. Population A total of 240 patients were included (training cohort, n = 188, age range 43-82 years; test cohort, n = 52, age range 41-79 years). Eligibility criteria were 1) magnetic resonance imaging (MRI)-targeted biopsy between 2015 and 2020; 2) PI-RADS 3 index lesion identified on multiparametric MRI; (3) biopsy performed within 1 year of MRI. The percentages of csPCa lesions were 10.6% and 15.4% in the training and test cohorts, respectively. Field strength/sequence A 3 T; T2 WI turbo-spin echo, diffusion-weighted spin-echo echo planar imaging, dynamic contrast-enhanced MRI with time-resolved T1-weighted imaging. Assessment Multislice volumes-of-interest (VOIs) were drawn in the PI-RADS 3 index lesions on T2 WI. A total of 107 radiomics features (first-order histogram and second-order texture) were extracted from the segmented lesions. Statistical tests A random forest classifier using the radiomics features as input was trained and validated for prediction of csPCa. The performance of the machine learning classifier, prostate specific antigen (PSA) density, and prostate volume for csPCa prediction was evaluated using receiver operating characteristic (ROC) analysis. Results The trained random forest classifier constructed from the T2 WI radiomics features good and statistically significant area-under-the-curves (AUCs) of 0.76 (P = 0.022) for prediction of csPCa in the test set. Prostate volume and PSA density showed moderate and nonsignificant performance (AUC 0.62, P = 0.275 and 0.61, P = 0.348, respectively) for csPCa prediction in the test set. Conclusion The machine learning classifier based on T2 WI radiomic features demonstrated good performance for prediction of csPCa in PI-RADS 3 lesions. Evidence level 4 TECHNICAL EFFICACY: 2.

NeuroGen: activation optimized image synthesis for discovery neuroscience.

Abstract: Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network's ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.

Ensembling Low Precision Models for Binary Biomedical Image Segmentation

Abstract: Segmentation of anatomical regions of interest such as vessels or small lesions in medical images is still a difficult problem that is often tackled with manual input by an ex-pert. One of the major challenges for this task is that the appearance of foreground (positive) regions can be similar to background (negative) regions. As a result, many automatic segmentation algorithms tend to exhibit asymmetric errors, typically producing more false positives than false negatives. In this paper, we aim to leverage this asymmetry and train a diverse ensemble of models with very high recall, while sacrificing their precision. Our core idea is straightforward: A diverse ensemble of low precision and high recall models are likely to make different false positive errors (classifying background as foreground in different parts of the image), but the true positives will tend to be consistent. Thus, in aggregate the false positive errors will cancel out, yielding high performance for the ensemble. Our strategy is general and can be applied with any segmentation model. In three different applications (carotid artery segmentation in a neck CT angiography, myocardium segmentation in a cardiovascular MRI and multiple sclerosis lesion segmentation in a brain MRI), we show how the proposed approach can significantly boost the performance of a baseline segmentation method.

HyperRecon: Regularization-Agnostic CS-MRI Reconstruction with Hypernetworks

Abstract: Reconstructing under-sampled k-space measurements in Compressed Sensing MRI (CS-MRI) is classically solved by minimizing a regularized least-squares cost function. In the absence of fully-sampled training data, this optimization approach can still be amortized via a neural network that minimizes the cost function over a dataset of under-sampled measurements. Here, a crucial design choice is the regularization function(s) and corresponding weight(s). In this paper, we introduce HyperRecon – a novel strategy of using a hypernetwork to generate the parameters of a main reconstruction network as a function of the regularization weight(s), resulting in a regularization-agnostic reconstruction model. At test time, for a given under-sampled image, our model can rapidly compute reconstructions with different amounts of regularization. We propose and empirically demonstrate an efficient and data-driven way of maximizing reconstruction performance given limited hypernetwork capacity. Our code will be made publicly available upon acceptance.

Predicting Individual Task Contrasts From Resting-state Functional Connectivity using a Surface-based Convolutional Network

Abstract: Task-based and resting-state represent the two most common experimental paradigms of functional neuroimaging. While resting-state offers a flexible and scalable approach for characterizing brain function, task-based techniques provide superior localization. In this paper, we build on recent deep learning methods to create a model that predicts task-based contrast maps from resting-state fMRI scans. Specifically, we propose BrainSurfCNN, a surface-based fully-convolutional neural network model that works with a representation of the brain’s cortical sheet. Our model achieves state of the art predictive accuracy on independent test data from the Human Connectome Project and yields individual-level predicted maps that are on par with the target-repeat reliability of the measured contrast maps. We also demonstrate that BrainSurfCNN can generalize remarkably well to novel domains with limited training data.

Deep learning-driven catheter tracking from bi-plane X-ray fluoroscopy of 3D printed heart phantoms

Abstract: Minimally invasive surgery (MIS) has changed not only the performance of specific operations but also the more effective strategic approach to all surgeries. Expansion of MIS to more complex surgeries demands further development of new technologies, including robotic surgical systems, navigation, guidance, visualizations, dexterity enhancement, and 3D printing technology. In the cardiovascular domain, 3D printed modeling can play a crucial role in providing improved visualization of the anatomical details and guide precision operations as well as functional evaluation of various congenital and congestive heart conditions. In this work, we propose a novel deep learning-driven tracking method for providing quantitative 3D tracking of mock cardiac interventions on customdesigned 3D printed heart phantoms. In this study, the position of the tip of a catheter is tracked from bi-plane fluoroscopic images. The continuous positioning of the catheter relative to the 3D printed model was co-registered in a single coordinate system using external fiducial markers embedded into the model. Our proposed method has the potential to provide quantitative analysis for training exercises of percutaneous procedures guided by bi-plane fluoroscopy. Page 2 of Torabinia et al. Mini-invasive Surg 2021;5:32 https://dx.doi.org/10.20517/2574-1225.2021.63 12

Hyper-Convolution Networks for Biomedical Image Segmentation.

Abstract: The convolution operation is a central building block of neural network architectures widely used in computer vision. The size of the convolution kernels determines both the expressiveness of convolutional neural networks (CNN), as well as the number of learnable parameters. Increasing the network capacity to capture rich pixel relationships requires increasing the number of learnable parameters, often leading to overfitting and/or lack of robustness. In this paper, we propose a powerful novel building block, the hyper-convolution, which implicitly represents the convolution kernel as a function of kernel coordinates. Hyper-convolutions enable decoupling the kernel size, and hence its receptive field, from the number of learnable parameters. In our experiments, focused on challenging biomedical image segmentation tasks, we demonstrate that replacing regular convolutions with hyper-convolutions leads to more efficient architectures that achieve improved accuracy. Our analysis also shows that learned hyper-convolutions are naturally regularized, which can offer better generalization performance. We believe that hyper-convolutions can be a powerful building block in future neural network architectures solving computer vision tasks.

Temporal Feature Fusion with Sampling Pattern Optimization for Multi-echo Gradient Echo Acquisition and Image Reconstruction

Abstract: Quantitative imaging in MRI usually involves acquisition and reconstruction of a series of images at multi-echo time points, which possibly requires more scan time and specific reconstruction technique compared to conventional qualitative imaging. In this work, we focus on optimizing the acquisition and reconstruction process of multi-echo gradient echo pulse sequence for quantitative susceptibility mapping as one important quantitative imaging method in MRI. A multi-echo sampling pattern optimization block extended from LOUPE-ST is proposed to optimize the k-space sampling patterns along echoes. Besides, a recurrent temporal feature fusion block is proposed and inserted into a backbone deep ADMM network to capture the signal evolution along echo time during reconstruction. Experiments show that both blocks help improve multi-echo image reconstruction performance.

Regularization-Agnostic Compressed Sensing MRI Reconstruction with Hypernetworks.

Abstract: Reconstructing under-sampled k-space measurements in Compressed Sensing MRI (CS-MRI) is classically solved with regularized least-squares. Recently, deep learning has been used to amortize this optimization by training reconstruction networks on a dataset of under-sampled measurements. Here, a crucial design choice is the regularization function(s) and corresponding weight(s). In this paper, we explore a novel strategy of using a hypernetwork to generate the parameters of a separate reconstruction network as a function of the regularization weight(s), resulting in a regularization-agnostic reconstruction model. At test time, for a given under-sampled image, our model can rapidly compute reconstructions with different amounts of regularization. We analyze the variability of these reconstructions, especially in situations when the overall quality is similar. Finally, we propose and empirically demonstrate an efficient and data-driven way of maximizing reconstruction performance given limited hypernetwork capacity. Our code is publicly available at this https URL.

Probabilistic dipole inversion for adaptive quantitative susceptibility mapping

Abstract: A learning-based posterior distribution estimation method, Probabilistic Dipole Inversion (PDI), is proposed to solve the quantitative susceptibility mapping (QSM) inverse problem in MRI with uncertainty estimation. In PDI, a deep convolutional neural network (CNN) is used to represent the multivariate Gaussian distribution as the approximate posterior distribution of susceptibility given the input measured field. Such CNN is first trained on healthy subjects via posterior density estimation, where the training dataset contains samples from the true posterior distribution. Domain adaptations are then deployed on patient datasets with new pathologies not included in pre-training, where PDI updates the pre-trained CNN's weights in an unsupervised fashion by minimizing the Kullback-Leibler divergence between the approximate posterior distribution represented by CNN and the true posterior distribution from the likelihood distribution of a known physical model and pre-defined prior distribution. Based on our experiments, PDI provides additional uncertainty estimation compared to the conventional MAP approach, while addressing the potential issue of the pre-trained CNN when test data deviates from training. Our code is available at https://github.com/Jinwei1209/Bayesian_QSM.

Text2Brain: Synthesis of Brain Activation Maps from Free-form Text Query

Abstract: Most neuroimaging experiments are under-powered, limited by the number of subjects and cognitive processes that an individual study can investigate. Nonetheless, over decades of research, neuroscience has accumulated an extensive wealth of results. It remains a challenge to digest this growing knowledge base and obtain new insights since existing meta-analytic tools are limited to keyword queries. In this work, we propose Text2Brain, a neural network approach for coordinate-based meta-analysis of neuroimaging studies to synthesize brain activation maps from open-ended text queries. Combining a transformer-based text encoder and a 3D image generator, Text2Brain was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published neuroimaging studies. We demonstrate that Text2Brain can synthesize anatomically-plausible neural activation patterns from free-form textual descriptions of cognitive concepts. Text2Brain is available at https://braininterpreter.com as a web-based tool for retrieving established priors and generating new hypotheses for neuroscience research.

Temporal Feature Fusion with Sampling Pattern Optimization for Multi-echo Gradient Echo Acquisition and Image Reconstruction

Abstract: Quantitative imaging in MRI usually involves acquisition and reconstruction of a series of images at multi-echo time points, which possibly requires more scan time and specific reconstruction technique compared to conventional qualitative imaging. In this work, we focus on optimizing the acquisition and reconstruction process of multi-echo gradient echo pulse sequence for quantitative susceptibility mapping as one important quantitative imaging method in MRI. A multi-echo sampling pattern optimization block extended from LOUPE-ST is proposed to optimize the k-space sampling patterns along echoes. Besides, a recurrent temporal feature fusion block is proposed and inserted into a backbone deep ADMM network to capture the signal evolution along echo time during reconstruction. Experiments show that both blocks help improve multi-echo image reconstruction performance.

Ex uno plures: Splitting One Model into an Ensemble of Subnetworks

Abstract: Monte Carlo (MC) dropout is a simple and efficient ensembling method that can improve the accuracy and confidence calibration of high-capacity deep neural network models. However, MC dropout is not as effective as more compute-intensive methods such as deep ensembles. This performance gap can be attributed to the relatively poor quality of individual models in the MC dropout ensemble and their lack of diversity. These issues can in turn be traced back to the coupled training and substantial parameter sharing of the dropout models. Motivated by this perspective, we propose a strategy to compute an ensemble of subnetworks, each corresponding to a non-overlapping dropout mask computed via a pruning strategy and trained independently. We show that the proposed subnetwork ensembling method can perform as well as standard deep ensembles in both accuracy and uncertainty estimates, yet with a computational efficiency similar to MC dropout. Lastly, using several computer vision datasets like CIFAR10/100, CUB200, and Tiny-Imagenet, we experimentally demonstrate that subnetwork ensembling also consistently outperforms recently proposed approaches that efficiently ensemble neural networks.

Text2Brain: Synthesis of Brain Activation Maps from Free-form Text Query

Abstract: Most neuroimaging experiments are under-powered, limited by the number of subjects and cognitive processes that an individual study can investigate. Nonetheless, over decades of research, neuroscience has accumulated an extensive wealth of results. It remains a challenge to digest this growing knowledge base and obtain new insights since existing meta-analytic tools are limited to keyword queries. In this work, we propose Text2Brain, a neural network approach for coordinate-based meta-analysis of neuroimaging studies to synthesize brain activation maps from open-ended text queries. Combining a transformer-based text encoder and a 3D image generator, Text2Brain was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published neuroimaging studies. We demonstrate that Text2Brain can synthesize anatomically-plausible neural activation patterns from free-form textual descriptions of cognitive concepts. Text2Brain is available at https://braininterpreter.com as a web-based tool for retrieving established priors and generating new hypotheses for neuroscience research.

Joint Optimization of Hadamard Sensing and Reconstruction in Compressed Sensing Fluorescence Microscopy.

Abstract: Compressed sensing fluorescence microscopy (CS-FM) proposes a scheme whereby less measurements are collected during sensing and reconstruction is performed to recover the image. Much work has gone into optimizing the sensing and reconstruction portions separately. We propose a method of jointly optimizing both sensing and reconstruction end-to-end under a total measurement constraint, enabling learning of the optimal sensing scheme concurrently with the parameters of a neural network-based reconstruction network. We train our model on a rich dataset of confocal, two-photon, and wide-field microscopy images comprising of a variety of biological samples. We show that our method outperforms several baseline sensing schemes and a regularized regression reconstruction algorithm.