Showing papers by "Tommaso Mansi published in 2016"

An Artificial Agent for Anatomical Landmark Detection in Medical Images

Abstract: Fast and robust detection of anatomical structures or pathologies represents a fundamental task in medical image analysis. Most of the current solutions are however suboptimal and unconstrained by learning an appearance model and exhaustively scanning the space of parameters to detect a specific anatomical structure. In addition, typical feature computation or estimation of meta-parameters related to the appearance model or the search strategy, is based on local criteria or predefined approximation schemes. We propose a new learning method following a fundamentally different paradigm by simultaneously modeling both the object appearance and the parameter search strategy as a unified behavioral task for an artificial agent. The method combines the advantages of behavior learning achieved through reinforcement learning with effective hierarchical feature extraction achieved through deep learning. We show that given only a sequence of annotated images, the agent can automatically and strategically learn optimal paths that converge to the sought anatomical landmark location as opposed to exhaustively scanning the entire solution space. The method significantly outperforms state-of-the-art machine learning and deep learning approaches both in terms of accuracy and speed on 2D magnetic resonance images, 2D ultrasound and 3D CT images, achieving average detection errors of 1-2 pixels, while also recognizing the absence of an object from the image.

An Artificial Agent for Robust Image Registration

Abstract: 3-D image registration, which involves aligning two or more images, is a critical step in a variety of medical applications from diagnosis to therapy. Image registration is commonly performed by optimizing an image matching metric as a cost function. However this task is challenging due to the non-convex nature of the matching metric over the plausible registration parameter space and insufficient approches for a robust optimization. As a result, current approaches are often customized to a specific problem and sensitive to image quality and artifacts. In this paper, we propose a completely different approach to image registration, inspired by how experts perform the task. We first cast the image registration problem as a "strategic learning" process, where the goal is to find the best sequence of motion actions (e.g. up, down, etc) that yields image alignment. Within this approach, an artificial agent is learned, modeled using deep convolutional neural networks, with 3D raw image data as the input, and the next optimal action as the output. To copy with the dimensionality of the problem, we propose a greedy supervised approach for an end-to-end training, coupled with attention-driven hierarchical strategy. The resulting registration approach inherently encodes both a data-driven matching metric and an optimal registration strategy (policy). We demonstrate on two 3-D/3-D medical image registration examples with drastically different nature of challenges, that the artificial agent outperforms several state-of-the-art registration methods by a large margin in terms of both accuracy and robustness.

Intelligent medical image landmark detection

Abstract: Intelligent image parsing for anatomical landmarks and/or organs detection and/or segmentation is provided. A state space of an artificial agent is specified for discrete portions of a test image. A set of actions is determined, each specifying a possible change in a parametric space with respect to the test image. A reward system is established based on applying each action of the set of actions and based on at least one target state. The artificial agent learns an optimal action-value function approximator specifying the behavior of the artificial agent to maximize a cumulative future reward value of the reward system. The behavior of the artificial agent is a sequence of actions moving the agent towards at least one target state. The learned artificial agent is applied on a test image to automatically parse image content.

Content-based medical image rendering based on machine learning

Abstract: An artificial intelligence agent is machine trained and used to provide physically-based rendering settings. By using deep learning and/or other machine training, settings of multiple rendering parameters may be provided for consistent imaging even in physically-based rendering.

Shaping the Future through Innovations: From Medical Imaging to Precision Medicine

Abstract: Medical images constitute a source of information essential for disease diagnosis, treatment and follow-up. In addition, due to its patient-specific nature, imaging information represents a critical component required for advancing precision medicine into clinical practice. This manuscript describes recently developed technologies for better handling of image information: photorealistic visualization of medical images with Cinematic Rendering, artificial agents for in-depth image understanding, support for minimally invasive procedures, and patient-specific computational models with enhanced predictive power. Throughout the manuscript we will analyze the capabilities of such technologies and extrapolate on their potential impact to advance the quality of medical care, while reducing its cost.

Three-dementional quantitative heart hemodynamics in medical imaging

Abstract: A medical system is provided for three-dimensional hemodynamic quantification. Comprehensive three-dimensional ( 3 D) plus time ( 3 D+t) assessment of flow patterns inside the heart are provided by a combination of lumped-parameter modeling and computational flow dynamic modeling. Using medical scanning, the lumped parameter model is personalized to a given patient. The personalized lumped-parameter model provides pressure curves (i.e., pressure as a function of time) for one or more locations. Using geometry of the patients heart segmented from the medical scanning and the pressure curves as boundary conditions, the computational flow dynamics model calculates the absolute pressure for any location (e.g., for a three-dimensional field of locations) in the patient heart at any one or more phases of the cardiac cycle. More accurate absolute pressure may be provided without invasive measurement.

Planning and guidance of electrophysiology therapies

Abstract: A processor acquires image data from a medical imaging system. The processor generates a first model from the image data. The processor generates a computational model which includes cardiac electrophysiology and cardiac mechanics estimated from the first model. The processor performs tests on the computational model to determine outcomes for therapies. The processor overlays the outcome on an interventional image. Using interventional imaging, the first heart model may be updated/overlaid during the therapy to visualize its effect on a patient's heart.

Challenges to Validate Multi-Physics Model of Liver Tumor Radiofrequency Ablation from Pre-clinical Data

Abstract: The planning and interventional guidance of liver tumor radiofrequency ablation (RFA) is difficult due to the cooling effect of large vessels and the large variability of tissue parameters. Subject-specific modeling of RFA is challenging as it requires the knowledge of model geometry and hemodynamics as well as the simulation of heat transfer and cell death mechanisms.In this paper, we propose to validate such a model from pre-operative multi-modal images and intra-operative signals (temperature and power) measured by the ablation device itself. In particular, the RFA computation becomes subject-specific after three levels of personalization: anatomical, heat transfer, and a novel cellular necrosis model. We propose an end-to-end pre-clinical validation framework that considers the most comprehensive dataset for model validation. This framework can also be used for parameter estimation and we evaluate its predictive power in order to fully assess the possibility to personalize our model in the future. Such a framework would therefore not require any necrosis information, thus better suited for clinical applications. We evaluated our approach on seven ablations from three healthy pigs.The predictive power of the model was tested: a mean point-to-mesh error between predicted and actual ablation extent of 3.5 mm was achieved.

A Self-Taught Artificial Agent for Multi-Physics Computational Model Personalization

Abstract: Personalization is the process of fitting a model to patient data, a critical step towards application of multi-physics computational models in clinical practice. Designing robust personalization algorithms is often a tedious, time-consuming, model- and data-specific process. We propose to use artificial intelligence concepts to learn this task, inspired by how human experts manually perform it. The problem is reformulated in terms of reinforcement learning. In an off-line phase, Vito, our self-taught artificial agent, learns a representative decision process model through exploration of the computational model: it learns how the model behaves under change of parameters. The agent then automatically learns an optimal strategy for on-line personalization. The algorithm is model-independent; applying it to a new model requires only adjusting few hyper-parameters of the agent and defining the observations to match. The full knowledge of the model itself is not required. Vito was tested in a synthetic scenario, showing that it could learn how to optimize cost functions generically. Then Vito was applied to the inverse problem of cardiac electrophysiology and the personalization of a whole-body circulation model. The obtained results suggested that Vito could achieve equivalent, if not better goodness of fit than standard methods, while being more robust (up to 11% higher success rates) and with faster (up to seven times) convergence rate. Our artificial intelligence approach could thus make personalization algorithms generalizable and self-adaptable to any patient and any model.

An Artificial Agent for Robust Image Registration

Abstract: 3-D image registration, which involves aligning two or more images, is a critical step in a variety of medical applications from diagnosis to therapy. Image registration is commonly performed by optimizing an image matching metric as a cost function. However, this task is challenging due to the non-convex nature of the matching metric over the plausible registration parameter space and insufficient approaches for a robust optimization. As a result, current approaches are often customized to a specific problem and sensitive to image quality and artifacts. In this paper, we propose a completely different approach to image registration, inspired by how experts perform the task. We first cast the image registration problem as a "strategy learning" process, where the goal is to find the best sequence of motion actions (e.g. up, down, etc.) that yields image alignment. Within this approach, an artificial agent is learned, modeled using deep convolutional neural networks, with 3D raw image data as the input, and the next optimal action as the output. To cope with the dimensionality of the problem, we propose a greedy supervised approach for an end-to-end training, coupled with attention-driven hierarchical strategy. The resulting registration approach inherently encodes both a data-driven matching metric and an optimal registration strategy (policy). We demonstrate, on two 3-D/3-D medical image registration examples with drastically different nature of challenges, that the artificial agent outperforms several state-of-art registration methods by a large margin in terms of both accuracy and robustness.

Device and method for the computer-assisted simulation of surgical interventions

Abstract: A first interface for reading image data of an anatomical region obtained by means of a medical imaging method is provided. A modeling module serves for establishing a volumetric biomechanical structure model of the anatomical region on the basis of the image data. Moreover, provision is made of a tracking module, couplable with a camera, for video-based registration of spatial gestures of a user. Furthermore, a simulation module, based on the biomechanical structure model, serves to assign a registered gesture to a simulated mechanical effect on the anatomical region, simulate a mechanical reaction of the anatomical region to the simulated mechanical effect, and modify the biomechanical structure model in accordance with the simulated mechanical reaction. Moreover, provision is made for a visualization module for the volumetric visualization of the biomechanical structure model.

Vorrichtung und Verfahren zum rechnergestützten Simulieren von chirurgischen Eingriffen

Abstract: Erfindungsgemas ist eine erste Schnittstelle (I1) zum Einlesen von mittels eines medizinischen Bildgebungsverfahrens gewonnenen Bilddaten (IMG) einer anatomischen Region (AR) vorgesehen Ein Modellierungsmodul (MM) dient zum Ermitteln eines volumetrischen biomechanischen Strukturmodells (BMS) der anatomischen Region (AR) anhand der Bilddaten (IMG) Daruber hinaus ist ein mit einer Kamera (C) koppelbares Trackingmodul (TM) zum videobasierten Erfassen von raumlichen Gesten eines Benutzers vorgesehen Weiterhin dient ein auf dem biomechanischen Strukturmodell (BMS) basierendes Simulationsmodul (SM) dazu, eine erfasste Geste einer simulierten mechanischen Einwirkung auf die anatomische Region (AR) zuzuordnen, eine mechanische Reaktion der anatomischen Region (AR) auf die simulierte mechanische Einwirkung zu simulieren sowie das biomechanische Strukturmodell (BMS) gemas der simulierten mechanischen Reaktion zu modifizieren Daruber hinaus ist ein Visualisierungsmodul (VM) zum volumetrischen Visualisieren des biomechanischen Strukturmodells (BMS) vorgesehen

Aortic and Mitral Valve Modeling From Multi-Modal Image Data

Abstract: The cardiac valvular apparatus is an essential part of the anatomical, functional, and hemodynamic characteristics of the heart and the cardiovascular system. Valvular heart diseases often involve multiple dysfunctions and require joint assessment and therapy. In this chapter, we introduce a modular patient-specific model of the aortic and mitral valvular apparatus. We present a discriminative learning-based framework that permits the estimation of patient-specific model parameters from multi-modal cardiac images. In addition, we introduce a shape representation, the ShapeForest, which is able to model complex shape variation, preserves local shape information, and incorporates prior knowledge during shape space inference. From the patient-specific model a wide range of clinical biomarkers can be derived during functional assessment and intervention planning. Experiments on cardiac computed tomography and transesophageal echocardiogram studies demonstrate the performance and clinical potential of the proposed method. Our method enables automatic quantitative evaluation of the left heart valvular apparatus based on noninvasive imaging techniques.