Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical image fusion. We characterize the medical image fusion research based on (1) the widely used image fusion methods, (2) imaging modalities, and (3) imaging of organs that are under study. This review concludes that even though there exists several open ended technological and scientific challenges, the fusion of medical images has proved to be useful for advancing the clinical reliability of using medical imaging for medical diagnostics and analysis, and is a scientific discipline that has the potential to significantly grow in the coming years.

Medical Image Fusion: A survey of the state of the art

Medical image fusion: A survey of the state of the art

Image matting for fusion of multi-focus images in dynamic scenes

Eye tracking technology is becoming easier and cheaper to use, resulting in its increasing application to numerous fields of research. The data collected during an eye tracking experiment can be analyzed by statistical methods and/or with visualization techniques. Visualizations can reveal characteristics of fixations, saccades, and scanpath structures. In this survey, we present an overview of visualization techniques for eye tracking data and describe their functionality. We classify the visualization techniques using nine categories. The categories are based on properties of eye tracking data, including aspects of the stimuli and the viewer, and on properties of the visualization techniques. The classification of about 90 publications including technical as well as application papers with modifications of common visualization techniques are described in more detail. We finally present possible directions for further research in the field of eye tracking data visualization.

State-of-the-Art of Visualization for Eye Tracking Data

A survey of infrared and visual image fusion methods

The prevalence of image fusion---the combination of images of different modalities, such as visible and infrared radiation---has increased the demand for accurate methods of image-quality assessment. The current study used a signal-detection paradigm, identifying the presence or absence of a target in briefly presented images followed by an energy mask, which was compared with computational metric and subjective quality assessment results. In Study 1, 18 participants were presented with fused infrared-visible light images, with a soldier either present or not. Two independent variables, image-fusion method (averaging, contrast pyramid, dual-tree complex wavelet transform) and JPEG compression (no compression, low and high compression), were used in a repeated-measures design. Participants were presented with images and asked to state whether or not they detected the target. In addition, subjective ratings and metric results were obtained. This process was repeated in Study 2, using JPEG2000 compression. The results showed a significant effect for fusion but not compression in JPEG2000 images, while JPEG images showed significant effects for both fusion and compression. Subjective ratings differed, especially for JPEG2000 images, while metric results for both JPEG and JPEG2000 showed similar trends. These results indicate that objective and subjective ratings can differ significantly, and subjective ratings should, therefore, be used with care.

Methods for the assessment of fused images

Image fusion is the process of combining images of differing modalities, such as visible and infrared (IR) images. Significant work has recently been carried out comparing methods of fused image assessment, with findings strongly suggesting that a task-centred approach would be beneficial to the assessment process. The current paper reports a pilot study analysing eye movements of participants involved in four tasks. The first and second tasks involved tracking a human figure wearing camouflage clothing walking through thick undergrowth at light and dark luminance levels, whilst the third and fourth task required tracking an individual in a crowd, again at two luminance levels. Participants were shown the original visible and IR images individually, pixel-averaged, contrast pyramid, and dual-tree complex wavelet fused video sequences. They viewed each display and sequence three times to compare inter-subject scanpath variability. This paper describes the initial analysis of the eye-tracking data gathered from the pilot study. These were also compared with computational metric assessment of the image sequences.

/pdf/scanpath-analysis-of-fused-multi-sensor-images-with-2sjxw7qg1g.pdf

Scanpath Analysis of Fused Multi-Sensor Images with Luminance Change: A Pilot Study

This paper investigates how the object tracking performance is affected by the fusion quality of videos from visible (VIZ) and infrared (IR) surveillance cameras, as compared to tracking in single modality videos. The videos have been fused using the simple averaging, and various multiresolution techniques. Tracking has been accomplished by means of a particle filter using colour and edge cues. The highest tracking accuracy has been obtained in IR sequences, whereas the VIZ video was affected by many artifacts and showed the worst tracking performance. Among the fused videos, the complex wavelet and the averaging techniques, offered the best tracking performance, comparable to that of IR. Thus, of all the methods investigated, the fused videos, containing complementary contextual information from both single modality input videos, are the best source for further analysis by a human observer or a computer program.

/pdf/the-influence-of-multi-sensor-video-fusion-on-object-21d6rztcvn.pdf

The influence of multi-sensor video fusion on object tracking using a particle filter

Accurate quality assessment of fused images, such as combined visible and infrared radiation images, has become increasingly important with the rise in the use of image fusion systems. We bring together three approaches, applying two objective tasks (local target analysis and global target location) to two scenarios, together with subjective quality ratings and three computational metrics. Contrast pyramid, shift-invariant discrete wavelet transform, and dual-tree complex wavelet transform fusion are applied, as well as levels of JPEG2000 compression. The differing tasks are shown to be more or less appropriate for differentiating among fusion methods, and future directions pertaining to the creation of task-specific metrics are explored.

Eduardo Fernandez Canga

Papers

Methods for the assessment of fused images

Scanpath Analysis of Fused Multi-Sensor Images with Luminance Change: A Pilot Study

The influence of multi-sensor video fusion on object tracking using a particle filter

Selection of image fusion quality measures: objective, subjective, and metric assessment.

Task-based scanpath assessment of multi-sensor video fusion in complex scenarios