Image fusion is the process in which substantial information taken through different sensors, different exposure values and at different focus points is integrated together to generate a composite image. In various applications, different types of data sets are captured with the help of different sensors like infrared (IR) region and visible region, Computed Tomography (CT) and Positron Emission Tomograph (PET) scan, multifocus images with different focal points and images taken by a static camera at different exposure values. A most promising area of image processing nowadays is image fusion. The picture fusion method seeks to incorporate two or more pictures into one picture that contains better data than each source picture without adding any artifacts. It plays an essential role in distinct applications like biomedical diagnostics, photography, object identification, surveillance, defense, and remote sensing satellite imaging. Three elements are taken into consideration in this review article that includes spatial domain fusion methodology, different transformation domain techniques, and image fusion performance evaluation metrics.

Review of Various Image Fusion Algorithms and Image Fusion Performance Metric

Multi-focus image fusion (MFIF) has attracted considerable interests due to its numerous applications. While much progress has been made in recent years with efforts on developing various MFIF algorithms, some issues significantly hinder the fair and comprehensive performance comparison of MFIF methods, such as the lack of large-scale test set and the random choices of objective evaluation metrics in the literature. To solve these issues, this paper presents a multi-focus image fusion benchmark (MFIFB) which consists a test set of 105 image pairs, a code library of 30 MFIF algorithms, and 20 evaluation metrics. MFIFB is the first benchmark in the field of MFIF and provides the community a platform to compare MFIF algorithms fairly and comprehensively. Extensive experiments have been conducted using the proposed MFIFB to understand the performance of these algorithms. By analyzing the experimental results, effective MFIF algorithms are identified. More importantly, some observations on the status of the MFIF field are given, which can help to understand this field better.

Multi-focus Image Fusion: A Benchmark.

Image fusion is a process that integrates similar types of images collected from heterogeneous sources into one image in which the information is more definite and certain. Hence, the resultant image is anticipated as more explanatory and enlightening both for human and machine perception. Different image combination methods have been presented to consolidate significant data from a collection of images into one image. As a result of its applications and advantages in variety of fields such as remote sensing, surveillance, and medical imaging, it is significant to comprehend image fusion algorithms and have a comparative study on them. This paper presents a review of the present state-of-the-art and well-known image fusion techniques. The performance of each algorithm is assessed qualitatively and quantitatively on two benchmark multi-focus image datasets. We also produce a multi-focus image fusion dataset by collecting the widely used test images in different studies. The quantitative evaluation of fusion results is performed using a set of image fusion quality assessment metrics. The performance is also evaluated using different statistical measures. Another contribution of this paper is the proposal of a multi-focus image fusion library, to the best of our knowledge, no such library exists so far. The library provides implementation of numerous state-of-the-art image fusion algorithms and is made available publicly at project website.

https://www.mdpi.com/2313-433X/6/7/60/pdf

Multi-Focus Image Fusion: Algorithms, Evaluation, and a Library.

Multi-focus image fusion is an effective approach to obtain the all-in-focus image. Focus detection is the key issue of multi-focus image fusion. Aiming at the shortcoming of spatial domain and transform domain algorithms for multi-focus image fusion, a novel multi-focus image fusion algorithm is proposed by combing focus detection in spatial domain and non-subsampled contourlet transform (NSCT) domain. At first, the focused pixels are detected by the sum-modified-Laplacian algorithm in spatial domain. At the same time, the focus detection method is proposed in NSCT domain, namely by MPCNN and voting fusion methods for high-frequency subbands of NSCT. Then, the morphological operation is utilized to correct the focus detection results in spatial domain and NSCT domain. At last, synthesis of detection results is implemented and the fused image can be obtained. Experimental results verified that the proposed algorithm outperformed some state-of-the-art fusion algorithms in terms of both subjective observation and objective evaluations.

Multi-focus image fusion algorithm based on focus detection in spatial and NSCT domain.

Machine learning and deep learning are finding applications in various computer vision problems such as object recognition, detection, and visual tracking. In addition, in computer vision, it is quite common to fuse information acquired in different spectral ranges, focusing, and lighting conditions to know more details of a particular scene. Hence, image fusion using machine learning especially deep learning would be a hot research topic in upcoming years due to recent advancements in both software and computing capabilities. In this chapter, we made an effort to present machine learning- and deep learning-based fusion in a simple manner for fundamental understanding. In Sect. 7.1, a general introduction to AI and its general classification is presented. Section 7.2 gives an overview of machine learning from basic definitions to advanced concepts. Image fusion based on machine learning is explained in Sect. 7.3. In Sect. 7.4, important and useful concepts of deep learning are described. Section 7.5 gives an overview of state-of-the-art deep learning based image fusion. Section 7.6 presents the future scope. Finally, Sect. 7.7 concludes the chapter.

Image Fusion Based on Machine Learning and Deep Learning

Image fusion is the process to combine two or more images such that the enhanced output image contains all the relevant information. This paper aims to evaluate algorithms that fuses multi-focus images. With an aim to enhance the sharpness of the final fused image and also to reduce the computational complexity, a novel method is proposed that uses the Sobel operator. This method is evaluated using mean squared error, peak signal to noise ratio, structural similarity index, entropy, mutual information, image quality index and run time. The optimum weights required to fuse two images are calculated from image statistics using genetic algorithm (GA). The results show that this superposition method performs at par with discrete wavelet transform (DWT) with reduction in run time.

/pdf/spatial-domain-decision-based-image-fusion-using-3ecfnidbzn.pdf

Spatial domain decision based image fusion using superimposition

Multi-focus images may be fused to get the relevant information of a particular scene. Due to the limited depth of field of a convex lens of a camera, some objects in the image may not be focused. These images are fused to get all-in-focus image. This paper proposes an innovative way to fuse multi-focus images. The proposed algorithm calculates weights indicating the sharp regions of input images with the help of Deep Belief Network (DBN) and then fuses input images using weighted superimposition fusion rule. The proposed algorithm is analyzed and examined using various parameters like entropy, mutual information, SSIM, IQI etc.

Multi-focus Image Fusion Using Deep Belief Network

A R T I C L E I N F O A B S T R A C T Article history: Received: 06 December, 2016 Accepted: 13 January, 2017 Online: 28 January, 2017 Optical imaging systems or cameras have a convex lens with limited depth of field. So, multifocus images are obtained when such systems are used to capture an image of a particular scene. These images are fused to get all-in-focus image. This paper proposes a new simpler method of multi-focus image fusion. Instead of decomposing input image into blocks in pyramid style, the proposed algorithm considers the complete image for edge detection. The proposed algorithm uses the genetic algorithm (GA) to find out the optimum weights from extracted edges and then fuses the images with the fusion rule based on optimized weights. Experimental results show that this superimposition method performs well; consumes less computation time and thus proves to be suitable for hardware implementation.

/pdf/optimized-multi-focus-image-fusion-using-genetic-algorithm-36mt4l7vn2.pdf

Optimized Multi-focus Image Fusion Using Genetic Algorithm

Enhanced Nature-Inspired Algorithm-based Hybrid Deep Learning for Character Recognition in Sanskrit Language

When distance of all objects in a scene from the camera lens is not equal, some objects appear clear while others appear blur in a photograph To have focus on each object in entire photograph, fusion proves to be reasonable solution Thus different photographs or pictures or images having focused on different objects are mixed together in fusion In this paper, artificial neural network is used for fusion and all-in-focus image is obtained Fused image is evaluated using objective parameters which show that the algorithm works effectively

Vaidehi Deshmukh

Papers

Spatial domain decision based image fusion using superimposition

Multi-focus Image Fusion Using Deep Belief Network

Optimized Multi-focus Image Fusion Using Genetic Algorithm

Enhanced Nature-Inspired Algorithm-based Hybrid Deep Learning for Character Recognition in Sanskrit Language

Multi-focus Image Fusion: Neural Network Approach