Objective evaluation of images is one of the most essential and practical aspects of image inpainting. The existing objective evaluation methods of image inpainting are functional only on an individual basis and do not provide an accurate and useful objective evaluation of inpainted images. Currently, there is no objective measure for evaluating inpainted images. In this study, an objective evaluation method was developed for image inpainting. In the proposed method, first, 100 images were inpainted using an exemplar-based algorithm. Then, the saliency map and its complementary region in the original image were obtained and a new objective measure was proposed for the evaluation of inpainted images based on the saliency map features. To make the assessment more realistic and comparable to human judgments, two terms, namely penalty and compensation, were taken into account. To assess the performance of our proposed objective measure, the inpainted images were also evaluated using a subjective test. The experiments demonstrates that the proposed objective measure correlated with the qualitative opinion in a human observer study. Finally, the objective measure was compared against three other measures, and the results showed that our proposed objective measure performed better than the other evaluation measures.

An objective method to evaluate exemplar‐based inpainted images quality using Jaccard index

In this paper we are addressing two main topics, as follows. First, a rigorous qualitative study is elaborated for a second-order parabolic problem, equipped with nonlinear anisotropic diffusion and cubic nonlinear reaction, as well as non-homogeneous Cauchy-Neumann boundary conditions. Under certain assumptions on the input data: f(t,x), w(t,x) and v0(x), we prove the well-posedness (the existence, a priori estimates, regularity, uniqueness) of a solution in the Sobolev space Wp1,2(Q), facilitating for the present model to be a more complete description of certain classes of physical phenomena. The second topic refers to the construction of two numerical schemes in order to approximate the solution of a particular mathematical model (local and nonlocal case). To illustrate the effectiveness of the new mathematical model, we present some numerical experiments by applying the model to image segmentation tasks.

https://www.mdpi.com/2227-7390/9/1/91/pdf

Rigorous Mathematical Investigation of a Nonlocal and Nonlinear Second-Order Anisotropic Reaction-Diffusion Model: Applications on Image Segmentation

In traditional inpainting method for object removal, the SSD (Sum of Squared Differences) is always used to measure the degree of similarity between exemplar patch and target patch. Although the matching rule is simple, there is a risk that the target patch is replaced by an unsuitable exemplar patch, which leads to the mismatch error. Even worse, the error may be constantly accumulated along with the process progresses, finally some unexpected objects will be introduced into target region, and the restored image cannot meet the requirements of human vision. In view of these problems, we propose an inpainting method based on adaptive two-round search strategy. Firstly, we define the DBP (Differences Between Patches) between target patch and exemplar patch, and use it to measure the degree of difference between the two patches. Then, based on SSD and DBP, we adaptively judge whether there is a mismatch error. If the mismatch error occurs, the two-round search strategy is implemented. We define a new matching rule and use it to re-search the exemplar patch. Finally, we use the exemplar patch to restore the target patch. Experimental results demonstrate the effectiveness of our method. It can effectively prevent the occurrence of mismatch error and error accumulation, improve the restoration effect.

/pdf/image-inpainting-for-object-removal-based-on-adaptive-two-6khj9r7khc.pdf

Image Inpainting for Object Removal Based on Adaptive Two-Round Search Strategy

In the inpainting method for object removal, SSD (Sum of Squared Differences) is commonly used to measure the degree of similarity between the exemplar patch and the target patch, which has a very important impact on the restoration results. Although the matching rule is relatively simple, it is likely to lead to the occurrence of mismatch error. Even worse, the error may be accumulated along with the process continues. Finally some unexpected objects may be introduced into the target region, making the result unable to meet the requirements of visual consistency. In view of these problems, we propose an inpainting method for object removal based on difference degree constraint. Firstly, we define the MSD (Mean of Squared Differences) and use it to measure the degree of differences between corresponding pixels at known positions in the target patch and the exemplar patch. Secondly, we define the SMD (Square of Mean Differences) and use it to measure the degree of differences between the pixels at known positions in the target patch and the pixels at unknown positions in the exemplar patch. Thirdly, based on MSD and SMD, we define a new matching rule and use it to find the most similar exemplar patch in the source region. Finally, we use the exemplar patch to restore the target patch. Experimental results show that the proposed method can effectively prevent the occurrence of mismatch error and improve the restoration effect.

/pdf/an-image-inpainting-method-for-object-removal-based-on-bv6dwy9x3f.pdf

An image inpainting method for object removal based on difference degree constraint

Images taken and stored digitally are often degraded by noise, so that the perceived image quality is significantly decreased in the presence of noise and human gaze behavior is affected by this noise as well. In recent years, image denoising has been modeled as partial differential equations (PDEs), which can be said to eliminate noise as well as preserve image detail is one of their main advantages. Numerical simulation of these PDEs, which are often nonlinear, high-order and high-dimensional, is one of the most challenging topics in this field. In this paper, one of the most powerful meshless numerical approaches, the operator splitting radial basis function (RBF) collocation method, is introduced to solve two modified versions of the Perona-Malik (PM) equation. The method splits each problem into several equations each of which is solved in one direction so as to make the major problem solvable in a smaller size. Furthermore, being in one direction makes it possible to solve the equations without any need for domain decomposition and its complexities. An unconditionally stable method (Crank-Nicolson) is used for discretizing time in this paper. Using meshless approaches in the image processing problems makes sense since the structure of digital images consist of pixels that corresponds to the distributed nodes of the meshless method so the complex time consuming domain construction of the finite element methods is avoided to get higher speeds in obtaining the desired denoised results. Some images with different noise levels are denoised using the presented method in the section of numerical results to show the desirable performance of the method. Also, the quality of the denoised images are concluded in tables which can be used to compare the obtained results. 

Efficient image denoising technique using the meshless method: Investigation of operator splitting RBF collocation method for two anisotropic diffusion-based PDEs

In this paper, we will present a variational PDE-based image inpainting model in which we have used the square of the $$L^2$$
 norm of Hessian of the image u as regularization term. The Euler–Lagrange equation will lead us to a fourth-order linear PDE. For time discretization, we have used convexity splitting and the resulting semi-discrete scheme is solved in Fourier domain. Stability analysis for the semi-discrete scheme is carried out. We will demonstrate some numerical results and compare with $$\text {TV}-L^2$$
 and $$\text {TV}-H^{-1}$$
 model.

A linear fourth-order PDE-based gray-scale image inpainting model

A PDE model based on an image specific auto generated multi-well potential function and a 4th order anisotropic diffusion with good performance with respect to loss due to smoothing effects has been proposed for grayscale image inpainting. An unconditionally stable numerical scheme using the notion of convexity splitting in time and Fourier spectral method in space has been derived. The stable scheme is both consistent and convergent. Numerical computations are done for some standard test images and results are compared with the results of other existing models in the literature using image analysis specs such as PSNR, SNR, and SSIM.

An anisotropic PDE model for image inpainting

In this paper, we have proposed a higher order non-linear PDE model containing diffusion terms of T V − L 2 and T V − H − 1 model for effective image denoising. Fourier spectral method for space and convexity splitting method for time have been used to solve the proposed PDE model. Stability analysis for the time discretized equation has been derived. Numerical experiment has been done on some test images and the results have been compared with the results of some existing models.

A TV−L2−H−1 PDE model for effective denoising

Higher order PDE based model for segmenting noisy image

https://doi.org/10.1016/j.cma.2022.115696

Abdul Halim

Papers

A linear fourth-order PDE-based gray-scale image inpainting model

An anisotropic PDE model for image inpainting

A TV−L2−H−1 PDE model for effective denoising

Higher order PDE based model for segmenting noisy image

A reduced order model for the finite element approximation of eigenvalue problems