Exploiting Multi-Direction Features in MRF-Based Image Inpainting Approaches

TLDR: Experimental results show that the proposed Structure Offsets Statistics based image inpainting algorithm is superior to several state-of-the-art approaches in terms of the abilities of maintaining structure coherence and neighborhood consistence and the computational efficiency.

Abstract: Image inpainting technique recovers the missing regions of an image using information from known regions and it has shown success in various application fields. As a popular kind of methods, Markov Random Field (MRF)-based methods are able to produce better results than earlier diffusion-based and sparse-based methods on inpainting images with big holes. However, for images with complex structures, the results are still not quite pleasant and some inpainting trails exist. The direction feature is an important factor for image understanding and human eye visual requirements, and exploiting multi-direction features is of great potential to further improve inpainting performance. Following the idea, this paper proposes a Structure Offsets Statistics based image inpainting algorithm by exploiting multiple direction features under the framework of MRF-based methods. Specifically, when selecting proper labels, multi-direction features are extracted and applied to construct a structure image and a non-structure image, and the candidate labels are chosen from the offsets of structure and non-structure images. Meanwhile, the multi-direction features are applied to construct a new smooth term for the energy equation which is then solved by graph-cut optimization technology. Experimental results show that on inpainting tasks with various complexities, the proposed method is superior to several state-of-the-art approaches in terms of the abilities of maintaining structure coherence and neighborhood consistence and the computational efficiency.