Image inpainting on satellite image using texture synthesis & region filling algorithm
03 Sep 2015-pp 1-5
TL;DR: An algorithm was proposed to synthesize the structure & texture as well as fill the hole that is left behind in an undetectable form of image inpainting, and an attempt has been made to compute actual color values using exemplar based texture synthesis and region filling method.
Abstract: Image inpainting is an interesting new research topic in image processing, which can be used in many thrust areas, such as computer graphics, image editing, film postproduction, image restoration and special effects and the restoration of old Photographs and damaged film, removal of superimposed text like dates, subtitles, or publicity and the removal of entire objects from the image. In image inpainting, missing (target) regions were filled by structural and textural information of an image in a visually plausible way, also known as image restoration. Though this technique is very useful, it is still a challenging problem in computer vision and computer graphics. In this paper, an algorithm is proposed for removing target objects from digital images. An algorithm was proposed to synthesize the structure & texture as well as fill the hole that is left behind in an undetectable form. An attempt has been made to compute actual color values using exemplar based texture synthesis and region filling method. A number of examples on removing occluding objects from real and satellite images demonstrate the effectiveness of proposed algorithm in terms of both inpainting quality and computational efficacy.
Citations
More filters
TL;DR: Results prove the efficiency of the proposed generalized scanline fill algorithm and its advantage over the state-of-the-art algorithms, and clearly show that optimized machine routine is capable of performing the real-time tasks.
Abstract: A generalized iterative scanline fill algorithm intended for use in real-time applications and its highly optimized machine code implementation are presented in this paper. The algorithm uses the linear image representation in order to achieve the fast memory access to the pixel intensity values. The usage of the linear image representation is crucial for achieving the highly optimized low-level machine code implementation. A few generalization features are also proposed, and discussion about the possible real-time applications is given. The proposed efficient machine code implementation is tested on several PC machines, and a set of numerical results is provided. The machine routine is compared with standard and optimized implementations of the 4-way flood fill algorithm and scanline fill algorithm. The machine code implementation performs approximately 2 times faster than the optimized scanline fill algorithm implementation and 6 times faster than standard iterative scanline fill algorithm implementation on two-dimensional image data structure. Furthermore, the machine routine proved to perform even more than 15 times faster than the optimized flood fill algorithm implementations. Provided results prove the efficiency of the proposed generalized scanline fill algorithm and its advantage over the state-of-the-art algorithms, and clearly show that optimized machine routine is capable of performing the real-time tasks.
6 citations
Cites background from "Image inpainting on satellite image..."
...Beside the aforementioned algorithms, diverse approaches were proposed for the hole filling and have already found its application, especially in medicine and processing of the satellite images [33]....
[...]
Proceedings Article•
01 Jul 2020TL;DR: The applicability of the setting and processing pipeline on affective state prediction based on front camera recordings during math-solving tasks and emotional stimuli from pictures shown on a tablet are demonstrated.
Abstract: Front camera data from tablets used in educational settings offer valuable clues to student behavior, attention, and affective state. Due to the camera’s angle of view, the face of the student is partially occluded and skewed. This hinders the ability of experts to adequately capture the learning process and student states. In this paper, we present a pipeline and techniques for image reconstruction of front camera recordings. Our setting consists of a cheap and unobtrusive mirror construction to improve the visibility of the face. We then process the image and use neural inpainting to reconstruct missing data in the recordings. We demonstrate the applicability of our setting and processing pipeline on affective state prediction based on front camera recordings (i.e., action units, eye gaze, eye blinks, and movement) during math-solving tasks (active) and emotional stimuli from pictures (passive) shown on a tablet. We show that our setup provides comparable performance for affective state prediction to recordings taken with an external and more obtrusive GoPro camera.
2 citations
Cites background from "Image inpainting on satellite image..."
..., object removal) [50], and image coding and transmission (e....
[...]
TL;DR: The experimental results show that the proposed methods are capable of reconstructing missing regions with good visual quality.
Abstract: The Consultative Committee for Space Data Systems proposed an efficient image compression standard that can do lossless compression (CCSDS-ICS). CCSDS-ICS is the most widely utilized standard for satellite communications. However, the original CCSDS-ICS is weak in terms of error resilience with even a single incorrect bit possibly causing numerous missing pixels. A restoration algorithm based on the neighborhood similar pixel interpolator is proposed to fill in missing pixels. The linear regression model is used to generate the reference image from other panchromatic or multispectral images. Furthermore, an adaptive search window is utilized to sieve out similar pixels from the pixels in the search region defined in the neighborhood similar pixel interpolator. The experimental results show that the proposed methods are capable of reconstructing missing regions with good visual quality.
2 citations
References
More filters
[...]
TL;DR: A novel algorithm for digital inpainting of still images that attempts to replicate the basic techniques used by professional restorators, and does not require the user to specify where the novel information comes from.
Abstract: Inpainting, the technique of modifying an image in an undetectable form, is as ancient as art itself. The goals and applications of inpainting are numerous, from the restoration of damaged paintings and photographs to the removal/replacement of selected objects. In this paper, we introduce a novel algorithm for digital inpainting of still images that attempts to replicate the basic techniques used by professional restorators. After the user selects the regions to be restored, the algorithm automatically fills-in these regions with information surrounding them. The fill-in is done in such a way that isophote lines arriving at the regions' boundaries are completed inside. In contrast with previous approaches, the technique here introduced does not require the user to specify where the novel information comes from. This is automatically done (and in a fast way), thereby allowing to simultaneously fill-in numerous regions containing completely different structures and surrounding backgrounds. In addition, no limitations are imposed on the topology of the region to be inpainted. Applications of this technique include the restoration of old photographs and damaged film; removal of superimposed text like dates, subtitles, or publicity; and the removal of entire objects from the image like microphones or wires in special effects.
3,830 citations
"Image inpainting on satellite image..." refers background in this paper
...In Bertalmio [1] and Bertalmio [6], the image smoothness information, approximated by the image Laplacian, is propagated along the isophotes directions, estimated by the image gradient rotated 90 degrees....
[...]
...[1]....
[...]
...The results would look natural enough that observers without prior knowledge of the original image will not notice the gaps [1], an algorithm imitates the traditional inpainting processes, such as determine the area to be corrected, exam the boundary of the region to be filled, and continuing lines of similar color....
[...]
TL;DR: The simultaneous propagation of texture and structure information is achieved by a single, efficient algorithm that combines the advantages of two approaches: exemplar-based texture synthesis and block-based sampling process.
Abstract: A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: 1) "texture synthesis" algorithms for generating large image regions from sample textures and 2) "inpainting" techniques for filling in small image gaps. The former has been demonstrated for "textures"-repeating two-dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one-dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. In this paper, the simultaneous propagation of texture and structure information is achieved by a single , efficient algorithm. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects, as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.
3,066 citations
TL;DR: A variational approach for filling-in regions ofMissing data in digital images is introduced, based on joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data.
Abstract: A variational approach for filling-in regions of missing data in digital images is introduced. The approach is based on joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed by solving the variational problem via its gradient descent flow, which leads to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist's principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given. We conclude the paper with a number of theoretical results on the proposed variational approach and its corresponding gradient descent flow.
969 citations
01 Jul 2003
TL;DR: A new method for completing missing parts caused by the removal of foreground or background elements from an image, iteratively approximating the unknown regions and composites adaptive image fragments into the image to synthesize a complete, visually plausible and coherent image.
Abstract: We present a new method for completing missing parts caused by the removal of foreground or background elements from an image. Our goal is to synthesize a complete, visually plausible and coherent image. The visible parts of the image serve as a training set to infer the unknown parts. Our method iteratively approximates the unknown regions and composites adaptive image fragments into the image. Values of an inverse matte are used to compute a confidence map and a level set that direct an incremental traversal within the unknown area from high to low confidence. In each step, guided by a fast smooth approximation, an image fragment is selected from the most similar and frequent examples. As the selected fragments are composited, their likelihood increases along with the mean confidence of the image, until reaching a complete image. We demonstrate our method by completion of photographs and paintings.
687 citations
Additional excerpts
..., [10]....
[...]
04 Oct 1998
TL;DR: It is shown in this paper how disocclusion can be performed by means of a level lines structure, which offers a reliable, complete and contrast-invariant representation of an image, in contrast to edges.
Abstract: Object recognition, robotic vision, occluding noise removal or photograph design require the ability to perform disocclusion. We call disocclusion the recovery of hidden parts of objects in a digital image by interpolation from the vicinity of the occluded area. It is shown in this paper how disocclusion can be performed by means of a level lines structure, which offers a reliable, complete and contrast-invariant representation of an image, in contrast to edges. Level lines based disocclusion yields a solution that may have strong discontinuities, which is not possible with PDE-based interpolation. Moreover, the proposed method is fully compatible with Kanizsa's (1996) theory of "amodal completion".
593 citations