scispace - formally typeset
Search or ask a question
Proceedings Article•DOI•

Enhanced image capture through fusion

Peter J. Burt1, R.J. Kolczynski1•
Sarnoff Corporation1
11 May 1993-pp 173-182
TL;DR: The authors present an extension to the pyramid approach to image fusion that provides greater shift invariance and immunity to video noise, and provides at least a partial solution to the problem of combining components that have roughly equal salience but opposite contrasts.
Abstract: The authors present an extension to the pyramid approach to image fusion. The modifications address problems that were encountered with past implementations of pyramid-based fusion. In particular, the modifications provide greater shift invariance and immunity to video noise, and provide at least a partial solution to the problem of combining components that have roughly equal salience but opposite contrasts. The fusion algorithm was found to perform well for a range of tasks without requiring adjustment of the algorithm parameters. Results were remarkably insensitive to changes in these parameters, suggesting that the procedure is both robust and generic. A composite imaging technique is outlined that may provide a powerful tool for image capture. By fusing a set of images obtained under restricted, narrowband, imaging conditions, it is often possible to construct an image that has enhanced information content when compared to a single image obtained directly with a broadband sensor. >

Content maybe subject to copyright    Report

N94- 25502
Enhanced Image Capture Through Fusion
Peter J. Burt
Keith Hanna
Raymond J. Kolczynski
David Sarnoff Research Center
ABSTRACT
__- -- .7,.23
. ,,
/
Image fusion may be used to combine images from different sensors, such as IR and
visible cameras, to obtain a single composite with extended information content. Fusion
may also be used to combine multiple images from a given sensor to form a composite
image in which information of interest is enhanced.
We present a general method for performing image fusion and show that this method is
effective for diverse fusion applications. We suggest that fusion may provide a powerful
tool for enhanced image capture with broad utility in image processing and computer
vision.
pI_.GEDING P_GE Bt-Ar_iK NOT FILMED
207
The Fusion Task
9
Changing Parameters
(iris, exposure, f_
Changing Sensor
(IR, visible)
Scene
of
Int e rest
Occluding Object
(smoke, foreground object)
I 12 13!
I
\ ,
Set of Source Images
Set of Fused Images
208
Image Fusion: Objectives
Combine two or more source images to obtain a single
composite with extended information content.
IR
Visible
Fused
Requirements
° retain all useful information from the source images
• not introduce fusion artifacts into the combined image
• look"natural"
209
Technical Challenge
Pixel averaging, results in ...
A
B
Fused
1. Loss of
Contrast
210
Technical Challenge
Pixel averaging, results in ...
A
Fuse
"_ 2. Double
Exposure
211
Citations
More filters
Book•
Computer Vision: Algorithms and Applications

[...]

Richard Szeliski
30 Sep 2010
TL;DR: Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images and takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene.
Abstract: Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

4,146 citations

Journal Article•DOI•
A survey of medical image registration.

[...]

J. B. Antoine Maintz1, Max A. Viergever1•
Utrecht University1
01 Mar 1998-Medical Image Analysis
TL;DR: A survey of recent publications concerning medical image registration techniques is presented, according to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic methods.
Abstract: The purpose of this paper is to present a survey of recent (published in 1993 or later) publications concerning medical image registration techniques. These publications will be classified according to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic methods. The statistics of the classification show definite trends in the evolving registration techniques, which will be discussed. At this moment, the bulk of interesting intrinsic methods is based on either segmented points or surfaces, or on techniques endeavouring to use the full information content of the images involved.

3,426 citations

Cites background from "Enhanced image capture through fusi..."

  • ...aMostly the area of segmentation-free image fusion addresses this problem, but its applications to medical image problems are severely limited (Burt, 1993; Wahl et al., 1993; H. Li et al., 1994, 1995; Wasserman et al., 1994; Zhou, 1994; Chou et al., 1995; Wasserman and Acharya, 1995; Pietrzyk et al., 1996)....

    [...]

Journal Article•DOI•
Interactive digital photomontage

[...]

Aseem Agarwala1, Mira Dontcheva1, Maneesh Agrawala2, Steven M. Drucker2, Alex Colburn2, Brian Curless1, David Salesin1, Michael Cohen2 •
University of Washington1, Microsoft2
01 Aug 2004
TL;DR: The framework makes use of two techniques primarily: graph-cut optimization, to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion, a process based on Poisson equations, to further reduce any remaining visible artifacts in the composite.
Abstract: We describe an interactive, computer-assisted framework for combining parts of a set of photographs into a single composite picture, a process we call "digital photomontage." Our framework makes use of two techniques primarily: graph-cut optimization, to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion, a process based on Poisson equations, to further reduce any remaining visible artifacts in the composite. Also central to the framework is a suite of interactive tools that allow the user to specify a variety of high-level image objectives, either globally across the image, or locally through a painting-style interface. Image objectives are applied independently at each pixel location and generally involve a function of the pixel values (such as "maximum contrast") drawn from that same location in the set of source images. Typically, a user applies a series of image objectives iteratively in order to create a finished composite. The power of this framework lies in its generality; we show how it can be used for a wide variety of applications, including "selective composites" (for instance, group photos in which everyone looks their best), relighting, extended depth of field, panoramic stitching, clean-plate production, stroboscopic visualization of movement, and time-lapse mosaics.

1,072 citations

Cites methods from "Enhanced image capture through fusi..."

  • ...This problem has also been addressed using Laplacian pyramids [Ogden et al. 1985; Burt and Kolczynski 1993]....

    [...]

  • ...For digital images, the earliest and most well-known work in image fusion used Laplacian pyramids and per-pixel heuristics of salience to fuse two images [Ogden et al. 1985; Burt and Kolczynski 1993]....

    [...]

Proceedings Article•DOI•
Radiometric self calibration

[...]

Tomoo Mitsunaga1, Shree K. Nayar2•
Sony Broadcast & Professional Research Laboratories1, Columbia University2
23 Jun 1999
TL;DR: A simple algorithm is described that computes the radiometric response function of an imaging system, from images of an arbitrary scene taken using different exposures, to fuse the multiple images into a single high dynamic range radiance image.
Abstract: A simple algorithm is described that computes the radiometric response function of an imaging system, from images of an arbitrary scene taken using different exposures. The exposure is varied by changing either the aperture setting or the shutter speed. The algorithm does not require precise estimates of the exposures used. Rough estimates of the ratios of the exposures (e.g. F-number settings on an inexpensive lens) are sufficient for accurate recovery of the response function as well as the actual exposure ratios. The computed response function is used to fuse the multiple images into a single high dynamic range radiance image. Robustness is tested using a variety of scenes and cameras as well as noisy synthetic images generated using 100 randomly selected response curves. Automatic rejection of image areas that have large vignetting effects or temporal scene variations make the algorithm applicable to not just photographic but also video cameras.

837 citations

Cites methods from "Enhanced image capture through fusi..."

  • ...To ensure that our algorithm can lFor imaging systems with known response functions, the problem of fusing multiple images taken under different exposures has been addressed by several others (see [3], [IO], [7] for examples)....

    [...]

Journal Article•DOI•
A general framework for multiresolution image fusion: from pixels to regions

[...]

Gemma Piella1•
Centrum Wiskunde & Informatica1
01 Dec 2003-Information Fusion
TL;DR: The aim is to reframe the multiresolution-based fusion methodology into a common formalism and to develop a new region-based approach which combines aspects of both object and pixel-level fusion.
Abstract: This paper presents an overview on image fusion techniques using multiresolution decompositions. The aim is twofold: (i) to reframe the multiresolution-based fusion methodology into a common formalism and, within this framework, (ii) to develop a new region-based approach which combines aspects of both object and pixel-level fusion. To this end, we first present a general framework which encompasses most of the existing multiresolution-based fusion schemes and provides freedom to create new ones. Then, we extend this framework to allow a region-based fusion approach. The basic idea is to make a multiresolution segmentation based on all different input images and to use this segmentation to guide the fusion process. Performance assessment is also addressed and future directions and open problems are discussed as well.

832 citations

Cites background or methods from "Enhanced image capture through fusi..."

  • ...Burt and Kolczynski [ 3 ] proposed to use the gradient pyramid (hence P ¼ 4) together with a combination algorithm that is based on an activity and a match measure....

    [...]

  • ...Image fusion at pixel-level means fusion at the lowest processing level referring to the merging of measured physical parameters [ 3 ,4]....

    [...]

  • ...Fig. 12(d) illustrates the fusion algorithm proposed by Burt and Kolczynski [ 3 ]....

    [...]

References
More filters
Journal Article•DOI•
Image fusion by a ratio of low-pass pyramid

[...]

Alexander Toet
01 May 1989-Pattern Recognition Letters
TL;DR: A hierarchial image merging scheme based on a multiresolution contrast decomposition (the ratio of low-pass pyramid) to preserve those details from the input images that are most relevant to visual perception is introduced.
Abstract: This paper introduces a hierarchial image merging scheme based on a multiresolution contrast decomposition (the ratio of low-pass pyramid). The composite images produced by this scheme preserve those details from the input images that are most relevant to visual perception. Some applications of the method are indicated

611 citations

"Enhanced image capture through fusi..." refers methods in this paper

  • ...Toet (1989) proposed using a ratio of low pass pyramid and the same selection rule to fuse IR and visible image....

    [...]

  • ...Toet (1989) proposed using a ratio of low pass pyramid and the same selection rule to fuse IR and visible image. Pavel. et. al. (1991) have used a related pyramid technique and a noise based selection rule to fuse millimeter wave sensor images with computer generated graphics....

    [...]

Fast Filter Transforms for Image Processing

[...]

P. J. Burt
01 Jan 1981

460 citations

Journal Article•DOI•
Fast filter transform for image processing

[...]

Peter J. Burt1•
University of Maryland, College Park1
01 May 1981-Computer Graphics and Image Processing
TL;DR: A highly efficient recursive algorithm is defined for simultaneously convolving an image (or other two-dimensional function) with a set of kernels which differ in width but not in shape, so that the algorithm generates aSet of low-pass or band-pass versions of the image.
Abstract: A highly efficient recursive algorithm is defined for simultaneously convolving an image (or other two-dimensional function) with a set of kernels which differ in width but not in shape. These kernels may closely resemble the Gaussian probability distribution, so that the algorithm generates a set of low-pass or band-pass versions of the image. Image correlation with spot, edge, and bar operators of many sized can be obtained with negligible additional computation

442 citations

Patent•
Depth-of-focus imaging process method

[...]

Edward H. Adelson1•
RCA Corporation1
18 May 1984
TL;DR: In this paper, an improved-focus 2D image is derived from an assemblage of M separately focused 2D images of the same 3D scene by employing the Burt Pyramid image analyzing technique to separately analyze each of the separately focused images into N similar sets of pixel samples.
Abstract: An improved-focus 2-D image is derived from an assemblage of M separately focused 2-D images of the same 3-D scene by (1) employing the Burt Pyramid image analyzing technique to separately analyze each of the M separately focused images into N similar sets of pixel samples, (2) selecting, on a pixel-by-pixel basis from each group of M corresponding sets of the assemblage, the best focused pixel, to derive a single analyzed image of N sets of improved-focus pixels, and (3) employing the Burt Pyramid image synthesizing technique to synthesize the improved-focus 2-D image from the single analyzed image of N sets.

152 citations

Proceedings Article•DOI•
Sensor fusion for synthetic vision

[...]

Misha Pavel1, James Larimer1, Albert J. Ahumada1•
Ames Research Center1
01 Jan 1991
TL;DR: In this article, an approach to fusion based on multiresolution image representation and processing is described which facilitates fusion of images differing in resolution within and between images, and a workstation-based simulation environment is developed.
Abstract: Display methodologies are explored for fusing images gathered by millimeter wave sensors with images rendered from an on-board terrain data base to facilitate visually guided flight and ground operations in low visibility conditions. An approach to fusion based on multiresolution image representation and processing is described which facilitates fusion of images differing in resolution within and between images. To investigate possible fusion methods, a workstation-based simulation environment is being developed.

50 citations

Related Papers (5)
Multisensor image fusion using the wavelet transform

[...]

01 May 1995-Graphical Models and Image Processing
Hui Li, B.S. Manjunath, Sanjit K. Mitra
The Laplacian Pyramid as a Compact Image Code

[...]

01 Apr 1983-IEEE Transactions on Communications
Peter J. Burt, Edward H. Adelson
Objective image fusion performance measure

[...]

17 Feb 2000-Electronics Letters
Costas Xydeas, Vladimir Petrovic
A theory for multiresolution signal decomposition: the wavelet representation

[...]

01 Jul 1989-IEEE Transactions on Pattern Analysis and Machine Intelligence
Stéphane Mallat
A wavelet-based image fusion tutorial

[...]

01 Sep 2004-Pattern Recognition
Gonzalo Pajares, Jesus M. de la Cruz