Scanning Electron Microscopy for Quantitative Small and Large Deformation Measurements Part II: Experimental Validation for Magnifications from 200 to 10,000

TLDR: In this paper, a combination of drift distortion removal and spatial distortion removal is performed to correct Scanning Electron Microscope (SEM) images at both ×200 and ×10,000 magnification.

Abstract: A combination of drift distortion removal and spatial distortion removal are performed to correct Scanning Electron Microscope (SEM) images at both ×200 and ×10,000 magnification. Using multiple, time-spaced images and in-plane rigid body motions to extract the relative displacement field throughout the imaging process, results from numerical simulations clearly demonstrate that the correction procedures successfully remove both drift and spatial distortions with errors on the order of ±0.02 pixels. A series of 2D translation and tensile loading experiments are performed in an SEM for magnifications at ×200 and ×10,000, where both the drift and spatial distortion removal methods described above are applied to correct the digital images and improve the accuracy of measurements obtained using 2D-DIC. Results from translation and loading experiments indicate that (a) the fully corrected displacement components have nearly random variability with standard deviation of 0.02 pixels (≈25 nm at ×200 and ≈0.5 nm at ×10,000) in each displacement component and (b) the measured strain fields are unbiased and in excellent agreement with expected results, with a spatial resolution of 43 pixels (≈54 μm at ×200 and ≈1.1 μm at ×10,000) and a standard deviation on the order of 6 × 10−5 for each component.

Figures

Fig. 3 Overall procedure employed for correcting distortion in an SEM. Relaxation methods are implemented during process to improve convergence

Fig. 11 Measured horizontal and vertical spatial distortion fields in an FEI Quanta 200 SEM at magnifications of (a) ×200 and (b) ×10,000

Fig. 16 Strain error fields, E( , measured when uncorrected images are translated five pixels horizontally (U) and vertically (V). Errors estimated using gradients of measured spatial distortion field

Fig. 4 Drift distortion and spatial distortion components for computer simulations. Spatial distortion ranges from 0.5 to+1.5 pixels within an image. Total drift displacements range up to 20 pixels over 120 min

Fig. 10 Typical drift distortion functions for (a) ×200 magnification at time t 36 min and (b) ×10,000 magnification at time t 30 min

Fig. 9 SEM BSE images of (a) lithographically applied gold pattern on aluminum surface for ×200 magnification and (b) rearranged gold pattern on aluminum surface for ×10,000 magnification

Full text

HAL Id: hal-01644895
https://hal.archives-ouvertes.fr/hal-01644895
Submitted on 23 Feb 2018
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Scanning electron microscopy for quantitative small and
large deformation measurements - part II: Experimental
validation for magnications from 200 to 10,000
M. A. Sutton, N. Li, David Garcia, Nicolas Cornille, Jean-José Orteu, S. R.
Mcneill, H. W. Schreier, Xiaojian Li, A. P. Reynolds
To cite this version:
M. A. Sutton, N. Li, David Garcia, Nicolas Cornille, Jean-José Orteu, et al.. Scanning electron mi-
croscopy for quantitative small and large deformation measurements - part II: Experimental validation
for magnications from 200 to 10,000. Experimental Mechanics, Society for Experimental Mechanics,
2007, 47 (6), p.789-804. �10.1007/s11340-007-9041-0�. �hal-01644895�

Scanning Electron Microscopy for Quantitative
Small and Large Deformation Measurements
Part II: Experimental Validation for Magnifications
from 200 to 10,000
M.A. Sutton & N. Li & D. Garcia & N. Cornille &
J.J. Orteu & S.R. McNeill & H.W. Schreier & X. Li &
A.P. Reyno lds
Abstract A combination of drift distortion removal and
spatial distortion removal are performed to correct Scan
ning Electron Microscope (SEM) images at both ×200 and
×10,000 magnification. Using multiple, time spaced images
and in plane rigid body motions to extract the relative
displacement field throughout the imaging process, results
from numerical simulations clearly demonstrate that the
correction procedures successfully remove both drift and
spatial distortions with errors on the order of ±0.02 pixels.
A series of 2D translation and tensile loading experiments
are performed in an SEM for magnifications at ×200 and
×10,000, where both the drift and spatial distortion removal
methods described above are applied to correct the digital
images and improve the accuracy of measurements
obtained using 2D DIC. Results from translation and
loading experiments indicate that (a) the fully corrected
displacement components have nearly random variability
with standard deviation of 0.02 pixels (≈25 nm at ×200 and
≈0.5 nm at ×10,000) in each displacement component and
(b) the measured strain fields are unbiased and in excellent
agreement with expected results, with a spatial resolution of
43 pixels (≈54 μm at ×200 and ≈1.1 μm at ×10,000) and a
standard deviation on the order of 6×10
5
for each
component.
Keywords Scanning electron microscopy
.
High and low
magnification
.
Uniaxial tension experiment
.
Drift and
distortion correction
.
2D digital image correlation
Introduction
Accurate calibration of single camera [1, 2] or stereo vision
systems at the micro scale, including the determination and
correction of the underlying distortions in the measurement
process, has received limited attention in the literature. One
reason for this difficulty is the complexity of high
magnification imaging systems, resulting in non parametric
disto rtions tha t invali date the common approa ches for
identifying and removing distortions in simple lens systems
[3, 4].
In his pioneering work, Schreier et al. proposed a new
methodology to calibrate accurately any imaging sensor by
correcting a priori for spatial distortion using a non
parametric model [5]. The correction process transforms
the imaging sensor plane into a virtual distortion free sensor
plane using simple translations of a speckled or gridded
planar target. If a speckled target is translated, then the
M.A. Sutton (*, SEM member)
:
N. Li (SEM member)
:
S.R. McNeill (SEM member)
:
X. Li (SEM member)
:
A.P. Reynolds
Department of Mechanical Engineering,
University of South Carolina,
Columbia, SC 29208, USA
e mail: Sutton@sc.edu
D. Garcia
108 Ennismore Common Lane,
Columbia, SC 29229, USA
H.W. Schreier (SEM member)
Correlated Solutions, Inc, 952 Sunset Boulevard,
West Columbia, SC 29252, USA
N. Cornille
:
J.J. Orteu
Ecole des Mines d’Albi, Albi, France

same target can be used to perform camera calibration of
the distortion corrected virtual imaging sensor using un
known arbitrary motions. As opposed to classical calibra
tion techniques (relying on a dedicated target marked with
fiducial points), this approach can be applied using any
randomly textured planar object. This type of dist ortion
depends only on the pixel position in the image and is
designated as spatial distortion in this work.
Due to the nature of white light, optical imaging systems
are limited to a maximum resolution that corresponds to a
magnification of ≈×1,000. For higher spatial resolution
imaging, systems based on electron microscopy (such as an
SEM) have been used successfully. Since the physics of
electron microscopy is quite different from optical micros
copy, it became apparent early in the studies that a novel
image analysis procedure, image model and calibration
process would be necessary so that accurate measurements
could be extracted from the digitized SEM images. For
example, SEM systems have not only spatial distortion but
also a temporal ly varying distortion, oftentimes known as
drift distortion. In fact, most papers and even commercial
SEM measurement systems simply ignore these effects and
consider a pure projection model [6 11]. Though a few
authors do take into account distortion (considering
parametric distortion models [12 14]), the effect of drift
distortion is generally not considered in experimental
studies [15].
In a recent study, the authors proposed an imaging model
and a distortion correction methodology to remove both
drift and spatial distortions from SEM images [16],
demonstrating that the method is effective for correcting
SEM images at relatively low magnification.
In this study, “Scan Process Modeling in an SEM” and
“Imaging Model for Planar Object” Sections presents
additional details for the method described in [16].
“Numerical Simulations” Section presents numerical simu
lation results confirming that the procedure for quantifying
drift and spatial distortion is effective. “Experiments”
Section overviews the pattern development, experimental
setup and image acquisition procedures, presenting results
from a series of 2D experiments using an SEM to obtain
data at ×200 and ×10,000 magnifications. “Discussion”
Section discu sses the results and highlights key aspects of
the measurement methodology. “Concluding Remarks”
Section provides concluding remarks.
Scan Process Modeling in an SEM
As discussed in Part I, each SEM image is generated pixel
by pixel following a rastering process. Each pixel requires a
dwell time, t
D
, to define the “intensity” of image at that
location, so that the required scanning time for an entire
row, t
R
, and an entire frame, t
F
, of image are given by
t
R
¼ Wt
D
þ t
j
ð1Þ
t
F
¼ Ht
R
¼ HWt
D
þ Ht
J
ð2Þ
where W is the width of the image in pixels and H is the
height of image in pixels, t
J
is the time delay in seconds for
repositioning the e beam and stabilizing it to the next row.
1
Since the (x,
y
) position in the image is in direct correspon
dence with the scan time, one can write
tx; yðÞ¼xt
D
þ yt
R
ð3Þ
where 0 % x % W & 1and0% y % H & 1. Thus, t =0
corresponds to the beginning of the scan for pixel (0,0).
Equation (3) implies that two pixel positions, and hence
two times, will generally experience much different drift
functions. Such differences have been observed for con
secutive rows in an image, with a clearly de fined shift in
drift measured when moving from the last pixel in one row
to the first pixel in the next row.
If one considers the time delay between acquiring
consecutive images, then the total time from the beginning
of the scan for the first image to the current time in the nth
image for a given location (x,y) is written
Tx; yðÞ¼
X
N 1
n 1
t
n
þ N & 1ðÞt
F
þ tx; yðÞ ð4Þ
where t
n
is the recorded delay time between images.
Imaging Model for Planar Object
It is assumed that (a) the specimen is nearly planar, (b)
translations and/or rotations occur within the plane of the
specimen, and (c) deformations applied to the specimen occur
within the plane of the specimen. Thus, the imaging process
can be viewed as being a plane to plane transformation.
Assuming a perspective projection model for the imaging
process, the equations in terms of undistorted coordinates in
the image plane can be written in the simple form
x
y
"#
¼
M
x
0
0 M
y
"#
x
0
y
0
"#
þ
C
x
C
y
"#
ð5Þ
where (x,y )
T
are in pixel coordinates; (M
x
,M
y
)
T
transform
object coordinates (x
0
,y
0
)
T
into the sensor plane and have
units pixel/mm on the object and (C
x
,C
y
)
T
designates the
location of the image center in the sensor plane. Additional
“se nsor p lane scale factors” may be determined by
1
For images integrated over M scans, the dwell time is the sum of the
dwell times for each integrated scan.

manufacturer information regarding the physical size of the
pixels that are used to digitize the image, though their
values are not required unless the true magnification factor
is desired. As shown in equation (5), there are four unknown
parameters in the image model.
Image Distortions in an SEM
It is assumed that the distortions in an SEM can be repre
sented by drift and spatial vector functions [16]. First,
D
dr
(t)=[ δ
x
(t), δ
y
(t)]
T
is defined as the drift distortion
function in two orthogonal directions, where t is the scan
time and t ∈ [0,∞). Second, D
sp
(r)=D
sp
(x,y) is defined as
the spatial distortion function in two orthogonal directions,
where r=(x,y)
T
is the undistorted pixel position of a point
on the image plane. Thus, the distorted position R of the
point can be written;
R ¼ r þ D
dr
tÞþD
sp
rðÞ
!
ð6Þ
Since the distorted positions of points, R,arethe
measurable quantities using DIC, the inverse functions for
both drift and spatial distortion are the quantities obtained
during the calibration process
2
so that the undistorted
(ideal) positions of the points, r, can be estimated.
Drift Distortion
Drift distortion, or temporally varying distortion, is present
at all magnifications in an SEM, though it is particularly
noticeable at high magnification. Figure 1 presents mea
surements of t he ve rtical drift displacement from the
beginning to the end of a scan for images acquired at three
different times. The data shows three trends that the authors
have observed at all magnifications. First, there is random,
pixel to pixel noise throughout the scan process. Second,
drift displacement changes with time in a non linear
manner. Third, the magnitude of the relative drift within
each image changes with time. As shown in Fig. 1, the drift
distortion within an image ranges up to 0.37 out of
1,024 pixels, introducing a strain error of ≈3.7×10
4
.
To quantify the drift distortion at each pixel throughout an
experiment, several approaches have been investigated.
2
Though the focus of this study is on 2D image correlation (a single
view), the procedure can be applied to each view in a stereo system to
remove distortions.
Fig. 1 Non uniform image drift in the vertical direction within each image at ×200. Drift variations are primarily due to time representation of the
data, where step changes occur at end of each line

Preliminary experiments demonstrated that a global model
cannot adequately represent experimental observatio ns
whereas a local model will provide good agreement with
experimental measurements. Based on these investigations, a
novel drift correction procedure has been developed [16]that
is consistent with experimentally observed SEM imaging.
Spatial Distortion
In simple lens systems, e.g. a typical digita l camera,
spatially varying distortion (spatial distortion or image
distortion) is a well known problem. The commonly used
method for modeling such imaging systems assumes that the
distortions (deviations from the ideal image positions) are
due to factors such as lenses aberrations, misalignment of
optical elements, non parallelism between image plane and
sensor plane, lens curvature imperfection, each of which can
be estimated using established parametric models.
Parametric model
Classical models used to estimate the spatial distortions are
parametric in nature [17 19], typical forms include radial
distortion, de centering distortion, prismatic distortion and
tangential distortion. For example, one may write a radial
distortion function, F, in the form
x
d
; y
d
ðÞ
T
¼ F r
m
; C
x
; C
y
!"
r
2
¼ x & C
x
ðÞ
2
þ y & C
y
!"
2
ð7Þ
where (x
d
, y
d
)
T
is the distorted position in the image plane
of the undistorted point (x, y)
T
and m is the power of r to be
a parameter used in the distortion model. Typically, the
distortion function parameters are obtained at the same time
as the imaging parameters in equation (5) using a non linear
optimization process.
Non parametric model
If non parametric models for distortion correction a re
employed, then the role of the center in the mapping
process generally is embedded in the distortion correction
process and is not determined separately. If dist ortion
correction is performed prior to calibration, then the only
parameters to be determined during the calibration process
are the magnification factors. The magnification factors
typically are determined through (a) known motions of
the object, (b) points on the object with know n spacing,
or (c) combinations of both. If distortion correction is not
performed prior to calibration, then the magnification
factors and the distortion param eters typically are
obtained through the calibration proces s.
Since the SEM imaging proces s is based upon the
interaction between atoms of the observed specimen and an
e beam, as well as scanning and focusing processes that
employ electro magnetic principles to perform the required
functions, pre specified classical distortions are unlikely to
be effective when used to estimate arbitrary aberrations or
unknown (but deterministic) distortions in a complex
imaging system such as an SEM. To deal wi th this, the
method outlined by Schreier et al. [5], which employs B
Splines or other general forms, can be used to quantify the
full field spatial distortions present in an SEM image.
Finally, it is noted that accurate grid targets are used in
typical imaging systems to quantify spatial distortions.
Since accurate grid targets used in previous non parametric
distortion studies [20, 21] are likely to be difficult to realize
at the micro or nano scale for the SEM, the method
developed recently [5, 22] that employs arbitrary, unknown
translations of the randomly speckled specimen for calibra
tion at reduced length scales, is used in this current study.
Drift and Spatial Distortion Correction for Calibration
Figure 2 presents a schematic of the proces ses used to
(a) quantify both drift and spatial distortion fields across the
images, (b) calibrate the SEM, and (c) measure the desired
deformations during an experiment. As shown on the left
side of Fig. 2, pairs of images are acquired of the specimen
as it is undergoing a series of in plane translations. Each
pair of images is acquired without specimen motion. The
in-plane translations
Calibration Phase
drift distortion estimation and
s
p
atial distortion determination
Image 2
Image 1
Image N
C
Image
N
C
–1
Image 4
Image 3
loading, heating, …
Measurement Phase
new drift distortion estimation with spatial distortion
removal, deformation determination
Image 2
Image 1
Image N
Image
N–1
Image 4
Image 3
Fig. 2 Schematic of overall image acquisition, image correction and deformation measurement process in the SEM

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "Scanning electron microscopy for quantitative small and large deformation measurements - part ii: experimental validation for magnifications from 200 to 10,000" ?

Sutton et al. this paper used a combination of drift distortion removal and spatial distortion removal to correct Scanning Electron Microscope ( SEM ) images at both ×200 and ×10,000 magnification. 

Q2. What have the authors stated for future works in "Scanning electron microscopy for quantitative small and large deformation measurements - part ii: experimental validation for magnifications from 200 to 10,000" ?

In sharp contrast with the approach of early SEM measure ments, where the investigators simply accepted the accura cy obtainable and successfully performed their studies for important problems amenable to such limitations, this work presents and validates a general approach that successfully extends the range of measurements obtainable in an SEM to the small deformation ( elastic ) regime so that full elastic plastic deformation studies can be performed in an SEM. The novel method outlined in this work relies on a combination of drift distortion correction and a priori spatial distortion correction so that accurate elastic and elastic plastic deformation measurements can be obtained using SEM images ; both corrections are essential to obtain accurate deformation measurements throughout the field. Given these issues, it is essential that baseline studies be performed to identify potential problems prior to performing the critical experi ments. Simulation results have shown that typical drift process es in an SEM can be adequately reconstructed using local drift velocity measurements. 

Q3. What is the significance of the spatial distortion correction procedure?

since temporally varying drift will introduce image trans lations, coupling exists at all times between spatial and drift distortions and confirms the importance of drift distortion removal for accurate, image based deformation measurements. 

Q4. What is the spatial distortion function used to correct all spatial positions?

The spatial distortion function obtained during calibration and the new local drift function obtained for the set of strain images are used to correct all spatial positions. 

Q5. What are the main issues that must be addressed and resolved?

Regarding the miniature loading frame used to load the specimen, issues such as specimen misalignment, inadequate specimen gripping, imprecise loading and/or applied displacement and instability of loading platform within the SEM must be addressed and resolved to ensure repeatability in the 2D measurements. 

Q6. What is the average strain of the displacement component?

With Gaussian noise having a standard deviation of 0.025 pixels in each displacement component,all strains have an average strain between *6) 10 5 and a standard deviation ≈5×10 5. 

Q7. What is the typical accuracy for rotational position on SEM systems?

In most SEM systems, (a) transla tion stage movements are performed by click and drag processes that are relatively inaccurate, (b) translation stage control systems are generally prone to backlash and/or overshoot, and (c) stages generally have few features available to maintain constant height and orientation of the stage; ±0.5 degrees is a typical accuracy for rotational position on most SEM systems. 

Q8. How many scans are used to represent the image?

For the ×10,000 experiments, image integration is performed with 16 scans combined to represent a single image; the total image acquisition time remains fixed at tF=75.68 s with all other parameters remaining the same. 

Q9. What are the sources of measurement error in an SEM?

Sources of measurement error in an SEM may also include the effects of environmental factors such as mechanical vibrations and sound. 

Q10. What is the procedure used to minimize the effects of drift distortion during the initial loading process?

The procedure whereby the drift distortion is computed separately for the calibration and measurement phases is used in practice to minimize the effects of specimen shifts during the initial loading process. 

Q11. What is the role of the center in the distortion correction process?

If non parametric models for distortion correction are employed, then the role of the center in the mappingprocess generally is embedded in the distortion correction process and is not determined separately. 

Q12. What is the definition of the distorted position of a point in the image plane?

Dsp(r)=Dsp(x,y) is defined as the spatial distortion function in two orthogonal directions, where r=(x,y)T is the undistorted pixel position of a point on the image plane. 

Q13. How many pairs of images are acquired during the measurement phase?

The number of pairs of images during the measurement phase will vary with the number of strain increments; for better estimation of B Spline function it should be more than six pairs of images acquired. 

Q14. What is the standard deviation of the measured strains?

after using the Butterworth Filter the standard deviation in the measured strains is less than 1×10 4 for all components with a spatial resolution of 43 pixels for themeasurements, the same spatial resolution expected using a 43×43 subset. 

Q15. How can noise be removed from the measurements?

Using Gaussian noise with this range, their studies have shown that the noise can be removed from the measurements using a Butterworth Filter with a spatial cutoff wavelength equal to half of the subset size (22 pixels), without decreasing the spatial frequency content of the underlying displacement measurements. 

Q16. How much strain error does the drift distortion in an image bring?

As shown in Fig. 1, the drift distortion within an image ranges up to 0.37 out of 1,024 pixels, introducing a strain error of ≈3.7×10 4.