Review Article
A Review of Machine Vision-Based Structural Health
Monitoring: Methodologies and Applications
X.W.Ye,C.Z.Dong,andT.Liu
Department of Civil Engineering, Zhejiang Uni versity, Ha n gzhou 310058, China
Correspondence should be addressed to X. W. Ye; cexwye@zju.edu.cn
Received March ; Revised September ; Accepted October
AcademicEditor:CalogeroM.Oddo
Copyright © X. W. Ye et al. is is an open access article distributed under the Creative Commons Attribution License, which
permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In the past two decades, a signicant number of innovative sensing and monitoring systems based on the machine vision-based
technology have been exploited in the eld of structural health monitoring (SHM). is technology has some inherent distinctive
advantages such as noncontact, nondestructive, long distance, high precision, immunity to electromagnetic interference, and
large-range and multiple-target monitoring. A lot of machine vision-based structural dynamic measurement and structural state
inspection methods have been proposed. Real-world applications are also carried out to measure the structural physical parameters
such as the displacement, strain/stress, rotation, vibration, crack, and spalling. e purpose of this review article is devoted
to presenting a summary of the basic theories and practical applications of the machine vision-base d technology employed in
structural monitoring as well as its systematic error sources and integration with other modern sensing techniques.
1. Introduction
e technology of structural health monito ring (SHM)
emerged with an essential goal of safeguarding the opera-
tional safety of engineering structures, through deploying
various types of sensors, monitoring diversied physical
quantities, assessing structural condition and perfo rmance,
and instructing routine inspection and maintenance [–].
With regard to a large-scale SHM system, the innovative sens-
ing technologies from a variety of elds, such as mechanics,
electricity, electromagnetism, optics, thermology, and chem-
istry, make great contribution in accurately acquiring the
huge amount of original data reecting the real environ-
mental and structural conditions. In the p ast three decades,
worldwide researchers have devoted a considerable number
of eorts in the development of novel sensing technologies
for application in the SHM research eld and achieved tre-
mendous progresses.
With the great advances in optics device and computer
science, the machine vision-based sensing and monitor-
ing technology has been a cutting-edge research eld and
increasingly gained attentions from the civil engineering
communities [–]. It is mainly due to its unique advantages
of noncontact, long distance, high precision, immunity to
electromagnetic interference in multipoint, and large-range
structural measurement/monitoring [–]. Up to now,
many vision-based analysis methods have been developed
for structural displacement measurement, strain/stress mon-
itoring, vibration response monitoring, crack or defection
inspection, and characterization, among others [–].
In the past years, review works referring to the machine
vision-based structural monitoring and condition assessment
were carried out by some researchers. Wu and Casciati []
gave a brief introduction of the vision-based positioning
system for structural monitoring. Jiang et al. [] reviewed the
development and application of close-range photogrammetry
in deformation and geometry measurement of bridges. Koch
et al. [] presented a comprehensive synthesis of the state of
the art in the concrete and asphalt structure defect detection
and condition assessment based on the computer vision
technology. However, an all-round summary of the vision-
based structural monitoring and condition assessment is still
desirable. is paper aims to provide a comprehensive review
of machine vision-based monitoring of civil engineering
infrastructure focusing on the relevant methodologies and
practical applications.
Hindawi Publishing Corporation
Journal of Sensors
Volume 2016, Article ID 7103039, 10 pages
http://dx.doi.org/10.1155/2016/7103039
Journal of Sensors
Structure
Field of view
Image
Patterns
Computer
Camera
Targets
F : Two-dimensional vision-based displacement measurement method.
Target
Plane a
Plane b
Actual coordinate
a coordinate
b coordinate
t
2
t
1
t
1
a
t
1
a
t
2
b
t
2
b
F : ree-dimensional vision-based displacement measurement method [].
2. Machine Vision Methods
A vision-based measurement system generally consists of the
image acquisition device (dig ita l camera, lens, and image
grabber), the computer, and an image processing soware
platform. In these mentioned components, the image pro-
cessing soware platfor m acts as the critical part which will
be integrated with the specic computational algorithms to
obtain the mechanical parameters in structural monitoring.
2.1. Image Processing Algorithms. As illustrated in Figure ,
theimagesincludingthepredenedtargetsarecapturedby
the digital camera. With the digital image processing and
pattern matching algorithm [ –], the targets are t racked
and the structural displacements at the target positions on
the structure can be obtained. In this occasion, the horizontal
and vertical displacements, calle d two-dimensional (D)
displacements, can be obtained with one digital camera by use
of the appropriate image processing method, such as digital
image correlation [], mean-shi tracking algorithm [],
CamShi tracking algorithm [], and Lucas-Kanade method
[].
As illustrated in Figure , when two cameras are used
to capture the targets simultaneously and the geometrical
relationship b etween the two cameras (camera 𝑎 and camera
b) is conrmed, the displacements of the targets in three coor-
dinate directions, called three-dimensional (D) displace-
ments, can be derived by reconstructing the actual spatial dis-
placements with the position changes on the two conrmed
camera coordinates (t he coordinates of Plane 𝑎 and Plane b)
and the relationship between the two cameras [].
For the image processing algorithms applied in structural
monitoring, a considerable amount of research has been
carried out over the last decades. Wang et al. [] presented
a method to get the displacement results from the interfer-
ometric images by use of the phase-shied image matching
algorithm. Pieraccini et al. [] obtained the structural dis-
placement of real-sca le buildings from t he images captured
by a microwave interferometer. Guo and Zhu [] proposed
a modied inverse composit ional algorithm to reduce the
computing time of the Lucas-Kanade template tracking algo-
rithmandimprovedtheeciencyofcomputervisionmethod
furtherintheremotemeasurementofdynamicdisplacement.
Fukudaetal.[]developedarobustobjectsearchalgorithm
Journal of Sensors
enabling accurate displacement measurement through track-
ing existing features on the structure.
Buscaetal.[]usedtwotypesofcamerastoacquire
multiple targets xed on a railway bridge during the passage
of a train and obtained the displacement responses by three
vision methods, that is, digital image correlation method,
edge detection method, and pattern matching method. Lee
et al. [] proposed a pose-graph optimized displacement
estimation method for reducing the estimation errors of a
visually served paired structured light system. Nayyerloo et
al. [] developed a vision system to monitor the seismic
respons e of struc tures with a line scan camera. Chan et al.
[] proposed a CCD camera-based method to measure the
vertical displacement of bridges. Santos et al. [] performed
the vision system calibration in structural displacement
measurement of long-deck suspension bridges.
2.2. Systematic Errors Assessment and Reduction. Dierent
kinds of errors will occur in the application of vision-
based measurement system. It is critical to nd the inuence
factors, assess the systematic errors, and develop appropriate
algorithms to reduce the errors. Lava et al. [, ] estimated
the errors of system in the digital image correlation procedure
for large plastic deformation monitoring and investigated
the inuence of dierent causes, including the subset shape
function interpolation order, adopted correlation coecient,
and subset size. Schreier et al. [, ] analyzed the errors
caused by the use of undermatched shape function and gray-
value interpolation in structural displacement measurement.
Bornert et al. [] studied the displacement errors assessment
from synthetic speckle images and identied various error
regimes. Yoneyama et al. [] evaluated the eects of lens
distortion on the displacement measurement and proposed
a c orrection method when using the digital image corre-
lation. Yu and Pan [] investigated the errors due to the
overma tched subset shape function via the numerical tests
with deformation and the images from real experiments with
high strain gradients. Baldi and Bertolino [] conducted
an experimental study to describe the errors caused by
interpolation options. Zhou et al. [] presented a method
about the adaptive image subset oset to decrease the system
errors in the incremental image correlation. Crammond et al.
[] investigated the relationship among the size and density
of speckles and the measurement error within a pattern and
identied that the physical properties had a sign icant impact
on the precision of the displacement measurement. Yaofeng
andPang[]investigatedtheeectofsubsetsizeonthe
accuracy of deformation measurement when using a digital
image correlation algorithm. Lava et al. [] estimated the
errors produced when the camera alignment was nonper-
pendicular to planar sheet metal specimen’s surface in a
numerical experiment. Lecompte et al. [] stated that the size
of the speckle and the used pixel subset notably aected the
error magnitude of the disp lacement measurement.
Santos et al. [] proposed a vision system calibra-
tion approach to obtain an initial estimation of the object
shape and camera parameters to reduce measurement errors.
Ribeiro et al. [] presented a video-based system for the
dynamic displacement measurement of railway bridges and
investigated several inuence factors which will aect the
measurement precision of the video-based system. Ma et
al. [] studied the error of strain measurement in digital
image correlation method caused by self-heating of digital
cameras. Haddadi and Belhabib [] investigated the strain
measurement errors due to digital image correlation tech-
nique with rigid-body motion. Fazzini et al. [] estimated
the errors due to digital image correlation in displacement
measurement based on the generation of composite image
models of genuine speckle patterns. Wu et al. [] mounted
a vision system to monitor the D plane dynamic response of
a reduced scale frame xed on a shaking table and discussed
the physical meanings of the camera parameters, the balance
between the system resolution and its eld-of-view, and the
upper limitation of maker density which would restrict the
systematic error and measurement resolution.
e risk of the measurement uncertainty in the applica-
tion of vision-based techniques to vibrating target measure-
ments is very likely increased due to the motion blur gener-
ated by the camera-target motion. e motion blur will lead
to signicant systematic errors and i ncomplete measurement
data because the target seeking process may not give exact
detection. In recent years, research eor ts have been devoted
to the development of deblurring and denoising algorithms
and blur image analysis methods [–]. Wang et al. []
proposed a method for vibration measurement based on the
blurred images with the aid of the relationship between the
geometric moments of the unblurred and blurred motion.
Peng et al. [] developed an image restoration method for
theimprovementofthequalityofdynamicparticleimages
forthepurposeofsolvingthemotion-blurredproblemin
an online particle imaging system for wear debris analysis.
Be cker [] conducted a study of motion blur evaluation
by use of dierent basic approaches and instruments with a
variety of parameter variations. Wu et al. [] presented a
ro w by row degradation model of the images and developed a
restoration approach to compensate the space-variant degra-
dation. Ishida et al. [] proposed a method to improve the
recognition accuracy of camera-captured charac ters without
restoring images.
3. Applications of Machine Vision Technology
3.1. Two-Dimensional (2D) Structural Displacement Monitor-
ing. With the image sequence, pattern matching algorithm,
edge detection algorithm, and other image pro cessing tech-
nologies, the structural displacement of predened targets
can be obtained. is can be used to obtain the dynamic
displacement of several selected points on a certain structure
forthepurposeofstructuralmonitoring.Fengetal.[]
proposed a vision system for noncontact structural displace-
ment measurement in real time with the aid of an advanced
template matching algorithm. Henke et al. [] measured
the deformation of building structures by use of digital
image processing technique regarding the LED as the vision
target. Park et al. [] proposed a displacement measurement
method based o n machine vision technology to monitor the
displacement of high-rise building structures by use of the
partitioning approach and the verication experiments were
Journal of Sensors
conducted on a exible steel column. J
´
auregui et al. []
employed the close-range terrestrial digital photogrammetry
to measure the vertical deection of bridges. Yoneyama et al.
[] used the digital image correlation to monitor the deec-
tion of a new-built steel girder bridge during load tests. Kohut
et al. [] validated the feasibility and precision of a vision
method for the measurement of steel bridge displacement.
Dworakowski et al. [] used the bridge deection c urve
obtained by the in-plane displacement measurement based
on the vision method to analyze the damage of the cantilever
beam structures.
Lee et al. [] employed the digital image processing
techniques to obtain the real-time displacement of bridges
and assess the bridge load carrying capacity. Ho et al. []
developed a synchronous vision system for the real-time
multipoint displacement measurement of civil infrastructure.
Yang et al. [] proposed an image-based method to measure
the structural displacement, plane strain eld, and cracks
on the surface of the specimens under seismic loads. F u
and Moosa [] proposed an optical method for displace-
ment measurement with a high-resolution C CD camera.
Olaszek [] presented a computer vision method for real-
time measuring the structural displacement and dynamic
characteristics of bridges. Lee and Shinozuka [] proposed a
vision-based system to measure the dynamic displacement of
bridges in real time with the aid of digital image processing
techniques. Wahbeh et al. [] developed a vision-based
method to measure the absolute displacements in real time
at selected locations of infrastructure.
e great advantage of the vision-based structural dis-
placement measurement method is that the measurement
targets can be multiple as soon as they are in the captured
images. Choi et a l. [] introduced a dynamic displacement
vision system which could perform the multimeasurement
positions using a handset digital camcorder and the region
of interest (ROI) was proposed to improv e the measurement
eciency. Jurjo et al. [] measured the large displacement at
several points of the membrane simultaneously and estimated
the strain and stress from the measured displacement. Lin
et al. [] presented a videogrammetry system to monitor
the dynamic behavior of membrane roof structures. Lee and
Shinozuka [] developed a real-time vision-based system
for structural displacement measurement of bridges by use
of digital image processing techniques.
3.2. ree-Dimensional (3D) Structural Displacement Mon-
itoring. As mentioned in Section ., the D (or in-plane)
structural displacement can be obtained with the image
sequence captured by only one camera. Combined with two
or more digital cameras, two dierent image sequences from
two shooting angles are captured, and the three-dimensional
(in-plane and out-plane) structural displacement of selected
points on a certain structures can be realized with vision
reconstruction techniques. Grano and Zinno [] designed a
computer vision system for displacement monitoring during
destructive tests. Park et al. [] presented an approach to
monitor the three-dimensional structural displacements with
the aid of a high speed motion-capture system which has the
advantages of high accuracy and high sampling rate. Jeon et
al. [] proposed a vision system with an articial marker
to monitor six-degree-of-freedom (-DOF) structural dis-
placements. Park et al. [] used a motion-capture system to
obtain the D displacement response of structures in wind
tunnel experiments and obtained the dynamic properties
of the test structure, including the natural f requency, mode
shape, and damping ratio. Jeon et al. [] developed vision
system to measure the -DOF structural displacement based
on the paired visual servoing method. Leifer et al. []
performed three-dimensional acceleration measurement by
a vide ogrammetry system through tracking the motion of
targets on a modal shaker. Synnergren and Sj
¨
odahl []
developed a photography system with stereoscopic digital
speckles for D displacement eld measurements and a cam-
era calibration algorithm was used to evaluate the eect of
lens distortion. Vi
´
eville and Lingrand [] developed a visual
motion perception module to estimate D displacements
without calibration.
Hu et al. [] proposed a -camera video system for D
motion measurement of deformable objects. Ji and Chang
[] presented a marker-free stereovision method to monitor
responses of the line-like structures in both spatial and tem-
poral domains. Chang and Ji [] proposed a videogrammet-
ricmethodbasedontheprincipleofdigitalphotogrammetry
in close-range and computer vision technology to measure
the D structural vibration response in lab and proposed a
two-step calibration process to overcome the lens distortion
problem. Lee et al. [] developed a vision-based displace-
ment measurement system with digital image processing
techniques for real-time structural health monitoring of civil
structures.ChangandXiao[]presentedasingle-camera
approach for simultaneously measuring the D motion
(including the translation and rotation) of a target attached
on civil structures. Greenbaum et al. [] developed a vision
method to measure rigid-body motion including (translation
and rotation) experimental displacement in three-dimension
rocking motion.
3.3. Structural Strain and Stress Monitoring. e vision-based
methods were applied to acquire the structural strain and
stress by use of the structural displacement obtained by
thevisionsystemandtherelationshipbetweenthestruc-
tural displacement and the strain and stress derived in the
eld of material mechanics. Carmo et al. [] developed a
method for assessment of steel strains on reinforced concrete
members using solely surface measurement (crack width
and spacing) with the aid of photogrammetry and image
processing. Patterson et al. [] described a material for the
purpose of reference and calibration of the optical system for
strain monitor ing and designed a standardized test materia l.
Winkler et al. [] employed the digital image correlation
method to measure the local deformations in steel monos-
trands.DePauwetal.[]useddigitalimagecorrelation
method to monitor the fatigue parameters of the coupon scale
fretting tests. Gales et al. [] introduced a digital image cor-
relation method to measure the deformation and strain of the
prest ressing steel during high-temperature tests. Wang and
Cuitio [] applied the digit al image correlation technique
to capture the deformation patterns of polymeric foams.
Journal of Sensors
McGinnis et al. [] applied the D digital image corre-
lation method for determination of in situ stresses of concrete
structures.ObaidatandAttom[]usedtwoCCDcameras
to obtain the strain in soil specimens of two soil tests.
Maekawa et al. [] proposed a noncontact measurement
method based on the optical displacement sensors using
LEDs to measure the vibration stress and used the acquired
stress to evaluate the vibration fatigue failure of small-bore
piping systems. Carroll et al. [] used the digital image
correlationmethodtomeasurethestrainofstructuresduring
the fatigue crack initiation and growth and evaluate the con-
dition of cracks. Moilanen et al. [] proposed an image-
based method to monitor the planar strain and stress distri-
bution in heterogeneous and so materials.
3.4. Vibration Monitoring and Dynamic Characteristics Identi-
cation. e structural displacemen t can be acquired with a
high speed camera at a high sample rate which can satisfy the
need of structural vibration monitoring and dynamic charac-
teristics identication such as the natural frequency, modal
damping ratio, and modal shape. Chen et al. [] proposed
a digital photogrammetry method for measurement of the
ambient vibration response and identication of the mode
shape ratio of stay cables with multiple camcorders. Oh et
al. [] presented a vision-based system for estimation of
the dynamic characteristics of the structure by using dis-
placement time histories for a motion-capture system. Jurjo
et al. [] proposed a structural displacement measurement
method based on digital image processing techniques to
conduct the dynamic analysis of slender structures. Fukuda
et al. [] developed a vision-based system to monitor the
dynamic response of large-scale civil infrastructure which
was more cost-eective.
Kim [] proposed a multitemplate matching algorithm
to obtain the modal parameters of a cable from blurred
motion images. Park et al. [] applied a motion-capture
system to monitor the D displacement response of a struc-
ture in wind tunnel experiments and to identify the dynamic
prop erties of the test structure. Caetano et al. [] developed
a vision-based system to monitor the vibration of slender
structures. Jeon et al. [] presented a method to conduct
modal tests using a camera image which could measure the
vibration of ma ny points at the same time. Chung et al.
[] applied image processing technique to acquire nonlin-
ear characteristic parameters of mechanical and structural
systems. Ji and Chang [] presented a nontarget image-
based method to measure small cable dynamic responses
using an optical ow method. Kohut and Kurowski []
developed a vision-based method to realize the D mea-
surement of structural vibration displacemen ts and modal
characteristics by use of operational modal analysis algo-
rithms.
e vibration-based structural damage detection meth-
ods have obtained great advances in the area of structural
health monitoring. e vision-based dynamic monitoring
methods can be applied in the procedure of the vibration-
based structural damage detection to give a stable signal
input. Poudel et al. [] obtained the structural dynamic
displacement time series using high-resolution subpixel
edge identication based image processing method and
developed the mode shape dierence function to detect
structural damage. Patsias and Staszewski [] developed
a new damage detection method with the aid of wavelets
andmodalshapedatawhichwasmeasuredoptically.Liet
al. [] developed a digital image processing method to
measure the rivulet vibration of an inclined cable in wind
tunnel tests and evaluate the rivulet vibration characteris-
tics.
3.5. Crack Inspection and Characterization. With the aid of
advanced image processing technology, the structural surface
featurescanbeanalyzedfromtheimagessuchascracks
and spalling on the steel and concrete structures. Yeum
and Dyke [] proposed a vision-based visual inspection
technique through the automatic process and analysis of a
large number of captured images for detection of the cracks
near bolts on the bridges. Liu et al. [] proposed a method
for automated surface crack monitoring and assessment of
concretestructuresbasedonadaptivedigitalimageprocess-
ing. Halfawy and Hengmeechai [] embedded the vision-
based defection recognition system into the closed circuit
television (CCTV) system mounted in the sewer to inspect
its defections automatically. Adhikari et al. [] presented
an approach of automated condition assessment of concrete
bridges based on digital image analyses.
German et al. [] developed a column damage index
for quantitative assessment of the visible damage (cracks
and spalling) on the RC structural members enhanced by
machine vision techniques to realize rapid building inspec-
tion aer earthquake. Ho et al. [] developed an image-
based system with three cameras attached to a cable climb-
ing robot to detect surface damage of cables using image
processing and pattern recognition techniques. Valenc¸a et al.
[]presentedavisualmethodwhichrecognizedthecon-
crete health monitoring automatically including measuring
the d isplacement and stain, detecting damages, identifying
cracks, and restoration.
Gul et al. [] proposed an image-based monitoring
method for the detection of open gears of movable bridges
in lubrication level to assess the condition and ma ke main-
tenance decision. Sakagami [] presented a remote nonde-
structive evaluation technique using infrared thermography
to detect fatigue cracks and assess the structural integrity. Liu
et al. [] combined t he two-dimensional image processing
technology and three-dimensional reconstr uction method
to assess the crack characteristics of concrete structures
automatically to solve the hindrance in the practical imple-
mentation of traditional two-dimensional method. Wu et al.
[] developed a crack defragmentation technique based on
image processing techniques to improve the crack-detection
accurac y in the road assessment task.
3.6. Integration Technology. rough integrating with other
sensing approaches, the machine vision technology was
broadened to be used in more specic application categories.
Vaghe et al. [] developed a combined nondestructive
imaging technology on a bridge deck to yield both surface
and subsurface indicators of condition. Stabile et al. []