Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks

TLDR: This article proposes a vision‐based method using a deep architecture of convolutional neural networks (CNNs) for detecting concrete cracks without calculating the defect features, and shows quite better performances and can indeed find concrete cracks in realistic situations.

Abstract: A number of image processing techniques IPTs have been implemented for detecting civil infrastructure defects to partially replace human-conducted onsite inspections. These IPTs are primarily used to manipulate images to extract defect features, such as cracks in concrete and steel surfaces. However, the extensively varying real-world situations e.g., lighting and shadow changes can lead to challenges to the wide adoption of IPTs. To overcome these challenges, this article proposes a vision-based method using a deep architecture of convolutional neural networks CNNs for detecting concrete cracks without calculating the defect features. As CNNs are capable of learning image features automatically, the proposed method works without the conjugation of IPTs for extracting features. The designed CNN is trained on 40 K images of 256 × 256 pixel resolutions and, consequently, records with about 98% accuracy. The trained CNN is combined with a sliding window technique to scan any image size larger than 256 × 256 pixel resolutions. The robustness and adaptability of the proposed approach are tested on 55 images of 5,888 × 3,584 pixel resolutions taken from a different structure which is not used for training and validation processes under various conditions e.g., strong light spot, shadows, and very thin cracks. Comparative studies are conducted to examine the performance of the proposed CNN using traditional Canny and Sobel edge detection methods. The results show that the proposed method shows quite better performances and can indeed find concrete cracks in realistic situations.

Figures

Figure A.1: CNN without shortcut

Figure 4.1: Object localization with sliding-window

Figure 4.4: Generic image data

Figure 2.10: Pooling operation with 1D array

Figure 2.16: Binary step function

Figure 4.6: Schematic diagram of SDDNet architecture [Choi and Cha (2020)]

Full text

Deep Learning Implemented Structural Defect
Detection on Digital Images
by
Wooram Choi
A Thesis submitted to the Faculty of Graduate Studies of
The University of Manitoba
In partial fulfilment of the requirements of the degree of
DOCTOR OF PHILOSOPHY
Civil Engineering
University of Manitoba
Winnipeg, Manitoba, Canada
Copyright © 2020 by Wooram Choi

Abstract
Periodical inspection is the dominant form of structural health monitoring (SHM). However, civil
engineering societies in North America have expressed the common consent that the current inspection
practice is not sufficient to ensure infrastructure safety. Moreover, the increasing number of aged
infrastructures will require an advanced form of inspection systems.
The processes of vision-based methods for identifying damage using image processing algorithms
(IPAs) are similar to human inspections because both use visual information. The outcomes of vision-
based methods are much more intuitive than systems with traditional contact sensors. Accordingly,
researchers have proposed a variety of different methods. For example, early research adopted IPAs
directly into damage detection problems. The results from IPAs are intuitive but require manual decision-
making processes. Further attempts have been made to establish automated decision-making systems
using machine learning algorithms (MLAs). However, real-life applications are rare. The unavailability is
mainly rooted in the fact that IPAs were developed and tested in controlled circumstances, while real-
world situations often cannot be controlled. Mobile units with cameras have attracted great attention in
the SHM discipline. This type of inspection can improve accessibility to infrastructures but still lacks
automated damage detection. Even if IPAs and MLAs are integrated, the combined system (mobile units,
IPAs, and MLAs) will likely be invalid in practice because this system inherits the limitations of IPAs. To
overcome these challenges, IPAs should be replaced by advanced computer vision techniques.
In this thesis, deep learning (DL) is considered the key for surpassing the current state of vision-
based approaches. Deep learning models are capable of learning features from raw data. Instead of
manually developing IPAs, feeding raw data that were collected in uncontrolled environments and leading
a machine to learn the features of the data may be a better approach. A deep learning model for classifying

images for damage detection into binary classes is introduced, and its performance is compared with IPAs.
The results of the classification DL model demonstrate the possibility of replacing IPAs with DL models.
A segmentation DL model is also introduced that demonstrates faster, more robust, more flexible, and
more intuitive than competitive methods.

CO-AUTHORSHIP
This thesis has been prepared in accordance with the regulation of the integrated-article format stipulated
by the Faculty of Graduate Studies at the University of Manitoba. Substantial parts of this thesis were
submitted for publication to peer-reviewed technical journals as follows:
Choi, W., & Cha, Y.-J. (2020). SDDNet: Real-Time Crack Segmentation. IEEE Transactions on
Industrial Electronics, 67(9), 8016–8025. DOI: 10.1109/TIE.2019.2945265, [Chapter 4]. I initiated this
project by proposing the plan for researching this topic in my thesis proposal defense. I contributed to
creating the dataset, designing and developing the deep learning model, conducting the comparative study,
visualizing the results, writing the draft, and responding to the reviewers’ comments.
Cha, Y.-J., Choi, W. & Büyüköztürk, O. (2017), Deep Learning-Based Crack Damage Detection Using
Convolutional Neural Networks, Computer-Aided Civil and Infrastructure Engineering, 32(5), 361-378,
DOI: 10.1111/mice.12263, [Chapter 3]. Dr. Cha initiated the project by providing an idea of damage
detection using deep learning. I contributed to building the dataset, designing and developing the deep
learning model, integrating the sliding-window along with the deep learning model, writing the draft
guided by Dr. Cha and Dr. Büyüköztürk, responding to the reviewers’ comments guided by Dr. Cha and
Dr. Büyüköztürk.

Acknowledgments
I express gratitude to all my committee members, Dr. Young-Jin Cha, Dr. Dimos Polyzois, Dr. Yang
Wang, and Dr. David Lattanzi for guiding me in my program. I am also grateful to Ms. Julia Osso and Dr.
Dagmar Svecova for consulting and helping me in a tough situation. I thank all my colleagues for being
sincere friends.
I acknowledge the support from the Natural Sciences and Engineering Research Council of Canada
(NSERC) via the Discovery grant (Common Personal Identifier: 1262624) and Engage grant (Application
No.: 533690-18), as well as the Canada Foundation for Innovation via the John R. Evans Leaders Fund
(Project 37394).

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Deep learning implemented structural defect detection on digital images" ?

Wang et al. this paper proposed a deep learning model for structural health monitoring ( SHM ), which is capable of learning features from raw data and demonstrates faster, more robust, more flexible and more intuitive than competitive methods. 

Q2. What is the effect of the gradients on the input of a layer?

Even if a non-saturating activation function, such as the ReLU, is applied, the gradients will remain vanished if the input of a layer has negative values. 

Q3. What was the last operation involved in the deep learning model?

The deep learning model used in this study was a classification model, and the last operation involved an FC layer that restricted the size of the input images to 256 × 256 pixels. 

Q4. What is the reason for the vanishing gradient problem in a DL model?

In addition, small gradients cause a more serious issue in a model with a deep architecture because small gradients are multiplied by the chain rule. 

Q5. How many images were required to train a CNN classifier?

According to the findings of this parametric study, at least 10K images are required to obtain a reasonable CNN classifier with a validation accuracy of 0.97 in the concrete crack detection problem. 

Q6. What is the prominent countermeasure to the augmentation of weights?

One of the prominent countermeasures is data augmentation, which is discussed in a previous subchapter (refer to Chapter 2.2.1) as a part of input processing. 

Q7. What are the main targets to be optimized in DL models?

Note that weights and biases are referred to as parameters (i.e., learnable parameters; refer to Chapter 2.1); they are the main targets to be optimized in DL models. 

Q8. What is the method for denoising an image?

A number of denoising techniques are available, but the edge-aware denoising6 method proposed by Gastal and Oliveira (2012) was chosen to preserve the features of the cracks (i.e., edges) from the original image. 

Q9. What is the weighted sum of the input and output of the l-th layer?

The 𝑛𝜙 number of weights at the l-th layer (𝜙1 (𝑙) , 𝜙2 (𝑙) , …, 𝜙𝑛𝜙 (𝑙) ) performs the weighted sum to the input of the layer, in which the dimensions (i.e., width, height, or length) of the weights is usually smaller than that of the layer’s input. 

Q10. How many images were intentionally taken from the camera?

The distances to concrete surfaces from the camera were approximately 1.0 to 1.5 m, but a few images were intentionally taken within 0.1 m for testing. 

Q11. How long did it take to train the model?

The total training duration was approximately 90 minutes on the GPU (refer to Chapter 3.6), but it may require several hours to train the model on a CPU. 

Q12. What are the parameters that can be controlled by an algorithm?

The behavior of an optimization algorithm can be controlled by several parameters, which are independently defined as hyperparameters.