Road condition monitoring (RCM) has been a demanding strategic research area in maintaining a large network of transport infrastructures. With advancements in computer vision and data mining techniques along with high computing resources, several innovative pavement distress evaluation systems have been developed in recent years. The majority of these technologies employ next-generation distributed sensors and vision-based artificial intelligence (AI) methodologies to evaluate, classify and localize pavement distresses using the measured data. This paper presents an exhaustive and systematic literature review of these technologies in RCM that have been published from 2017–2022 by utilizing next-generation sensors, including contact and noncontact measurements. The various methodologies and innovative contributions of the existing literature reviewed in this paper, together with their limitations, promise a futuristic insight for researchers and transport infrastructure owners. The decisive role played by smart sensors and data acquisition platforms, such as smartphones, drones, vehicles integrated with non-intrusive sensors, such as RGB, and thermal cameras, lasers and GPR sensors in the performance of the system are also highlighted. In addition to sensing, a discussion on the prevalent challenges in the development of AI technologies as well as potential areas for further exploration paves the way for an all-inclusive and well-directed futuristic research on RCM.

/pdf/road-condition-monitoring-using-smart-sensing-and-artificial-3uf0fcql.pdf

Road Condition Monitoring Using Smart Sensing and Artificial Intelligence: A Review

In modeling forest stand growth and yield, crown width, a measure for stand density, is among the parameters that allows for estimating stand timber volumes. However, accurately measuring tree crown size in the field, in particular for mature trees, is challenging. This study demonstrated a novel method of applying machine learning algorithms to aerial imagery acquired by an unmanned aerial vehicle (UAV) to identify tree crowns and their widths in two loblolly pine plantations in eastern Texas, USA. An ortho mosaic image derived from UAV-captured aerial photos was acquired for each plantation (a young stand before canopy closure, a mature stand with a closed canopy). For each site, the images were split into two subsets: one for training and one for validation purposes. Three widely used object detection methods in deep learning, the Faster region-based convolutional neural network (Faster R-CNN), You Only Look Once version 3 (YOLOv3), and single shot detection (SSD), were applied to the training data, respectively. Each was used to train the model for performing crown recognition and crown extraction. Each model output was evaluated using an independent test data set. All three models were successful in detecting tree crowns with an accuracy greater than 93%, except the Faster R-CNN model that failed on the mature site. On the young site, the SSD model performed the best for crown extraction with a coefficient of determination (R2) of 0.92, followed by Faster R-CNN (0.88) and YOLOv3 (0.62). As to the mature site, the SSD model achieved a R2 as high as 0.94, follow by YOLOv3 (0.69). These deep leaning algorithms, in particular the SSD model, proved to be successfully in identifying tree crowns and estimating crown widths with satisfactory accuracy. For the purpose of forest inventory on loblolly pine plantations, using UAV-captured imagery paired with the SSD object detention application is a cost-effective alternative to traditional ground measurement.

Measuring loblolly pine crowns with drone imagery through deep learning

Concrete is one of the primary and most commonly used materials for the construction of buildings, roads, bridges, and dams. But it may lose its strength with age, moisture content, or due to other factors. As a result, little damage (crack and spall) to these structures may lead to sudden collapse or breakage, which in turn can decimate many lives with economic losses. Hence, to ensure their strength with the proper load, accurate screening of concrete surface damage is necessary for maintenance engineers to understand and evaluate the severity of the damage. Few preceding vision-based studies have proposed object detection and semantic segmentation approaches to carry out damage detection; however, the developed models could not segment different objects of the same class. The semantic segmentation approach separates regions that contain only objects of the same class. Concrete surface damage, on the other hand, may contain multiple objects, such as the Horizontal Crack, Vertical Crack, Diagonal Crack, Branch Crack, and Spall, that need to be segmented separately so that specific measures can be taken. This study utilizes Mask Region-Based Convolutional Neural Network (Mask R-CNN) to manage concrete damage images and employ it to detect and segment defects in civil infrastructure with multiple objects. As we understand, this is the first research involving a Structural Health Monitoring (SHM) application for damage detection using instance segmentation. Our proposed model has been trained and tested on a dataset containing 800 images (480*480 pixels) of different types of crack and spall are collected from distinct structures of CSIR-CEERI, Pilani, campus. To test and validate the generosity, an additional 96 damage images are downloaded using Google. The pre-processed and annotated images are used to train and validate the Mask R-CNN Classifier. Our experimental results show that damage can be classified efficiently with 95.13% accuracy on a custom dataset and 96.87% on randomly picked images.

/pdf/automatic-multiclass-instance-segmentation-of-concrete-3ait174mo1.pdf

Automatic Multiclass Instance Segmentation of Concrete Damage Using Deep Learning Model

Mobile Laser Scanning (MLS) system can provide high-density and accurate 3D point clouds that enable rapid pavement crack detection for road maintenance tasks. Supervised learning-based algorithms have been proved pretty effective for handling such a large amount of inhomogeneous and unstructured point clouds. However, these algorithms often rely on a lot of annotated data, which is labor-intensive and time-consuming. This paper presents a semi-supervised point-level approach to overcome this challenge. We propose a graph-widen module to construct a reasonable graph structure for point clouds, increasing the detection performance of graph convolutional networks (GCN). The constructed graph characterizes the local features from a small amount of annotated data, avoiding information loss and dramatically reduces the dependence on annotated data. The MLS point clouds acquired by a commercial RIEGL VMX-450 system are used in this study. The experimental results demonstrate that our method outperforms the state-of-the-art point-level methods in terms of recall, F1 score, and efficiency while achieving comparable accuracy.

GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds

The automatic recognition of cracks is an essential requirement for the cost‐efficient maintenance of concrete structures, such as bridges, buildings, and roads. It should allow the localization and the determination of the crack type and the evaluation of the crack severity by providing information on the shape, orientation, and crack area and width. The first step in this direction is the automatized segmentation of cracks. This paper provides a concrete crack data set (370 images) and proposes two solutions that achieve the best results on two different crack data sets. Our first solution concerns the segmentation architecture. We provide an encoder–decoder‐based network with a particular interconnection of layers between the encoder and decoder parts that outperforms several other methods. In addition, this network is enhanced by squeeze‐and‐excitation blocks equipped with a modified sigmoid activation function. We introduce a stretch coefficient into the sigmoid function and declare it a trainable parameter, allowing more differentiated calibration of the feature map during network training. Our second solution concerns kernel initialization by transfer learning (TL). We propose the Copy‐Edit‐Paste Transfer Learning (CEP TL). By copying, geometric editing, and pasting crack masks onto new concrete background images, we generate thousands of semisynthetic images used to pretrain the network. This CEP TL method increases model performance with significant differences. For data set A (ours), we achieve F1‐scores 76.06 ± 0.06% without CEP TL and 92.32 ± 0.82% with CEP TL. For data set B (DeepCrack data set), we achieve F1‐scores 88.56 ± 0.01% without CEP TL and 90.59 ± 0.80% with CEP TL.

https://onlinelibrary.wiley.com/doi/pdfdirect/10.1111/mice.12844

A sigmoid‐optimized encoder–decoder network for crack segmentation with copy‐edit‐paste transfer learning

In this study, multispectral Light Detection and Ranging (LiDAR) data were utilized to improve delineation of individual tree crowns (ITC) as an important step in individual tree analysis. A framework to integrate spectral and height information for ITC delineation was proposed, and the multi-scale algorithm for treetop detection developed in one of our previous studies was improved. In addition, an advanced region-based segmentation method that used detected treetops as seeds was proposed for segmentation of individual crowns based on their spectral, contextual, and height information. The proposed methods were validated with data acquired using Teledyne Optech's Titan LiDAR sensor. The sensor was operated at three wavelengths (1550 nm, 1064 nm, and 532 nm) within a study area located in the city of Toronto, ON, Canada. The proposed method achieved 80% accuracy, compared with manual delineation of crowns, considering both matched and partially matched crowns, which was 12% higher than that obtained by the earlier marker-controlled watershed (MCW) segmentation technique. Furthermore, the results showed that the integration of spectral and height information improved ITC delineation using either the proposed framework or MCW segmentation, compared with using either spectral or height information individually.

https://www.mdpi.com/1424-8220/19/24/5421/pdf

Individual Tree Crown Delineation Using Multispectral LiDAR Data.

With the advance of deep learning networks, their applications in the assessment of pavement conditions are gaining more attention. A convolutional neural network (CNN) is the most commonly used ne...

/pdf/exploitation-of-deep-learning-in-the-automatic-detection-of-1nu06iaw0h.pdf

Faizaan Naveed

Papers

Individual Tree Crown Delineation Using Multispectral LiDAR Data.

Exploitation of deep learning in the automatic detection of cracks on paved roads