Showing papers on "Visual inspection published in 2022"
TL;DR: In this article , the authors proposed an efficient and effective machine vision system based on the state-of-the-art deep learning techniques and stacking ensemble methods to offer a non-destructive and cost-effective solution for automating the visual inspection of fruits' freshness and appearance.
27 citations
TL;DR: A framework to create a virtual visual inspection testbed using 3D synthetic environments that can enable end-to-end testing of autonomous inspection strategies and demonstrates the use of PBGMs as an effective testbed for the development and validation of strategies for autonomous vision-based inspections of civil infrastructure.
Abstract: Manual visual inspection of civil infrastructure is high-risk, subjective, and time-consuming. The success of deep learning and the proliferation of low-cost consumer robots has spurred rapid growth in research and application of autonomous inspections. The major components of autonomous inspection include data acquisition, data processing, and decision making, which are usually studied independently. However, for robust real-world applicability, these three aspects of the overall process need to be addressed concurrently with end-to-end testing, incorporating scenarios such as variations in structure type, color, damage level, camera distance, view angle, lighting, etc. Developing real-world datasets that span all these scenarios is nearly impossible. In this paper, we propose a framework to create a virtual visual inspection testbed using 3D synthetic environments that can enable end-to-end testing of autonomous inspection strategies. To populate the 3D synthetic environment with virtual damaged buildings, we propose the use of a non-linear finite element model to inform the realistic and automated visual rendering of different damage types, the damage state, and the material textures of what are termed herein physics-based graphics models (PBGMs). To demonstrate the benefits of the autonomous inspection testbed, three experiments are conducted with models of earthquake damaged reinforced concrete buildings. First, we implement the proposed framework to generate a new large-scale annotated benchmark dataset for post-earthquake inspections of buildings termed QuakeCity. Second, we demonstrate the improved performance of deep learning models trained using the QuakeCity dataset for inference on real data. Finally, a comparison of deep learning-based damage state estimation for different data acquisition strategies is carried out. The results demonstrate the use of PBGMs as an effective testbed for the development and validation of strategies for autonomous vision-based inspections of civil infrastructure.
23 citations
TL;DR: In this article , the authors present a systematic literature review of methods and techniques used in procedures for the visual inspection of aircraft and also show some insights into the automation of these processes with robotics and computer vision.
14 citations
TL;DR: In this article , a hybrid multistage system of stacked deep neural networks (SH-DNN) is proposed for defect detection in the semiconductor industry, which allows the localization of the finest structures within pixel size via a classical computer vision pipeline, while the classification process is realized by deep neural network.
Abstract: Abstract In the semiconductor industry, automated visual inspection aims to improve the detection and recognition of manufacturing defects by leveraging the power of artificial intelligence and computer vision systems, enabling manufacturers to profit from an increased yield and reduced manufacturing costs. Previous domain-specific contributions often utilized classical computer vision approaches, whereas more novel systems deploy deep learning based ones. However, a persistent problem in the domain stems from the recognition of very small defect patterns which are often in the size of only a few $$\mu $$ μ m and pixels within vast amounts of high-resolution imagery. While these defect patterns occur on the significantly larger wafer surface, classical machine and deep learning solutions have problems in dealing with the complexity of this challenge. This contribution introduces a novel hybrid multistage system of stacked deep neural networks (SH-DNN) which allows the localization of the finest structures within pixel size via a classical computer vision pipeline, while the classification process is realized by deep neural networks. The proposed system draws the focus over the level of detail from its structures to more task-relevant areas of interest. As the created test environment shows, our SH-DNN-based multistage system surpasses current approaches of learning-based automated visual inspection. The system reaches a performance (F1-score) of up to 99.5%, corresponding to a relative improvement of the system’s fault detection capabilities by 8.6-fold. Moreover, by specifically selecting models for the given manufacturing chain, runtime constraints are satisfied while improving the detection capabilities of currently deployed approaches.
13 citations
TL;DR: In this paper , a review of some techniques based on vision system for detection surface defects is presented and the purpose of this paper is to check the status of research progress in the field of vision inspection system, hardware system, software system, lighting method and its selection, image processing and control system have been discussed.
12 citations
TL;DR: In this paper , the authors provide an overview on current bridge inspection practices in the United States and conduct a systematic literature review on innovations in the field bridge inspections planning while investigating research gaps and future needs.
Abstract: Inspections are important to ensuring adequate safety and performance of a bridge throughout its service life. Bridge inspections are highly connected with maintenance decisions and can help in managing maintenance activities while maintaining a reliable bridge network. Routine inspections are the most common type of highway bridge inspections in the United States. The National Bridge Inspection Standards (NBIS) requires that, for almost all bridges, a routine inspection should be conducted at least every 24 months. However, limitations of current bridge inspection practices impact the quality of information provided about bridge conditions and the subsequent decisions made based on that information. Much research in the field of bridge inspection planning has been conducted to assist bridge inspectors in the inspection planning process and improving routine inspections. Accordingly, the goal of this study is to provide an overview on current bridge inspection practices in the United States and conduct a systematic literature review on innovations in the field bridge inspections planning while investigating research gaps and future needs. This paper provides a background on the history of bridge inspection in the United States, including current bridge inspection practices and their limitations and analyzes the connections between nondestructive evaluation techniques, deterioration models, and bridge inspection management. The primary emphasis of the paper is a thorough analysis of research proposing and investigating different methodologies for inspection planning and scheduling. Studies were analyzed and categorized into three main types of inspection planning approaches, based on: reliability, risk-analysis, and optimization approaches. The study revealed gaps and limitations in the current proposed techniques for inspection planning. The findings of this review will help in characterizing the current state of bridge inspection programs and future research needs to enhance inspection programs and reduce the gap between inspection practice and research.
12 citations
TL;DR: Wang et al. as discussed by the authors proposed a deep learning-based anomaly detection framework to detect lace defects, which consists of three stages, namely video pre-processing stage, pixel reconstruction stage and pixel classification stage.
12 citations
TL;DR: Wang et al. as mentioned in this paper proposed a deep learning-based anomaly detection framework to detect lace defects, which consists of three stages, namely video pre-processing stage, pixel reconstruction stage and pixel classification stage.
12 citations
TL;DR: Inspections are important to ensuring adequate safety and performance of a bridge throughout its service life as mentioned in this paper, and bridge inspections are highly connected with maintenance decisions and can he be he...
Abstract: Inspections are important to ensuring adequate safety and performance of a bridge throughout its service life. Bridge inspections are highly connected with maintenance decisions and can he...
12 citations
TL;DR: In this paper , a hybrid multistage system of stacked deep neural networks (SH-DNN) is proposed for defect detection in the semiconductor industry, which allows the localization of the finest structures within pixel size via a classical computer vision pipeline, while the classification process is realized by deep neural network.
Abstract: Abstract In the semiconductor industry, automated visual inspection aims to improve the detection and recognition of manufacturing defects by leveraging the power of artificial intelligence and computer vision systems, enabling manufacturers to profit from an increased yield and reduced manufacturing costs. Previous domain-specific contributions often utilized classical computer vision approaches, whereas more novel systems deploy deep learning based ones. However, a persistent problem in the domain stems from the recognition of very small defect patterns which are often in the size of only a few $$\mu $$ μ m and pixels within vast amounts of high-resolution imagery. While these defect patterns occur on the significantly larger wafer surface, classical machine and deep learning solutions have problems in dealing with the complexity of this challenge. This contribution introduces a novel hybrid multistage system of stacked deep neural networks (SH-DNN) which allows the localization of the finest structures within pixel size via a classical computer vision pipeline, while the classification process is realized by deep neural networks. The proposed system draws the focus over the level of detail from its structures to more task-relevant areas of interest. As the created test environment shows, our SH-DNN-based multistage system surpasses current approaches of learning-based automated visual inspection. The system reaches a performance (F1-score) of up to 99.5%, corresponding to a relative improvement of the system’s fault detection capabilities by 8.6-fold. Moreover, by specifically selecting models for the given manufacturing chain, runtime constraints are satisfied while improving the detection capabilities of currently deployed approaches.
11 citations
TL;DR: In this article , the performance of human operators with image processing, artificial intelligence software and 3D scanning for different types of inspection was statistically analyzed in terms of inspection accuracy, consistency and time.
Abstract: Background—Aircraft inspection is crucial for safe flight operations and is predominantly performed by human operators, who are unreliable, inconsistent, subjective, and prone to err. Thus, advanced technologies offer the potential to overcome those limitations and improve inspection quality. Method—This paper compares the performance of human operators with image processing, artificial intelligence software and 3D scanning for different types of inspection. The results were statistically analysed in terms of inspection accuracy, consistency and time. Additionally, other factors relevant to operations were assessed using a SWOT and weighted factor analysis. Results—The results show that operators’ performance in screen-based inspection tasks was superior to inspection software due to their strong cognitive abilities, decision-making capabilities, versatility and adaptability to changing conditions. In part-based inspection however, 3D scanning outperformed the operator while being significantly slower. Overall, the strength of technological systems lies in their consistency, availability and unbiasedness. Conclusions—The performance of inspection software should improve to be reliably used in blade inspection. While 3D scanning showed the best results, it is not always technically feasible (e.g., in a borescope inspection) nor economically viable. This work provides a list of evaluation criteria beyond solely inspection performance that could be considered when comparing different inspection systems.
TL;DR: A review of railway inspection robots can be found in this article , where a prototype of a typical track inspection robot is tested in a railway line and test results are presented. And the current limitations and future developments for railway robot are discussed.
TL;DR: The implementation and verification of such an approach together with the proposed methodology of the visual inspection process of car tires to obtain better classification results for six different defect classes can be considered as the main novel feature of the presented research.
Abstract: The article discusses the possibility of object detector usage in field of automated visual inspection for objects with specific parameters, specifically various types of defects occurring on the surface of a car tire. Due to the insufficient amount of input data, as well as the need to speed up the development process, the Transfer Learning principle was applied in a designed system. In this approach, the already pre-trained convolutional neural network AlexNet was used, subsequently modified in its last three layers, and again trained on a smaller sample of our own data. The detector used in the designed camera inspection system with the above architecture allowed us to achieve the accuracy and versatility needed to detect elements (defects) whose shape, dimensions and location change with each occurrence. The design of a test facility with the application of a 12-megapixel monochrome camera over the rotational table is briefly described, whose task is to ensure optimal conditions during the scanning process. The evaluation of the proposed control system with the quantification of the recognition capabilities in the individual defects is described at the end of the study. The implementation and verification of such an approach together with the proposed methodology of the visual inspection process of car tires to obtain better classification results for six different defect classes can be considered as the main novel feature of the presented research. Subsequent testing of the designed system on a selected batch of sample images (containing all six types of possible defect) proved the functionality of the entire system while the highest values of successful defect detection certainty were achieved from 85.15% to 99.34%.
TL;DR: In this article , a review of the state-of-the-art of vision-based surface defect inspection technology for steel products is presented, including hardware system, automated visionbased inspection method, existing problems and latest development.
Abstract: Steel plays an important role in industry, and the surface defect detection for steel products based on machine vision has been widely used during the last two decades. This paper attempts to review state-of-art of vision-based surface defect inspection technology of steel products by investigating about 170 publications. This review covers the overall aspects of vision-based surface defect inspection for steel products including hardware system, automated vision-based inspection method, existing problems and latest development. The types of steel product surface defects composition of visual inspection system are briefly described, and image acquisition system is introduced as well. The image processing algorithms for surface defect detection of steel products are reviewed, including image pre-processing, region of interest (ROI) detection, image segmentation for ROI, feature extraction and selection and defect classification. The important problems such as small sample and real time of steel surface defect detection are discussed. Finally, the challenge and development trend of steel surface defect detection are prospected.
TL;DR: In this article , a qualitative model is proposed to describe ASR-induced surface deterioration in concrete under distinct confinement configurations, and a normalized Cracking Index value is proposed that correlates strongly with the measured expansion level.
TL;DR: In this paper , a collection of eighty-six papers with image data and datasets pertaining to structural inspection for machine learning algorithms is presented, which provides an exceptionally rich starting point for researchers to use when beginning their next machine learning application in visual inspection.
TL;DR: In this paper , the authors presented trajectory planning for three-dimensional autonomous multi-UAV volume coverage and visual inspection of infrastructure based on the Heat Equation Driven Area Coverage (HEDAC) algorithm.
TL;DR: A detailed and methodical review of leather surface defect inspection with image analysis and machine learning is presented, which can be used as a source of guidelines for designing and developing new solutions in this field.
Abstract: Machine-vision-based surface defect inspection is one of the key technologies to realize intelligent manufacturing. This paper provides a systematic review on leather surface defect inspections based on machine vision. Leather products are regarded as the most traded products all over the world. Automatic detection, location, and recognition of leather surface defects are very important for the intelligent manufacturing of leather products, and are challenging but noteworthy tasks. This work investigates a large amount of literature related to leather surface defect inspection. In addition, we also investigate and evaluate the performance of some edge detectors and threshold detectors for leather defect detection, and the identification accuracy of the classical machine learning method SVM for leather surface defect identification. A detailed and methodical review of leather surface defect inspection with image analysis and machine learning is presented. Main challenges and future development trends are discussed for leather surface defect inspection, which can be used as a source of guidelines for designing and developing new solutions in this field.
TL;DR: In this paper , the monitoring approaches of sewage pipes in the form of operational monitoring, structural monitoring, and durability monitoring are outlined, and the capabilities and limitations of these technologies are discussed and summarized in tables.
TL;DR: In this article , a review of quality assessment in various production processes is presented, along with a summary of the four industrial revolutions that have occurred in manufacturing, highlighting the need to detect anomalies in assembly lines, detecting the features of the assembly line, the use of machine learning algorithms, the research challenges, the computing paradigms, and the using of state-of-the-art sensors in Industry 4.0.
Abstract: The quality-control process in manufacturing must ensure the product is free of defects and performs according to the customer’s expectations. Maintaining the quality of a firm’s products at the highest level is very important for keeping an edge over the competition. To maintain and enhance the quality of their products, manufacturers invest a lot of resources in quality control and quality assurance. During the assembly line, parts will arrive at a constant interval for assembly. The quality criteria must first be met before the parts are sent to the assembly line where the parts and subparts are assembled to get the final product. Once the product has been assembled, it is again inspected and tested before it is delivered to the customer. Because manufacturers are mostly focused on visual quality inspection, there can be bottlenecks before and after assembly. The manufacturer may suffer a loss if the assembly line is slowed down by this bottleneck. To improve quality, state-of-the-art sensors are being used to replace visual inspections and machine learning is used to help determine which part will fail. Using machine learning techniques, a review of quality assessment in various production processes is presented, along with a summary of the four industrial revolutions that have occurred in manufacturing, highlighting the need to detect anomalies in assembly lines, the need to detect the features of the assembly line, the use of machine learning algorithms in manufacturing, the research challenges, the computing paradigms, and the use of state-of-the-art sensors in Industry 4.0.
TL;DR: A remote visual inspection system to perform predictive maintenance on infrastructures such as bridges is developed based on the fusion between advanced robotic technologies and the Automated Visual Inspection that guarantees objective results, high-level of safety and low processing time of the results.
Abstract: Predictive maintenance on infrastructures is currently a hot topic. Its importance is proportional to the damages resulting from the collapse of the infrastructure. Bridges, dams and tunnels are placed on top on the scale of severity of potential damages due to the fact that they can cause loss of lives. Traditional inspection methods are not objective, tied to the inspector’s experience and require human presence on site. To overpass the limits of the current technologies and methods, the authors of this paper developed a unique new concept: a remote visual inspection system to perform predictive maintenance on infrastructures such as bridges. This is based on the fusion between advanced robotic technologies and the Automated Visual Inspection that guarantees objective results, high-level of safety and low processing time of the results.
TL;DR: In this paper , a UAV-based automatic damage detection and bridge condition evaluation were performed on existing bridges, from the process of preparing for inspection to the management of inspection data, the entire bridge inspection process was performed through field tests and the necessary element techniques for each stage were explained and the results were confirmed.
Abstract: As the number of old bridges increases, the number of bridges with structural defects is also increasing. Timely inspection and maintenance of bridges are required because structural degradation is accelerated after bridge damage. Recently, in the field of structural health monitoring, a bridge inspection using an unmanned aerial vehicle system (UAS) is receiving a lot of attention. In this paper, UAS-based automatic damage detection and bridge condition evaluation were performed on existing bridges. From the process of preparing for inspection to the management of inspection data, the entire bridge inspection process was performed through field tests. The necessary element techniques for each stage were explained and the results were confirmed. Finally, UAS-based results were compared with conventional human-based visual inspection results. As a result, it was confirmed that the UAS-based bridge inspection is faster and more objective than the existing technology. Therefore, it was confirmed that the automatic bridge inspection method based on unmanned aerial vehicles can be applied to the field as a promising technology.
TL;DR: In this article , an anomaly detection approach based on a convolutional autoencoder was used for defect detection during inspection to encounter the challenge of lacking and biased training data.
TL;DR: In this paper , a multi degree of freedom rail mounted inspection robot in GIS Substation and its adaptive pose adjustment method is presented, which can accurately collect most of the equipment instrument data with different pose, and provide data sample support for analyzing and judging whether the equipment has abnormal conditions.
TL;DR: Zhang et al. as mentioned in this paper proposed an unsupervised learning-based computationally inexpensive, efficient, and interpretable model UzADL for AVI to address the aforementioned problems.
TL;DR: A systematized methodology for bridge inspections in communication routes using images acquired by unmanned aerial vehicle (UAV) flights is proposed, which makes it possible to safely accurately identify diverse damages in structures without the need for a specialised engineer to go to the site.
Abstract: Many bridges and other structures worldwide present a lack of maintenance or a need for rehabilitation. The first step in the rehabilitation process is to perform a bridge inspection to know the bridge′s current state. Routine bridge inspections are usually based only on visual recognition. In this paper, a methodology for bridge inspections in communication routes using images acquired by unmanned aerial vehicle (UAV) flights is proposed. This provides access to the upper parts of the structure safely and without traffic disruptions. Then, a standardized and systematized novel image acquisition protocol is applied for data acquisition. Afterwards, the images are studied by civil engineers for damage identification and description. Then, specific structural inspection forms are completed using the acquired information. Recommendations about the need of new and more detailed inspections should be included at this stage when needed. The suggested methodology was tested on two railway bridges in France. Image acquisition of these structures was performed using an UAV for its ability to provide an expert assessment of the damage level. The main advantage of this method is that it makes it possible to safely accurately identify diverse damages in structures without the need for a specialised engineer to go to the site. Moreover, the videos can be watched by as many engineers as needed with no personal movement. The main objective of this work is to describe the systematized methodology for the development of bridge inspection tasks using a UAV system. According to this proposal, the in situ inspection by a specialised engineer is replaced by images and videos obtained from an UAV flight by a trained flight operator. To this aim, a systematized image/videos acquisition method is defined for the study of the morphology and typology of the structural elements of the inspected bridges. Additionally, specific inspection forms are proposed for every type of structural element. The recorded information will allow structural engineers to perform a postanalysis of the damage affecting the bridges and to evaluate the subsequent recommendations.
TL;DR: In this article , an automated precast concrete member crack detection system that enables automatic crack detection based on mobile and web servers using deep learning and imaging technologies was proposed to enable more accurate and efficient on-site PC member quality inspection.
Abstract: Recently, the construction industry has benefited from the increased application of smart construction led by the core technologies of the fourth industrial revolution, such as BIM, AI, modular construction, and AR/VR, which enhance productivity and work efficiency. In addition, the importance of “Off-Site Construction (OSC)”, a factory-based production method, is being highlighted as modular construction increases in the domestic construction market as a means of productivity enhancement. The problem with OSC construction is that the quality inspection of Precast Concrete (PC) members produced at the factory and brought to the construction site is not carried out accurately and systematically. Due to the shortage of quality inspection manpower, a lot of time and money is wasted on inspecting PC members on-site, compromising inspection efficiency and accuracy. In this study, the major inspection items to be checked during the quality inspection are classified based on the existing PC member quality inspection checklist and PC construction specifications. Based on the major inspection items, the items to which AI technology can be applied (for automatic quality inspection) were identified. Additionally, the research was conducted focusing on the detection of cracks, which are one of the major types of defects in PC members. However, accurate detection of cracks is difficult since the inspection mostly relies on a visual check coupled with subjective experience. To automate the detection of cracks for PC members, video images of cracks and non-cracks on the surface were collected and used for image training and recognition using Convolutional Neural Network (CNN) and object detection, one of the deep learning technologies commonly applied in the field of image object recognition. Detected cracks were classified according to set thresholds (crack width and length), and finally, an automated PC member crack detection system that enables automatic crack detection based on mobile and web servers using deep learning and imaging technologies was proposed. This study is expected to enable more accurate and efficient on-site PC member quality inspection. Through the smart PC member quality inspection system proposed in this study, the time required for each phase of the existing PC member quality inspection work was reduced. This led to a reduction of 13 min of total work time, thereby improving work efficiency and convenience. Since quality inspection information can be stored and managed in the system database, human errors can be reduced while managing the quality of OSC work systematically and accurately. It is expected that through optimizing and upgrading our proposed system, quality work for the precise construction of OSC projects can be ensured. At the same time, systematic and accurate quality management of OSC projects is achievable through inspection data. In addition, the smart quality inspection system is expected to establish a smart work environment that enables efficient and accurate quality inspection practices if applied to various construction activities other than the OSC projects.
TL;DR: In this article, automation-enabled facade visual inspection has become a prevailing trend in both traditional and automation-assisted visual inspection of facades in both commercial and residential buildings, respectively.
Abstract: Given that traditional facade visual inspection entails laborious, dangerous, and inefficient manual work, automation-enabled facade visual inspection has become a prevailing trend in both ...
TL;DR: In this article , the authors developed a framework to overcome conventional inspection regimes' limitations by deploying multiple non-destructive testing (NDT) technologies to carry out digital visual inspections of masonry railway bridges.
Abstract: Purpose The utilisation of emerging technologies for the inspection of bridges has remarkably increased. In particular, non-destructive testing (NDT) technologies are deemed a potential alternative for costly, labour-intensive, subjective and unsafe conventional bridge inspection regimes. This paper aims to develop a framework to overcome conventional inspection regimes' limitations by deploying multiple NDT technologies to carry out digital visual inspections of masonry railway bridges. Design/methodology/approach This research adopts an exploratory case study approach, and the empirical data is collected through exploratory workshops, interviews and document reviews. The framework is implemented and refined in five masonry bridges as part of the UK railway infrastructure. Four NDT technologies, namely, terrestrial laser scanner, infrared thermography, 360-degree imaging and unmanned aerial vehicles, are used in this study. Findings A digitally enhanced visual inspection framework is developed by using complementary optical methods. Compared to the conventional inspection regimes, the new approach requires fewer subjective interpretations due to the additional qualitative and quantitative analysis. Also, it is safer and needs fewer operators on site, as the actual inspection can be carried out remotely. Originality/value This research is a step towards digitalising the inspection of bridges, and it is of particular interest to transport agencies and bridge inspectors and can potentially result in revolutionising the bridge inspection regimes and guidelines.