Mixed Reality-Based Dataset Generation for Learning-Based Scan-to-BIM
10 Jan 2021-pp 389-403
TL;DR: In this paper, an interactive user interface using a game engine within a mixed reality environment is presented for fusing as-is spatial information with the AR/VR based information in Unity 3D.
Abstract: Generating as-is 3D Models is constantly explored for various construction management applications. The industry has been dependent on either manual or semi-automated workflows for the Scan-to-BIM process, which is laborious as well as time taking. Recently machine learning has opened avenues to recognize geometrical elements from point clouds but has not been much used because of the insufficient labeled dataset. This study aims to set up a semi-automated workflow to create labeled data sets which can be used to train ML algorithms for element identification purpose. The study proposes an interactive user interface using a gaming engine within a mixed reality environment. A workflow for fusing as-is spatial information with the AR/VR based information is presented in Unity 3D. A user-friendly UI is then developed and integrated with the VR environment to help the user to choose the category of the component by visualization. This results in the generation of an accurate as-is 3D Model, which does not require much computation or time. The intention is to propose a smooth workflow to generate datasets for learning-based methodologies in a streamlined Scan-to-BIM Process. However, this process requires user domain knowledge and input. The dataset can be continuously increased and improved to get automated results later.
Citations
More filters
TL;DR: In this article , the authors developed an integrated process framework for Computer-Vision-Based Construction Progress Monitoring (CV-CPM), which comprises: data acquisition and 3D-reconstruction, as-built modelling, and progress assessment.
19 citations
TL;DR: In this article , a universal workflow to synthesize point clouds containing both geometry and color information by utilizing the IFC model or its 3D geometry model to automatically generate annotated point clouds for semantic segmentation in deep learning is described.
4 citations
TL;DR: In this paper , the relationship between data quality and model quality in scan-to-BIM process was investigated using a case study on mechanical, electrical and plumbing (MEP) scenes, where two MEP scenes were scanned with different scanning settings (angular resolutions and scanning locations).
3 citations
24 Jun 2022
TL;DR: In this paper , the authors identified and defined twelve key factors affecting data acquisition technology and eight factors affecting sensor mounting, and a questionnaire survey was designed, and responses from professionals were used to evaluate the Relative Importance Index (RII) for the individual factors for these technologies and methods.
Abstract: The accuracy of computer vision-based progress monitoring of construction projects depends on the quality of data acquired. The data acquisition can be conducted through different vision-based sensors combined with several options for sensor mounting. Several factors affect this combination and considering these factors in selecting the acquisition technology and sensor mounting combination is critical for acquiring accurate vision-based data for the project. Currently, their definition and impact of these factors on the selection of these technologies are both subjective, and there are no formal studies to evaluate the impact. Hence, in this study, we first identify and define twelve key factors affecting data acquisition technology and eight factors affecting sensor mounting. Next, a questionnaire survey was designed, and responses from professionals were used to evaluate the Relative Importance Index (RII) for the individual factors for these technologies and methods. The obtained ratings were compared to the author's initial assessment, and the cause for a few variations obtained was justified. This study provides a clear assessment of these factors and forms a basis for selection based on the factors involved with the project requirements.
TL;DR: In this paper , the authors identify several progress monitoring methods and classifies them based on the technology they use to support progress monitoring, and then they are evaluated by highlighting their advantages and limitations.
Abstract: -Progress monitoring is one of the essential tasks while executing a construction project. Effective monitoring will lead to an accurate and timely analysis of the project’s progress which is required to make vital decisions for project control. On the other hand, inefficient and delayed updates regarding the project’s progress, which is estimated by comparing the as-built status with the as-planned status, will lead to time and cost overruns. Automated progress monitoring techniques are preferred over the conventional manual data entry method as the latter is time-consuming and complex, especially if the project scope is vast. Numerous tools and technologies are being used for progress monitoring of construction projects. Therefore, it is necessary to systematically classify and evaluate them based on their advantages and limitations for successful and appropriate implementation. Hence, this article identifies several progress monitoring methods and classifies them based on the technology they use to support progress monitoring. Then they are evaluated by highlighting their advantages and limitations. Several qualitative and quantitative factors affecting the selection of these technologies for implementation have also been identified. In future, a framework for objectively identifying the project-specific technology will be developed.
References
More filters
Proceedings Article•
07 Jun 2017TL;DR: PointNet++ as discussed by the authors applies PointNet recursively on a nested partitioning of the input point set to learn local features with increasing contextual scales, and proposes novel set learning layers to adaptively combine features from multiple scales.
Abstract: Few prior works study deep learning on point sets. PointNet is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.
3,316 citations
21 Jul 2017
TL;DR: This work introduces ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations, and shows that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks.
Abstract: A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets. Unfortunately, in the context of RGB-D scene understanding, very little data is available – current datasets cover a small range of scene views and have limited semantic annotations. To address this issue, we introduce ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations. To collect this data, we designed an easy-to-use and scalable RGB-D capture system that includes automated surface reconstruction and crowdsourced semantic annotation. We show that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks, including 3D object classification, semantic voxel labeling, and CAD model retrieval.
2,305 citations
27 Jun 2016
TL;DR: This paper argues that identification of structural elements in indoor spaces is essentially a detection problem, rather than segmentation which is commonly used, and proposes a method for semantic parsing the 3D point cloud of an entire building using a hierarchical approach.
Abstract: In this paper, we propose a method for semantic parsing the 3D point cloud of an entire building using a hierarchical approach: first, the raw data is parsed into semantically meaningful spaces (e.g. rooms, etc) that are aligned into a canonical reference coordinate system. Second, the spaces are parsed into their structural and building elements (e.g. walls, columns, etc). Performing these with a strong notation of global 3D space is the backbone of our method. The alignment in the first step injects strong 3D priors from the canonical coordinate system into the second step for discovering elements. This allows diverse challenging scenarios as man-made indoor spaces often show recurrent geometric patterns while the appearance features can change drastically. We also argue that identification of structural elements in indoor spaces is essentially a detection problem, rather than segmentation which is commonly used. We evaluated our method on a new dataset of several buildings with a covered area of over 6, 000m2 and over 215 million points, demonstrating robust results readily useful for practical applications.
1,320 citations
Posted Content•
TL;DR: The ScanNet dataset as discussed by the authors contains 2.5M RGB-D views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations.
Abstract: A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets. Unfortunately, in the context of RGB-D scene understanding, very little data is available -- current datasets cover a small range of scene views and have limited semantic annotations. To address this issue, we introduce ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations. To collect this data, we designed an easy-to-use and scalable RGB-D capture system that includes automated surface reconstruction and crowdsourced semantic annotation. We show that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks, including 3D object classification, semantic voxel labeling, and CAD model retrieval. The dataset is freely available at this http URL.
978 citations
TL;DR: Recent developments from research efforts inautomated acquisition of as-built point-cloud models from unordered daily site photo collections and geo-registration of site images, and automated generation of four-dimensiona models are reported.
Abstract: The significant advancement in digital imaging and widespread popularity of digital cameras for capturing a comprehensive visual record of construction performance in the architecture, engineering, construction, and facility management (AEC/FM) industries have triggered an extensive growth in the rate of site photography, allowing hundreds of images to be stored for a project on a daily basis. Meanwhile, collaborative AEC technologies centering around building information models (BIMs) are becoming widely applied to support various architectural, structural, and preconstruction decision-making tasks. These models, if integrated with the as-built perspective of a construction, have great potential to extensively add value during the construction phase of a project. This paper reports recent developments from research efforts in (1) automated acquisition of as-built point-cloud models from unordered daily site photo collections and geo-registration of site images, (2) automated generation of four-dimensiona...
177 citations