Author
Hyojoon Bae
Other affiliations: Apple Inc.
Bio: Hyojoon Bae is an academic researcher from Virginia Tech. The author has contributed to research in topics: Augmented reality & Mobile device. The author has an hindex of 6, co-authored 12 publications receiving 202 citations. Previous affiliations of Hyojoon Bae include Apple Inc..
Papers
More filters
••
TL;DR: A new vision-based mobile augmented reality system that allows field personnel to query and access 3D cyber-information on-site by using photographs taken from standard mobile devices and the localization speed and empirical accuracy of the system provides the ability to use the system on real-world construction sites.
Abstract: Many context-aware techniques have been proposed to deliver cyber-information, such as project specifications or drawings, to on-site users by intelligently interpreting their environment. However, these techniques primarily rely on RF-based location tracking technologies (e.g., GPS or WLAN), which typically do not provide sufficient precision in congested construction sites or require additional hardware and custom mobile devices. This paper presents a new vision-based mobile augmented reality system that allows field personnel to query and access 3D cyber-information on-site by using photographs taken from standard mobile devices. The system does not require any location tracking modules, external hardware attachments, and/or optical fiducial markers for localizing a user’s position. Rather, the user’s location and orientation are purely derived by comparing images from the user’s mobile device to a 3D point cloud model generated from a set of pre-collected site photographs. The experimental results show that 1) the underlying 3D reconstruction module of the system generates complete 3D point cloud models of target scene, and is up to 35 times faster than other state-of-the-art Structure-from-Motion (SfM) algorithms, 2) the localization time takes at most few seconds in actual construction site. The localization speed and empirical accuracy of the system provides the ability to use the system on real-world construction sites. Using an actual construction case study, the perceived benefits and limitations of the proposed method for on-site context-aware applications are discussed in detail.
146 citations
••
TL;DR: A new vision-based context-aware approach for mobile augmented reality that allows users to query and access semantically-rich 3D cyber-information related to real-world physical objects and see it precisely overlaid on top of imagery of the associated physical objects.
Abstract: A key problem in mobile computing is providing people access to cyber-information associated with their surrounding physical objects. Mobile augmented reality is one of the emerging techniques that addresses this problem by allowing users to see the cyber-information associated with real-world physical objects by overlaying that cyber-information on the physical objects’ imagery. This paper presents a new vision-based context-aware approach for mobile augmented reality that allows users to query and access semantically-rich 3D cyber-information related to real-world physical objects and see it precisely overlaid on top of imagery of the associated physical objects. The approach does not require any RF-based location tracking modules, external hardware attachments on the mobile devices, and/or optical/fiducial markers for localizing a user’s position. Rather, the user’s 3D location and orientation are automatically and purely derived by comparing images from the user’s mobile device to a 3D point cloud model generated from a set of pre-collected photographs. Our approach supports content authoring where collaboration on editing the content stored in the 3D cloud is possible and content added by one user can be immediately accessible by others. In addition, a key challenge of scalability for mobile augmented reality is addressed in this paper. In general, mobile augmented reality is required to work regardless of users’ location and environment, in terms of physical scale, such as size of objects, and in terms of cyber-information scale, such as total number of cyber-information entities associated with physical objects. However, many existing approaches for mobile augmented reality have mainly tested their approaches on limited real-world use-cases and have challenges in scaling their approaches. By designing a multi-model based direct 2D-to-3D matching algorithms for localization, as well as applying a caching scheme, the proposed research consistently supports near real-time localization and information association regardless of users’ location, size of physical objects, and number of cyber-physical information items. Empirical results presented in the paper show that the approach can provide millimeter-level augmented reality across several hundred or thousand objects without the need for additional non-imagery sensor inputs.
31 citations
••
TL;DR: This paper presents a scalable markerless and infrastructure-independent augmented reality method that provides content authoring capability in three dimensions necessary for reciprocal application of augmented reality for field reporting and asynchronous personnel-to-personnel communications.
Abstract: Recent research efforts to improve information management on construction sites have focused on devising mobile augmented reality methods to provide on-demand access to project information and facilitate field reporting. To track position and orientation of the users, prior research focused on radio frequency based location tracking methods, fiduciary markers, or parallel visual tracking and mapping techniques. These methods have potential in providing fast information access to field personnel, yet their application heavily depends on preinstalled infrastructure, may suffer from accumulated drift errors, and in the case of visual methods do not typically scale well. Furthermore, these methods do not provide content authoring capability in three dimensions, which is necessary for reciprocal application of augmented reality for field reporting and asynchronous personnel-to-personnel communications. To overcome these limitations, this paper presents a scalable markerless and infrastructure-independe...
30 citations
••
06 Aug 2013TL;DR: The experimental results show that the underlying 3D reconstruction module of the system generates more complete 3D point cloud models, and faster than other state-of-the-art Structure-from-Motion(SfM) algorithms.
Abstract: This paper presents a new context-aware mobile augmented reality system that provides rapid and robust on-site access to construction project information such as drawings, specifications, schedules, and budgets. The mobile augmented reality system does not need any RF-based location tracking (e.g., GPS or WLAN) or optical fiducial markers for tracking a user’s position. Rather, the user’s location and orientation are automatically and purely derived by comparing photographs from the user’s phone to a 3D point cloud model created from a set of site photographs. After generating a 3D point cloud model of construction site, field personnel can use mobile devices to take pictures of building elements and be presented on-site with a detailed list of project information related to the visible construction elements in an augmented reality format. The experimental results show that (1) the underlying 3D reconstruction module of the system generates more complete 3D point cloud models, and faster than other state-of-the-art Structure-from-Motion(SfM) algorithms; (2) the localization method is an order of magnitude more accurate than the state-of-the-art solutions, and can provide acceptable tolerances of most on-site engineering applications. Using an actual construction case study, the perceived benefits and limitations of the proposed method for on-site context-aware applications are discussed in detail.
14 citations
••
01 Dec 2015
TL;DR: A new SfM framework that is optimized for mobile augmented reality and rapidly generates a complete 3D point cloud of a target scene up to 28 times faster than prior approaches is presented.
Abstract: Mobile augmented reality is an emerging technique which allows users to use a mobile device's camera to capture real-world imagery and view real-world physical objects and their associated cyber-information overlaid on top of imagery of them. One key challenge for mobile augmented reality is the fast and precisely localization of a user in order to determine what is visible in their camera view. Recent advances in Structure-from-Motion (SfM) enable the creation of 3D point clouds of physical objects from an unordered set of photographs taken by commodity digital cameras. The generated 3D point cloud can be used to identify the location and orientation of the camera relative to the point cloud. While this SfM-based approach provides complete pixel-accurate camera pose estimation in 3D without relying on external GPS or geomagnetic sensors, the preparation of initial 3D point cloud typically takes from hours to a day, making it difficult to use in mobile augmented reality applications. Furthermore, creating 3D cyber-information and associating it with the 3D point cloud is also a challenge of using SfM-based approach for mobile augmented reality. To overcome these challenges in 3D point cloud creation and cyber-physical content authoring, the paper presents a new SfM framework that is optimized for mobile augmented reality and rapidly generates a complete 3D point cloud of a target scene up to 28 times faster than prior approaches. Key improvements in the proposed SfM framework stem from the use of (1) state-of-the-art binary feature descriptors, (2) new filtering approach for accurate 3D modeling, (3) optimized point cloud structure for augmented reality, and (4) hardware/software parallelism. The paper also provides a new image-based 3D content authoring method designed specifically for the limited user interfaces of mobile devices. The proposed content authoring method generates 3D cyber-information from a single 2D image and automatically associates it with the 3D point cloud.
14 citations
Cited by
More filters
••
TL;DR: A generic taxonomy consisting of VR/AR technology characteristics, application domains, safety scenarios and evaluation methods is brought up to assist both researchers and industrial practitioners with appreciating the research and practice frontier ofVR/AR-CS and soliciting the latest VR/ AR applications.
532 citations
••
TL;DR: The results indicate a high fragmentation among hardware, software and AR solutions which lead to a high complexity for selecting and developing AR systems.
Abstract: Augmented Reality (AR) technologies for supporting maintenance operations have been an academic research topic for around 50 years now. In the last decade, major progresses have been made and the AR technology is getting closer to being implemented in industry. In this paper, the advantages and disadvantages of AR have been explored and quantified in terms of Key Performance Indicators (KPI) for industrial maintenance. Unfortunately, some technical issues still prevent AR from being suitable for industrial applications. This paper aims to show, through the results of a systematic literature review, the current state of the art of AR in maintenance and the most relevant technical limitations. The analysis included filtering from a large number of publications to 30 primary studies published between 1997 and 2017. The results indicate a high fragmentation among hardware, software and AR solutions which lead to a high complexity for selecting and developing AR systems. The results of the study show the areas where AR technology still lacks maturity. Future research directions are also proposed encompassing hardware, tracking and user-AR interaction in industrial maintenance is proposed.
479 citations
•
01 Jan 2016
TL;DR: It’s time to dust off the gloves and get ready for the cold weather.
Abstract: 1 インフラを構築する(AWSにおけるインフラ;VPCを構成する;VPCとオンプレミス環境とを接続する) 2 ファイルオブジェクトを保存・共有・公開する(オブジェクトストレージS3の機能;ファイルストレージとして利用する;Webサーバーを構築する;信頼性とコストのバランスをとりたい) 3 アプリケーションサーバーを構築する(Amazon EC2とAWS Lambda;スケーラビリティーを高める;サーバーレスでプログラムを動かす;データベースサービスを活用する) 4 AWSシステムを管理する(リソース監視と異常検知・通報;耐障害性を高める仕組みとバックアップ&リカバリー;構成管理)
350 citations
••
TL;DR: It is demonstrated that, extended to the site via the “hand” of AR, the BIM solution can address more real problems, such as low productivity in retrieving information, tendency of committing error in assembly, and low efficiency of communication and problem solving.
198 citations
••
TL;DR: This paper extensively reviews these state-of-the-art vision-based construction performance monitoring methods and divides them into two categories (namely project level: visual monitoring of civil infrastructure or building elements vs. operation level:Visual monitoring of construction equipment and workers).
194 citations