Augment-able reality: situated communication through physical and digital spaces
Summary (3 min read)
1 Introduction
- Augmented Reality (AR) systems are designed to provide an enhanced view of the real world through see-through head-mounted displays[16] or hand-held devices[11].
- Various kinds of context sensing technologies, such as position sensors[5, 4], or ID readers[11], are used to determine digital information according to the user’s current physical context.
- Taking into account recent advances in wearable devices, the authors can expect that people will soon carry their wearables at all times, and that their lives will be constantly supported by context-sensitive information from those wearables.
- In that sense, current AR systems are essentially ‘‘context-sensitive browsers’’ for the real world, and information is limited to a one-way flow.
- The user of traditional computers can also add data to the physical environment through the Web and E-Mail interfaces.
2 Augment-able Environments
- Let us start with a scenario when augment-able reality becomes commonplace.
- You create a voice-note with a still picture of the damaged tape and attach it to the VCR for other users.
- When you go outside for lunch, you find several virtual messages are floating in front of restaurants.
- From the system’s point of view, this is achieved by storing a created data item with contextual information obtained from several wearable sensors.
- There are several possibilities about data types that can be attached to the environment.
Context-sensitive information notification
- Similar to other AR systems, the attached data can be browsed by (the same or other) users who are wearing computers.
- Conversely, normal computer users can attach data to the physical context through the Web or E-Mail interfaces, and wearable users will notice the data when they come to the corresponding situation.
- Instead of sending messages to people, the authors can place message in "physical contexts" to indirectly communicate with other people.
- After having dinner at a restaurant, for example, people might leave their impressions or ratings of that restaurant at that location.
- Even when messages are directed to the person, the authors can select several contextual attributes to do that.
3 System Design
- To achieve the proposed concept described in the previous section, the authors are currently developing a set of prototype systems based on wearable and normal desktop computers.
- Figure 2 shows the overall system design of the prototype.
- Systems share the same database server on the network through wired or wireless communication.
- If one user attaches a data item to the particular physical context (e.g., location), this effect immediately becomes visible from other computers.
- The following subsections present details of each subsystem as well as their user interfaces.
3.1 Environmental Support
- To make it easier for wearable computers to capture surrounding contexts, the authors have deployed two kinds ID systems in the environment.
- One noticeable difference between these two ID systems is emission range.
- While IR beacons can cover room-size areas and are relatively robust regarding orientation of the sensors, printed IDs are more sensitive to distance and the camera orientation.
- Since 2D codes are virtually costless and printable as well, there are some usages that could not be achieved by other ID systems.
- This card can convey digital data such as voice notes or photographs with attachment of the ID.
Hardware
- The head-worn part of the wearable system consists of a monocular see-through head-up display (based on Sony Glasstron), a CCD camera, an infrared sensor (based on a remote commander chip for consumer electronics).
- These devices are connected to a sub-note PC (Mitsubishi AMiTY) communicating to the network through a spread-spectrum wireless LAN (Netwave Air- Surfer).
- The user controls the system with a miniature pointing device.
- The head-worn camera is primarily used to detect 2D codes in the environment, but is also used to take still image pictures.
- Figure 5 shows a user creating a voicenote with the wearable unit.
User interface
- As a context-aware browser for the real world, this wearable system acts mostly the same as their previous AR system NaviCam[11].
- The system recognizes the surrounding environment by recognizing attached visual tags on physical objects, or by detecting infrared beacons installed at locations in the environment.
- These panes represent location-level and object-level contexts.
- An icon representing this newly created voice note then appears on the personal tray.
- Afterwards, other users (wearing a computer) who try to use the VCR will find your warning.
Time Machine Mode
- The context-aware panes are not always directly linked to the current physical situation.
- When the user taps on the left side of the window, the system switches to what the authors call the ‘‘Time Machine Mode’’ .
- Combining these navigation techniques allows a user to attach data to any context at any time.
- Suppose that you want to attach a meeting agenda to the conference room but it should not become visible until next Monday.
- To encourage this technique, the personal tray is separated into two parts (left and right) during the Time Machine Mode.
Web (Java Applet) Interface
- Attached information can also be accessed from normal computing environments.
- The authors have developed a Java applet for retrieving and adding information to the physical environment .
- The user can display or playback attached data simply by clicking a corresponding icon on the floor map.
- The user can also attach data to a physical location or an object through the map window of the applet.
- This attachment immediately becomes visible to other users.
E-Mail interface
- When users wish to attach a message to the meeting room, they can simply send mail to that room, such as: To: room3@ar.csl.sony.co.jp.
- Msg=Today’s meeting is cancelled, also known as Subject.
- This capability encourages a user to attach information from a remote and mobile environment (typically using a PDA with a limited communication bandwidth).
Graceful notification
- Based-on their initial experience with the prototype system, the authors feel that the key design issue on augment-able reality is how the system can gracefully notify situated information.
- When a user notices the presence of situated data, they can browse it through a palmtop or wristtop display.
- The authors believe that there are several features that can only be available by digitally attaching the data.
- Secondly, the authors can apply several information retrieval techniques to filter attached information.
- While Post-it notes are visible to anybody, digital data could be disclosed to selected people only.
6 Conclusion and Future Directions
- The authors have described ‘‘augment-able reality’’ where people can dynamically create digital data and attach it to the physical context.
- The authors have developed working prototypes to explore this concept and has created user interfaces that supports easy data transfer between personal and context-aware information space.
- The authors are also currently working on a system using GPS, based on the same concept.
- The user will be able to digitally leave a hand-written note or other data type on the current geographic location, where others can observe the attached data from wearable and traditional computing environments.
Did you find this useful? Give us your feedback
Citations
436 citations
Cites background from "Augment-able reality: situated comm..."
...Finally, the paper will discuss some of the challenges and opportunities facing researchers in this area....
[...]
402 citations
Cites background from "Augment-able reality: situated comm..."
...Another common strategy is to use post-it notes, paper or virtual [2,15], placed in locations where the intended recipients can view them....
[...]
345 citations
Cites methods from "Augment-able reality: situated comm..."
...used an augmented reality system to attach notes to objects or locations [12]....
[...]
341 citations
Cites background from "Augment-able reality: situated comm..."
...Some have used goggles as access medium [21]....
[...]
...Marmasse & Schmandt’s [18] GPS enabled ComMotion system and Rekimoto’s [21] goggle system focus on user’s annotations, but these are mainly directed to oneself as reminders and documents....
[...]
310 citations
References
1,825 citations
"Augment-able reality: situated comm..." refers background in this paper
...Augmented Reality (AR) systems are designed to provide an enhanced view of the real world through see-through head-mounted displays[16] or hand-held devices[11]....
[...]
1,032 citations
"Augment-able reality: situated comm..." refers background in this paper
...Various kinds of context sensing technologies, such as position sensors[5, 4], or ID readers[11], are used to determine digital information according to the user’s current physical context....
[...]
...For example, KARMA[4], which is a well-known AR system, displays information about the laser printer based on the current physical position of a head-worn display....
[...]
916 citations
753 citations
"Augment-able reality: situated comm..." refers methods in this paper
...‘‘Pick and Drop’’[10] provides a method for carrying digital data within a physical space using a stylus....
[...]
573 citations
Related Papers (5)
Frequently Asked Questions (10)
Q2. What is the key to a successful AR and wearable systems?
Since humans rely on artificial markers such as traffic signals or indication panels, the authors believe that having appropriate artificial supports in the physical environment is the key to a successful AR and wearable systems.
Q3. What browser can be used to access the attached data?
Since this applet can be started from Java-enabled web browsers (such as Netscape or Internet Explorer), the user can access the attached data from virtually anywhere.
Q4. What is the purpose of this paper?
This paper presents an environment that supports information registration in the real world contexts through wearable and traditional computers.
Q5. What are the different types of sensors used to determine digital information according to the user’s current?
Various kinds of context sensing technologies, such as position sensors[5, 4], or ID readers[11], are used to determine digital information according to the user’s current physical context.
Q6. What is the inspiration for the interface design for attaching data?
Their interface design for attaching data is partially inspired by ‘‘Fix and Float’’[13], an interface technique for carrying data within a virtual 3D environment.
Q7. What is the key design issue on augmentable reality?
Based-on their initial experience with the prototype system, the authors feel that the key design issue on augment-able reality is how the system can gracefully notify situated information.
Q8. What is the head-worn part of the wearable system?
The head-worn part of the wearable system (Figure 4, above) consists of a monocular see-through head-up display (based on Sony Glasstron), a CCD camera, an infrared sensor (based on a remote commander chip for consumer electronics).
Q9. What is the purpose of the paper?
In this paper, the authors have described ‘‘augment-able reality’’ where people can dynamically create digital data and attach it to the physical context.
Q10. What is the current approach to displaying information on a see-through headup display?
Their current approach is to overlay information on a see-through headsup display; the authors expect this approach to be less obtrusive when the display is embedded in eyeglasses [14].