Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pokemon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have bought us various degrees of digital transformation. The term `metaverse' has been coined to further facilitate the digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent, and shared realm. While the metaverse may seem futuristic, catalysed by emerging technologies such as Extended Reality, 5G, and Artificial Intelligence, the digital `big bang' of our cyberspace is not far away. This survey paper presents the first effort to offer a comprehensive framework that examines the latest metaverse development under the dimensions of state-of-the-art technologies and metaverse ecosystems, and illustrates the possibility of the digital `big bang'. First, technologies are the enablers that drive the transition from the current Internet to the metaverse. We thus examine eight enabling technologies rigorously - Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing, and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. Therefore, we discuss six user-centric factors -- Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy, and Trust and Accountability. Finally, we propose a concrete research agenda for the development of the metaverse.

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

We explore how to add haptics to walls and other heavy objects in virtual reality. When a user tries to push such an object, our system actuates the user's shoulder, arm, and wrist muscles by means of electrical muscle stimulation, creating a counter force that pulls the user's arm backwards. Our device accomplishes this in a wearable form factor. In our first user study, participants wearing a head-mounted display interacted with objects provided with different types of EMS effects. The repulsion design (visualized as an electrical field) and the soft design (visualized as a magnetic field) received high scores on "prevented me from passing through" as well as "realistic". In a second study, we demonstrate the effectiveness of our approach by letting participants explore a virtual world in which all objects provide haptic EMS effects, including walls, gates, sliders, boxes, and projectiles.

Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation

What is Mixed Reality (MR)? To revisit this question given the many recent developments, we conducted interviews with ten AR/VR experts from academia and industry, as well as a literature survey of 68 papers. We find that, while there are prominent examples, there is no universally agreed on, one-size-fits-all definition of MR. Rather, we identified six partially competing notions from the literature and experts' responses. We then started to isolate the different aspects of reality relevant for MR experiences, going beyond the primarily visual notions and extending to audio, motion, haptics, taste, and smell. We distill our findings into a conceptual framework with seven dimensions to characterize MR applications in terms of the number of environments, number of users, level of immersion, level of virtuality, degree of interaction, input, and output. Our goal with this paper is to support classification and discussion of MR applications' design and provide a better means to researchers to contextualize their work within the increasingly fragmented MR landscape.

What is Mixed Reality

Ungrounded haptic devices for virtual reality (VR) applications lack the ability to convincingly render the sensations of a grasped virtual object's rigidity and weight. We present Grabity, a wearable haptic device designed to simulate kinesthetic pad opposition grip forces and weight for grasping virtual objects in VR. The device is mounted on the index finger and thumb and enables precision grasps with a wide range of motion. A unidirectional brake creates rigid grasping force feedback. Two voice coil actuators create virtual force tangential to each finger pad through asymmetric skin deformation. These forces can be perceived as gravitational and inertial forces of virtual objects. The rotational orientation of the voice coil actuators is passively aligned with the real direction of gravity through a revolute joint, causing the virtual forces to always point downward. This paper evaluates the performance of Grabity through two user studies, finding promising ability to simulate different levels of weight with convincing object rigidity. The first user study shows that Grabity can convey various magnitudes of weight and force sensations to users by manipulating the amplitude of the asymmetric vibration. The second user study shows that users can differentiate different weights in a virtual environment using Grabity.

https://dl.acm.org/doi/pdf/10.1145/3126594.3126599

Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality

Metamaterials exhibit properties beyond those exhibited by conventional materials in conventional scenarios. These have been investigated both theoretically and experimentally at length. In many cases the underpinning physical understanding of metamaterials has greatly preceded our ability to manufacture constituent structures. However, the development of additive manufacturing techniques gives new possibilities for the fabrication of complex metamaterial structures, many of which cannot be realised through conventional fabrication methods. The literature to date contains contributions from a diverse group of researchers from the physical sciences, mathematics, and manufacturing technology in the creation of metamaterials for electromagnetic, acoustic and mechanical applications. It is proposed that additive manufacturing holds the key to realise the capabilities of this vibrant research community and permit the creation of new paradigms in fundamental structures but also exploitation through application. For this purpose, a literature review is presented which identifies key advances in metamaterials alongside additive manufacturing and proposes new opportunities for researchers to work together through intra/inter disciplinary research to realise structures which exhibit extraordinary behaviour(s). This review represents a comprehensive account of the state-of-the-art in the production of such metamaterials using additive manufacturing methods and highlights areas, which, based on trends observed in the literature, are worthy of further research and require a coordinated effort on behalf of the afore mentioned disciplines in order to advance the state-of-the-art.

Additive manufacturing of metamaterials: A review

We present impacto, a device designed to render the haptic sensation of hitting or being hit in virtual reality. The key idea that allows the small and light impacto device to simulate a strong hit is that it decomposes the stimulus: it renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impact to the hit by thrusting the user's arm backwards using electrical muscle stimulation. The device is self-contained, wireless, and small enough for wearable use, thus leaves the user unencumbered and able to walk around freely in a virtual environment. The device is of generic shape, allowing it to also be worn on legs, so as to enhance the experience of kicking, or merged into props, such as a baseball bat. We demonstrate how to assemble multiple impacto units into a simple haptic suit. Participants of our study rated impact simulated using impacto's combination of solenoid hit and electrical muscle stimulation as more realistic than either technique in isolation.

Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation

Recently, researchers started to engineer not only the outer shape of objects, but also their internal microstructure. Such objects, typically based on 3D cell grids, are also known as metamaterials. Metamaterials have been used, for example, to create materials with soft and hard regions.So far, metamaterials were understood as materials-we want to think of them as machines. We demonstrate metamaterial objects that perform a mechanical function. Such metamaterial mechanisms consist of a single block of material the cells of which play together in a well-defined way in order to achieve macroscopic movement. Our metamaterial door latch, for example, transforms the rotary movement of its handle into a linear motion of the latch. Our metamaterial Jansen walker consists of a single block of cells-that can walk. The key element behind our metamaterial mechanisms is a specialized type of cell, the only ability of which is to shear.In order to allow users to create metamaterial mechanisms efficiently we implemented a specialized 3D editor. It allows users to place different types of cells, including the shear cell, thereby allowing users to add mechanical functionality to their objects. To help users verify their designs during editing, our editor allows users to apply forces and simulates how the object deforms in response.

Metamaterial Mechanisms

We propose a new way of eyes-free interaction for wearables. It is based on the user's proprioceptive sense, i.e., rather than seeing, hearing, or feeling an outside stimulus, users feel the pose of their own body. We have implemented a wearable device called Pose-IO that offers input and output based on proprioception. Users communicate with Pose-IO through the pose of their wrists. Users enter information by performing an input gesture by flexing their wrist, which the device senses using a 3-axis accelerometer. Users receive output from Pose-IO by find-ing their wrist posed in an output gesture, which Pose-IO actuates using electrical muscle stimulation. This mechanism allows users to interact with Pose-IO without visual or auditory senses, but through the proprioceptive sense alone. We developed three simple applications that demonstrate symmetric proprioceptive interaction, where input and output occur through the same limb, as well as asymmetric interaction, where input and output occur through different limbs. In a first user study, participants using a symmetric proprioceptive interface re-entered poses received from Pose-IO with an average accuracy of 5.8° despite the minimal bandwidth offered by the device. In a second, exploratory study, we investigated participants' emotional response to asymmetric proprioceptive interaction and the concept of the user's body serving as interface. Participants reported to enjoy the experience (4.6 out of 5).

Proprioceptive Interaction

In this paper we present the results of a study of human preferences in using mid-air gestures for directing other humans. Rather than contributing a specific set of gestures, we contribute a set of gesture types, which together make a set of the core actions needed to complete any of our six chosen tasks in the domain of human-to-human gestural communication without the speech channel. We observed 12 participants, cooperating to accomplish different tasks only using hand gestures to communicate. We analyzed 5,500 gestures in terms of hand usage and gesture type, using a novel classification scheme which combines three existing taxonomies in order to better capture this interaction space. Our findings indicate that, depending on the meaning of the gesture, there is preference in the usage of gesture types, such as pointing, pantomimic acting, direct manipulation, semaphoric, or iconic gestures. These results can be used as guidelines to design purely gesture driven interfaces for interactive environments and surfaces.

/pdf/understanding-mid-air-hand-gestures-a-study-of-human-19qwjvcl7p.pdf

Understanding Mid-Air Hand Gestures: A Study of Human Preferences in Usage of Gesture Types for HCI

In this paper, we explore how to embody mechanical computation into 3D printed objects, i.e., without electronic sensors, actuators, or controllers typically used for this purpose. A key benefit of our approach is that the resulting objects can be 3D printed in one piece and thus do not require assembly. We are building on 3D printed cell structures, also known as metamaterials. We introduce a new type of cell that propagates a digital mechanical signal using an embedded bistable spring. When triggered, the embedded spring discharges and the resulting impulse triggers one or more neighboring cells, resulting in signal propagation. We extend this basic mechanism to implement simple logic functions. We demonstrate interactive objects based on this concept, such as a combination lock. We present a custom editor that allows users to model 3D objects, route signals, simulate signal flow, and synthesize cell patterns.

Alexandra Ion

Papers

Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation

Metamaterial Mechanisms

Proprioceptive Interaction

Understanding Mid-Air Hand Gestures: A Study of Human Preferences in Usage of Gesture Types for HCI

Digital Mechanical Metamaterials