scispace - formally typeset
Search or ask a question
Author

Bipul Mohanto

Bio: Bipul Mohanto is an academic researcher from University of Rostock. The author has contributed to research in topics: Artificial intelligence & Human visual system model. The author has co-authored 1 publications.

Papers
More filters
Journal ArticleDOI
TL;DR: Foveated rendering as mentioned in this paper adapts the image synthesis process to the user's gaze by exploiting the human visual system's limitations, in particular in terms of reduced acuity in peripheral vision, it strives to deliver high-quality visual experiences at very reduced computational, storage and transmission costs.

6 citations


Cited by
More filters
Journal ArticleDOI
TL;DR: In this article , a systematic summary of the latest research on various materials, sensors, and integrated systems for monitoring eye movements and enabling human-machine interfaces is presented, including soft materials, biocompatible materials, manufacturing methods, sensor functions, systems' performances, and their applications.

1 citations

Proceedings ArticleDOI
01 Mar 2023
TL;DR: In this article , the tradeoff between fixed foveated rendering (FFR), gaze-tracked Foveated Rendering (TFR), and conventional, non-foveated renderings is studied.
Abstract: Extended reality (XR) devices, including augmented, virtual, and mixed reality, provide a deeply immersive experience. However, practical limitations like weight, heat, and comfort put extreme constraints on the performance, power consumption, and image quality of such systems. In this paper, we study how these constraints form the tradeoff between Fixed Foveated Rendering (FFR), Gaze-Tracked Foveated Rendering (TFR), and conventional, non-foveated rendering. While existing papers have often studied these methods, we provide the first comprehensive study of their relative feasibility in practical systems with limited battery life and computational budget. We show that TFR with the added cost of the gaze-tracker can often be more expensive than FFR. Thus, we co-design a gaze-tracked foveated renderer considering its benefits in computation, power efficiency, and tradeoffs in image quality. We describe principled approximations for eye tracking which provide up to a 9x speedup in runtime performance with approximately a 20x improvement in energy efficiency when run on a mobile GPU. In isolation, these approximations appear to significantly degrade the gaze quality, but appropriate compensation in the visual pipeline can mitigate the loss. Overall, we show that with a highly optimized gaze-tracker, TFR is feasible compared to FFR, resulting in up to 1.25x faster frame times while also reducing total energy consumption by over 40%.
Journal ArticleDOI
TL;DR: A robot remote operating system based on virtual reality and digital twin based on the Unity 3D engine and the combination of eye-tracking-based eye movement interaction and the unique controller interaction of virtual reality system is proposed, achieving a multi-sensory multi-input collaborative interaction method.
Abstract: To realize efficient remote human-computer interaction of robots, a robot remote operating system based on virtual reality and digital twin is proposed. The system builds a digital twin model based on the Unity 3D engine to establish a connection with the robot entity, assisting the online remote programming and real-time manipulation of the robot unit. The system uses HTC VIVE to build a virtual reality framework. To actualize the mutual drive between the real space and the virtual space, a mathematical model of the robot is constructed through the forward and inverse kinematics of the robot. Through the combination of eye-tracking-based eye movement interaction and the unique controller interaction of virtual reality system, a multi-sensory multi-input collaborative interaction method is accomplished. The method realizes the robot joints driving of users using multiple interaction methods simultaneously, simplifies the robot programming and control procedure, and optimizes the operation experience. Tests demonstrate that the system is capable of effectively providing monitoring, teleoperation and programming services for remote interaction of robots.
Book ChapterDOI
01 Jan 2023
TL;DR: In this paper , a taxonomy of 3D display systems and categorization of them based on whether headgear or visual separators are required for stereo cues and how the depth cue is reproduced (varifocal vs multifocal displays).
Abstract: Computational 3D display is an emerging class of display systems, which integrates spatial light modulators (SLMs) and optical components by computational methods. Those displays aim to reproduce a complete set of 3D visual cues. In this chapter, we first introduce different aspects of the human visual perception for display systems by interpreting spatial, spectral, temporal, and intensity parameters of the plenoptic function. Next, we provide a taxonomy of 3D display systems and categorize them based on whether headgear or visual separators are required for stereo cues (autostereoscopic vs stereoscopic displays), and how the depth cue is reproduced (varifocal vs multifocal displays). Then, we focus on a subclass of computational 3D displays, optical see-through displays, as a representative display type widely used in AR. Finally, we introduce perception-driven rendering algorithms, which can enhance the quality and performance of the applicable computational 3D displays.
Journal ArticleDOI
TL;DR: In this article , a human-robot interface using a head-mounted device (HMD) is presented to enable high-precision interaction and specialized visualization in particle physics accelerators.
Abstract: In hazardous environments, where conditions present risks for humans, the maintenance and interventions are often done with teleoperated remote systems or mobile robotic manipulators to avoid human exposure to dangers. The increasing need for safe and efficient teleoperation requires advanced environmental awareness and collision avoidance. The up-to-date screen-based 2D or 3D interfaces do not fully allow the operator to immerse in the controlled scenario. This problem can be addressed with the emerging Mixed Reality (MR) technologies with Head-Mounted Devices (HMDs) that offer stereoscopic immersion and interaction with virtual objects. Such human-robot interfaces have not yet been demonstrated in telerobotic interventions in particle physics accelerators. Moreover, robotic operations often require a few experts to collaborate, which increases the system complexity and requires sharing a multi-user Augmented Reality (AR) workspace. The multi-user telerobotics with shared control in the AR has not yet been approached in the industrial state-of-the-art. In this work, the developed MR human-robot interface using the AR HMD is presented. The interface adapts to the constrained wireless networks in particle accelerator facilities and provides reliable high-precision interaction and specialized visualization. The multimodal operation uses hands, eyes and user motion tracking, and voice recognition for control, as well as offers video, 3D point cloud and audio feedback from the robot. Multiple experts can collaborate in the AR workspace locally or remotely, share the robot’s control and monitor robotic teleoperation. Ten (10) operators tested the interface in intervention scenarios in the European Organization for Nuclear Research (CERN) with complete network characterization and measurements to conclude if operational requirements were met and if the network architecture could support single and multi-user communication load. The interface system has proved to be operationally ready at the Technical Readiness Level (TRL) 8 - and was validated through successful tests and demonstration in single and multi-user missions. Some areas of system limitations and further work were identified, such as optimising the network architecture for multi-user scenarios or high-level interface actions applying automatic interaction strategies with the robot depending on network conditions.