Showing papers on "Graphics published in 2022"

Instant neural graphics primitives with a multiresolution hash encoding

Abstract: Neural graphics primitives, parameterized by fully connected neural networks, can be costly to train and evaluate. We reduce this cost with a versatile new input encoding that permits the use of a smaller network without sacrificing quality, thus significantly reducing the number of floating point and memory access operations: a small neural network is augmented by a multiresolution hash table of trainable feature vectors whose values are optimized through stochastic gradient descent. The multiresolution structure allows the network to disambiguate hash collisions, making for a simple architecture that is trivial to parallelize on modern GPUs. We leverage this parallelism by implementing the whole system using fully-fused CUDA kernels with a focus on minimizing wasted bandwidth and compute operations. We achieve a combined speedup of several orders of magnitude, enabling training of high-quality neural graphics primitives in a matter of seconds, and rendering in tens of milliseconds at a resolution of 1920×1080.

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Abstract: For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other. Recently, however, the community has realized that progress towards robust intelligent systems such as self-driving cars requires a concerted effort across the different fields. This motivated us to develop KITTI-360, successor of the popular KITTI dataset. KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization to facilitate research at the intersection of vision, graphics and robotics. For efficient annotation, we created a tool to label 3D scenes with bounding primitives and developed a model that transfers this information into the 2D image domain, resulting in over 150k images and 1B 3D points with coherent semantic instance annotations across 2D and 3D. Moreover, we established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset, e.g., semantic scene understanding, novel view synthesis and semantic SLAM. KITTI-360 will enable progress at the intersection of these research areas and thus contribute towards solving one of today's grand challenges: the development of fully autonomous self-driving systems.

GPUMD: A package for constructing accurate machine-learned potentials and performing highly efficient atomistic simulations.

Abstract: We present our latest advancements of machine-learned potentials (MLPs) based on the neuroevolution potential (NEP) framework introduced in Fan et al. [Phys. Rev. B 104, 104309 (2021)] and their implementation in the open-source package gpumd. We increase the accuracy of NEP models both by improving the radial functions in the atomic-environment descriptor using a linear combination of Chebyshev basis functions and by extending the angular descriptor with some four-body and five-body contributions as in the atomic cluster expansion approach. We also detail our efficient implementation of the NEP approach in graphics processing units as well as our workflow for the construction of NEP models and demonstrate their application in large-scale atomistic simulations. By comparing to state-of-the-art MLPs, we show that the NEP approach not only achieves above-average accuracy but also is far more computationally efficient. These results demonstrate that the gpumd package is a promising tool for solving challenging problems requiring highly accurate, large-scale atomistic simulations. To enable the construction of MLPs using a minimal training set, we propose an active-learning scheme based on the latent space of a pre-trained NEP model. Finally, we introduce three separate Python packages, viz., gpyumd, calorine, and pynep, that enable the integration of gpumd into Python workflows.

Physics-Based Graphics Models in 3D Synthetic Environments as Autonomous Vision-Based Inspection Testbeds

Abstract: Manual visual inspection of civil infrastructure is high-risk, subjective, and time-consuming. The success of deep learning and the proliferation of low-cost consumer robots has spurred rapid growth in research and application of autonomous inspections. The major components of autonomous inspection include data acquisition, data processing, and decision making, which are usually studied independently. However, for robust real-world applicability, these three aspects of the overall process need to be addressed concurrently with end-to-end testing, incorporating scenarios such as variations in structure type, color, damage level, camera distance, view angle, lighting, etc. Developing real-world datasets that span all these scenarios is nearly impossible. In this paper, we propose a framework to create a virtual visual inspection testbed using 3D synthetic environments that can enable end-to-end testing of autonomous inspection strategies. To populate the 3D synthetic environment with virtual damaged buildings, we propose the use of a non-linear finite element model to inform the realistic and automated visual rendering of different damage types, the damage state, and the material textures of what are termed herein physics-based graphics models (PBGMs). To demonstrate the benefits of the autonomous inspection testbed, three experiments are conducted with models of earthquake damaged reinforced concrete buildings. First, we implement the proposed framework to generate a new large-scale annotated benchmark dataset for post-earthquake inspections of buildings termed QuakeCity. Second, we demonstrate the improved performance of deep learning models trained using the QuakeCity dataset for inference on real data. Finally, a comparison of deep learning-based damage state estimation for different data acquisition strategies is carried out. The results demonstrate the use of PBGMs as an effective testbed for the development and validation of strategies for autonomous vision-based inspections of civil infrastructure.

The Importance of the Application of the Metaverse in Education

Abstract: In the early 90's, and especially in some American universities, with the emergence of virtual reality, virtual environments and their manipulation began to be implemented, achieving important advances that have led to improvements in research through changes in the perception of the subject, modeling, communication processes and the development of 3D virtual classrooms. These advances have led to the metaverse, as an environment where humans interact socially as avatars, and especially its application in the world of education in the fields of entertainment, tele-education, educational research, learning environments, etc. According to technologists, this year will separate the thinkers from the builders, and then the technical advances of the last few years will produce the first steps this year in making the metaverse a reality. Advances will largely be attributed to ever-better graphics processing units (GPUs), photorealistic 3D engines, faster content generation through volumetric video and artificial intelligence, the increasing prevalence of cloud computing and 5G, as well as a more sophisticated and better understood blockchain infrastructure. But from a human experience perspective, one development stands out above all others: extended reality (XR) technologies. These include virtual reality (VR), augmented reality (AR) and brain-computer interfaces (BCI), which together are positioned as the next computing platforms in their own right.

Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments

Abstract: Alternative text is critical in communicating graphics to people who are blind or have low vision. Especially for graphics that contain rich information, such as visualizations, poorly written or an absence of alternative texts can worsen the information access inequality for people with visual impairments. In this work, we consolidate existing guidelines and survey current practices to inspect to what extent current practices and recommendations are aligned. Then, to gain more insight into what people want in visualization alternative texts, we interviewed 22 people with visual impairments regarding their experience with visualizations and their information needs in alternative texts. The study findings suggest that participants actively try to construct an image of visualizations in their head while listening to alternative texts and wish to carry out visualization tasks (e.g., retrieve specific values) as sighted viewers would. The study also provides ample support for the need to reference the underlying data instead of visual elements to reduce users' cognitive burden. Informed by the study, we provide a set of recommendations to compose an informative alternative text.

CityFFD – City fast fluid dynamics for urban microclimate simulations on graphics processing units

Abstract: Urban microclimate analysis is often associated with computational fluid dynamics simulations of urban aerodynamics and thermal conditions, air quality, heat island, humidity, and rain at a building level. Conventional microclimate analyses are often limited by the size of the simulation domain, turbulence models, and sparse digital urban data available. This study introduces CityFFD with its supporting web portal of digital cities for large-scale urban simulation problems. A few novel algorithms, including backward and forward sweep interpolation schemes, time-adaptive technique, and the 2nd-order temporal scheme, were implemented to enhance the computing speed and accuracy of the model. Parallel computing on graphics processing units (GPU) speeds up the simulation. CityFFD is equipped with a large eddy simulation method to capture the turbulence behavior in the urban roughness sublayer. This paper reports the structure of CityFFD, and a group of benchmark cases, including airflow around buildings, natural ventilation, and thermal stratifications around building blocks in an actual urban area, and CityFFD is shown to predict acceptable results of the urban scale wind speeds, temperatures, and flow patterns. This study demonstrates a new approach to modeling urban microclimate by CFD simulations on GPUs. • A new urban microclimate model with a multi-scale simulation portal is presented. • The new model runs on GPUs and OpenMP with unconditional stability. • The web portal enables interactive and real-time data integration for cities. • The new model was validated by benchmark cases including actual urban community.

The Effects of Smart Home Interface Touch Button Design Features on Performance among Young and Senior Users

Abstract: Touch technology-based smart homes have become increasingly prevalent, as they can help people with independent daily life, especially for the elderly. The aim of this study was to investigate the effects of button features (i.e., button size, graphics/text ratio, and icon style) in smart home interfaces on user performance across two age groups. Participants in the young group (n = 15) and senior group (n = 15) completed a clicking task. Button size ranged from 10 mm to 25 mm with 5 mm increments. The three levels of graphics/text ratio were 3:1, 1:1, and 1:3, while icon style was either flat or skeuomorphic. Results showed that button size and graphics/text ratio had significant effects on user performance in both groups, whereas icon style only had an effect in the senior group. It was observed that the elderly were fond of buttons with a larger size of 20 mm with larger texts and skeuomorphic icons, whereas the young preferred a button size of 15 mm with equal-sized graphics and text. These results may help to improve the accessibility and usability of smart home interface design.

Flexplot: Graphically-based data analysis.

Abstract: The human visual processing system has enormous bandwidth, able to interpret vast amounts of data in fractions of a second (Otten et al., 2015). Despite this amazing ability, there is a troubling lack of graphics in scientific literature (Healy & Moody, 2014), and the graphics most traditionally used tend to bias perception in unintentional ways (Weissgerber et al., 2015). I suspect the reason for the underuse and misuse of graphics is because sound graphs are difficult to produce with existing software (Wainer, 2010). While ggplot2 allows immense flexibility in creating graphics, its learning curve is quite steep, and even basic graphics require multiple lines of code. flexplot is an R package that aims to address these issues by providing a formula-based suite of tools that simplifies and automates much of the graphical decision-making. Additionally, flexplot pairs well with statistical modeling, making it easy for researchers to produce graphs that map onto statistical procedures. With one-line functions, users can visualize bivariate statistical models (e.g., scatterplots for regression, beeswarm plots for ANOVA/t-tests), multivariate statistical models (e.g., ANCOVA and multiple regression), and even more sophisticated models like multilevel models and logistic regressions. Further, this package utilizes old tools (e.g., added variable plots and coplots) as well as introduces new tools for complex visualizations, including ghost lines and point sampling. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A Survey on SPH Methods in Computer Graphics

Abstract: Throughout the past decades, the graphics community has spent major resources on the research and development of physics simulators on the mission to computer‐generate behaviors achieving outstanding visual effects or to make the virtual world indistinguishable from reality. The variety and impact of recent research based on Smoothed Particle Hydrodynamics (SPH) demonstrates the concept's importance as one of the most versatile tools for the simulation of fluids and solids. With this survey, we offer an overview of the developments and still‐active research on physics simulation methodologies based on SPH that has not been addressed in previous SPH surveys. Following an introduction about typical SPH discretization techniques, we provide an overview over the most used incompressibility solvers and present novel insights regarding their relation and conditional equivalence. The survey further covers recent advances in implicit and particle‐based boundary handling and sampling techniques. While SPH is best known in the context of fluid simulation we discuss modern concepts to augment the range of simulatable physical characteristics including turbulence, highly viscous matter, deformable solids, as well as rigid body contact handling. Besides the purely numerical approaches, simulation techniques aided by machine learning are on the rise. Thus, the survey discusses recent data‐driven approaches and the impact of differentiable solvers on artist control. Finally, we provide context for discussion by outlining existing problems and opportunities to open up new research directions.

A Comparison of SYCL, OpenCL, CUDA, and OpenMP for Massively Parallel Support Vector Machine Classification on Multi-Vendor Hardware

Abstract: In scientific computing and Artificial Intelligence (AI), which both rely on massively parallel tasks, frameworks like the Compute Unified Device Architecture (CUDA) and the Open Computing Language (OpenCL) are widely used to harvest the computational power of accelerator cards, in particular of Graphics Processing Units (GPUs). A few years ago, GPUs from NVIDIA were used almost exclusively for these tasks but meanwhile, AMD and Intel are increasing their shares of the GPUs market. This introduces many new challenges for code development, as the prevailing CUDA code can only run on NVIDIA hardware and must be adapted or even completely rewritten to run on GPUs from AMD or Intel. In this paper, we compare the different competing programming frameworks OpenMP, CUDA, OpenCL, and SYCL, paying special attention to the two SYCL implementations hipSYCL and DPC++. Thereby, we investigate the different frameworks with respect to their usability, performance, and performance portability on a variety of hardware platforms from different vendors, i.e., GPUs from NVIDIA, AMD, and Intel and Central Processing Units (CPUs) from AMD and Intel. Besides discussing the runtimes of these frameworks on the different hardware platforms, we also focus our comparison on the differences between the nd_range kernel formulation and the SYCL specific hierarchical kernels. Our Parallel Least Squares Support Vector Machine (PLSSVM) library implements backends for the four previously mentioned programming frameworks for a Least Squares Support Vector Machines (LS-SVMs). At its example, we show which of the frameworks is best suited for a standard workload that is frequently employed in scientific computing and AI, depending on the target hardware: The most computationally intensive part of our PLSSVM library is solving a system of linear equations using the Conjugate Gradient (CG) method. Specifically, we parallelize the implicit matrix-vector multiplication inside the CG method, a workload common in many scientific codes. The PLSSVM code, utility scripts, and documentation are all available on GitHub: https://github.com/SC-SGS/PLSSVM.

3D modeling and reconstruction of plants and trees: A cross-cutting review across computer graphics, vision, and plant phenotyping

Abstract: This paper reviews the past and current trends of three-dimensional (3D) modeling and reconstruction of plants and trees. These topics have been studied in multiple research fields, including computer vision, graphics, plant phenotyping, and forestry. This paper, therefore, provides a cross-cutting review. Representations of plant shape and structure are first summarized, where every method for plant modeling and reconstruction is based on a shape/structure representation. The methods were then categorized into 1) creating non-existent plants (modeling) and 2) creating models from real-world plants (reconstruction). This paper also discusses the limitations of current methods and possible future directions.

Multi-hetero Acceleration by GPU and FPGA for Astrophysics Simulation on oneAPI Environment

Abstract: GPU (Graphics Processing Unit) computing is one of the most popular accelerating methods for various high-performance computing applications. For scientific computations based on multi-physical phenomena, however, a single device solution on a GPU is insufficient, where the single timescale or degree of parallelism is not simply supported by a simple GPU-only solution. We have been researching a combination of a GPU and FPGA (Field Programmable Gate Array) for such complex physical simulations. The most challenging issue is how to program these multiple devices using a single code. OneAPI, recently provided by Intel, is a programming paradigm supporting such a solution on a single language platform using DPC++ based on SYCL 2020. However, there are no practical applications utilizing its full features or supporting heterogeneous multi-device programming to demonstrate its potential capability. In this study, we present the implementation and performance evaluation of our astrophysics code ARGOT used to apply the oneAPI solution with a GPU and an FPGA. To realize our concept of Cooperative Heterogeneous Acceleration by Reconfigurable Multidevices, also known as CHARM, as a type of next-generation accelerated supercomputing for complex multi-physical simulations, this study was conducted on our multi-heterogeneous accelerated cluster machine running at the University of Tsukuba. Through the research, we found that current oneAPI framework is effective not only for its typical programming by DPC++ but also for utilizing traditionally developed applications coded by several other languages such as CUDA or OpenCL to support multiple types of accelerators. As an example of real application, we successfully implemented and executed an early stage universe simulation by fundamental astrophysics code to utilize both GPU and FPGA effectively. In this paper, we demonstrate the actual procedure for this method to program multi-device acceleration over oneAPI.

Graphics Peeping Unit: Exploiting EM Side-Channel Information of GPUs to Eavesdrop on Your Neighbors

Abstract: As the popularity of graphics processing units (GPUs) grows rapidly in recent years, it becomes very critical to study and understand the security implications imposed by them. In this paper, we show that modern GPUs can “broadcast” sensitive information over the air to make a number of attacks practical. Specifically, we present a new electromagnetic (EM) side-channel vulnerability that we have discovered in many GPUs of both NVIDIA and AMD. We show that this vulnerability can be exploited to mount realistic attacks through two case studies, which are website fingerprinting and keystroke timing inference attacks. Our investigation recognizes the commonly used dynamic voltage and frequency scaling (DVFS) feature in GPU as the root cause of this vulnerability. Nevertheless, we also show that simply disabling DVFS may not be an effective countermeasure since it will introduce another highly exploitable EM side-channel vulnerability. To the best of our knowledge, this is the first work that studies realistic physical side-channel attacks on non-shared GPUs at a distance.

OUP accepted manuscript

Abstract: Abstract Summary CellWalkR is an R package that integrates single-cell open chromatin data with cell type labels and bulk epigenetic data to identify cell type-specific regulatory regions. A Graphics Processing Unit (GPU) implementation and downsampling strategies enable thousands of cells to be processed in seconds. CellWalkR’s user-friendly interface provides interactive analysis and visualization of cell labels and regulatory region mappings. Availability and implementation CellWalkR is freely available as an R package under a GNU GPL-2.0 License and can be accessed from https://github.com/PFPrzytycki/CellWalkR with an accompanying vignette. Supplementary information Supplementary data are available at Bioinformatics online.

Bibliometric Analysis of Visualizations in Computer Graphics: A Study

Abstract: This study examined the field of visualizations in computer graphics (VCG) using bibliometric methods, including performance and science mapping analysis. A dataset of documents from SCOPUS, 1986 to 2019, was analyzed and visualized using VOSviewer. The results showed an increasing trend of new documents in VCG. The five most cited publications were all related to data visualization software. The most prolific authors, examined by Citation per Paper (CPP) and Relative Citation Impact (RCI), contributed to research advances in computer graphics, information visualization, interactive data analysis, human computer interfaces, and visualization software development. A document source analysis identified the major scientific journals and conference proceedings in VCG research, and showed that researchers in VCG tend to publish extensively in conference proceedings, but that articles in scientific journals have a higher citation impact. A co-citation analysis of cited sources revealed seven clusters of thematically similar sources in computer science, genomics, neuroimaging, physics & chemistry, mathematics, education, and communication. Co-authorship analysis of countries pointed out collaboration networks in scientific research, where the USA, UK, Germany, France, and Italy collaborated most frequently. Collaborations were fostered by the same language group, or the geographical proximity. The co-occurrence of research terms showed six clusters of related concepts, thematically grouped around search queries, graphical processing, education, genetics, scientific numerical data, and medicine. The study contributed to a better understanding of the field and is expected to help researchers and educators identify research areas, developments, quality scientific literature, and appropriate journals for publishing their own findings related to VCG.

Design on Intelligent Feature Graphics Based on Convolution Operation

Abstract: With the development and application of artificial intelligence, the technical methods of intelligent image processing and graphic design need to be explored to realize the intelligent graphic design based on traditional graphics such as pottery engraving graphics. An optimized method is aimed to be explored to extract the image features from traditional engraving graphics on historical relics and apply them into intelligent graphic design. For this purpose, an image feature extracted model based on convolution operation is proposed. Parametric test and effectiveness research are conducted to evaluate the performance of the proposed model. Theoretical and practical research shows that the image-extracted model has a significant effect on the extraction of image features from traditional engraving graphics because the image brightness processing greatly simplifies the process of image feature extraction, and the convolution operation improves the accuracy. Based on the brightness feature map output from the proposed model, the design algorithm of intelligent feature graphic is presented to create the feature graphics, which can be directly applied to design the intelligent graphical interface. Taking some pottery engraving graphics from the Neolithic Age as an example, we conduct the practice on image feature extraction and feature graphic design, the results of which further verify the effectiveness of the proposed method. This paper provides a theoretical basis for the application of traditional engraving graphics in intelligent graphical interface design for AI products such as smart tourism products, smart museums, and so on.

Optimization Techniques for GPU Programming

Abstract: In the past decade, Graphics Processing Units have played an important role in the field of high-performance computing and they still advance new fields such as IoT, autonomous vehicles, and exascale computing. It is therefore important to understand how to extract performance from these processors, something that is not trivial. This survey discusses various optimization techniques found in 450 articles published in the last 14 years. We analyze the optimizations from different perspectives which shows that the various optimizations are highly interrelated, explaining the need for techniques such as auto-tuning.

Easy2Hard: Learning to Solve the Intractables From a Synthetic Dataset for Structure-Preserving Image Smoothing

Abstract: Image smoothing is a prerequisite for many computer vision and graphics applications. In this article, we raise an intriguing question whether a dataset that semantically describes meaningful structures and unimportant details can facilitate a deep learning model to smooth complex natural images. To answer it, we generate ground-truth labels from easy samples by candidate generation and a screening test and synthesize hard samples in structure-preserving smoothing by blending intricate and multifarious details with the labels. To take full advantage of this dataset, we present a joint edge detection and structure-preserving image smoothing neural network (JESS-Net). Moreover, we propose the distinctive total variation loss as prior knowledge to narrow the gap between synthetic and real data. Experiments on different datasets and real images show clear improvements of our method over the state of the arts in terms of both the image cleanness and structure-preserving ability. Code and dataset are available at https://github.com/YidFeng/Easy2Hard .

A Perceptual Color-Matching Method for Examining Color Blending in Augmented Reality Head-Up Display Graphics.

Abstract: Augmented reality (AR) offers new ways to visualize information on-the-go. As noted in related work, AR graphics presented via optical see-through AR displays are particularly prone to color blending, whereby intended graphic colors may be perceptually altered by real-world backgrounds, ultimately degrading usability. This work adds to this body of knowledge by presenting a methodology for assessing AR interface color robustness, as quantitatively measured via shifts in the CIE color space, and qualitatively assessed in terms of users' perceived color name. We conducted a human factors study where twelve participants examined eight AR colors atop three real-world backgrounds as viewed through an in-vehicle AR head-up display (HUD); a type of optical see-through display used to project driving-related information atop the forward-looking road scene. Participants completed visual search tasks, matched the perceived AR HUD color against the WCS color palette, and verbally named the perceived color. We present analysis that suggests blue, green, and yellow AR colors are relatively robust, while red and brown are not, and discuss the impact of chromaticity shift and dispersion on outdoor AR interface design. While this work presents a case study in transportation, the methodology is applicable to a wide range of AR displays in many application domains and settings.

Accelerators for Classical Molecular Dynamics Simulations of Biomolecules

Abstract: Atomistic Molecular Dynamics (MD) simulations provide researchers the ability to model biomolecular structures such as proteins and their interactions with drug-like small molecules with greater spatiotemporal resolution than is otherwise possible using experimental methods. MD simulations are notoriously expensive computational endeavors that have traditionally required massive investment in specialized hardware to access biologically relevant spatiotemporal scales. Our goal is to summarize the fundamental algorithms that are employed in the literature to then highlight the challenges that have affected accelerator implementations in practice. We consider three broad categories of accelerators: Graphics Processing Units (GPUs), Field-Programmable Gate Arrays (FPGAs), and Application Specific Integrated Circuits (ASICs). These categories are comparatively studied to facilitate discussion of their relative trade-offs and to gain context for the current state of the art. We conclude by providing insights into the potential of emerging hardware platforms and algorithms for MD.

Overview of Screen Content Coding in Recently Developed Video Coding Standards

Abstract: In recent years, computer-generated texts, graphics, and animations have drawn more attention than ever. These types of media, also known as screen content, have become increasingly popular due to their widespread applications. To address the need for efficient coding of such content, several coding tools have been developed and have made great advances in terms of coding efficiency. The inclusion of screen content coding features in some recently developed video coding standards (namely, HEVC SCC, VVC, AVS3, AV1 and EVC) demonstrates the importance of supporting such features. This paper provides an overview and comparative study of screen content coding technologies, as well as discussions on the performance and complexity of the tools developed in these standards.