Showing papers on "Graphics published in 2019"

Dynamic Graph CNN for Learning on Point Clouds

Abstract: Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS.

ggplot2: Elegant Graphics for Data Analysis (2nd ed.)

Abstract: In this book review, we offer chapter-by-chapter review and general comments of Hadley Wickham’s (2016) ggplot2: Elegant Graphics for Data Analysis. Two examples of 2-way interaction plots were inc...

Learning Shape Templates With Structured Implicit Functions

Abstract: Template 3D shapes are useful for many tasks in graphics and vision, including fitting observation data, analyzing shape collections, and transferring shape attributes. Because of the variety of geometry and topology of real-world shapes, previous methods generally use a library of hand-made templates. In this paper, we investigate learning a general shape template from data. To allow for widely varying geometry and topology, we choose an implicit surface representation based on composition of local shape elements. While long known to computer graphics, this representation has not yet been explored in the context of machine learning for vision. We show that structured implicit functions are suitable for learning and allow a network to smoothly and simultaneously fit multiple classes of shapes. The learned shape template supports applications such as shape exploration, correspondence, abstraction, interpolation, and semantic segmentation from an RGB image.

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer

Abstract: Many machine learning models operate on images, but ignore the fact that images are 2D projections formed by 3D geometry interacting with light, in a process called rendering. Enabling ML models to understand image formation might be key for generalization. However, due to an essential rasterization step involving discrete assignment operations, rendering pipelines are non-differentiable and thus largely inaccessible to gradient-based ML techniques. In this paper, we present DIB-Render, a novel rendering framework through which gradients can be analytically computed. Key to our approach is to view rasterization as a weighted interpolation, allowing image gradients to back-propagate through various standard vertex shaders within a single framework. Our approach supports optimizing over vertex positions, colors, normals, light directions and texture coordinates, and allows us to incorporate various well-known lighting models from graphics. We showcase our approach in two ML applications: single-image 3D object prediction, and 3D textured object generation, both trained using exclusively 2D supervision.

DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing

Abstract: We propose a differentiable sphere tracing algorithm to bridge the gap between inverse graphics methods and the recently proposed deep learning based implicit signed distance function. Due to the nature of the implicit function, the rendering process requires tremendous function queries, which is particularly problematic when the function is represented as a neural network. We optimize both the forward and backward passes of our rendering layer to make it run efficiently with affordable memory consumption on a commodity graphics card. Our rendering method is fully differentiable such that losses can be directly computed on the rendered 2D observations, and the gradients can be propagated backwards to optimize the 3D geometry. We show that our rendering method can effectively reconstruct accurate 3D shapes from various inputs, such as sparse depth and multi-view images, through inverse optimization. With the geometry based reasoning, our 3D shape prediction methods show excellent generalization capability and robustness against various noises.

Learning Shape Templates with Structured Implicit Functions

Abstract: Template 3D shapes are useful for many tasks in graphics and vision, including fitting observation data, analyzing shape collections, and transferring shape attributes. Because of the variety of geometry and topology of real-world shapes, previous methods generally use a library of hand-made templates. In this paper, we investigate learning a general shape template from data. To allow for widely varying geometry and topology, we choose an implicit surface representation based on composition of local shape elements. While long known to computer graphics, this representation has not yet been explored in the context of machine learning for vision. We show that structured implicit functions are suitable for learning and allow a network to smoothly and simultaneously fit multiple classes of shapes. The learned shape template supports applications such as shape exploration, correspondence, abstraction, interpolation, and semantic segmentation from an RGB image.

MGPUSim: enabling multi-GPU performance modeling and optimization

Abstract: The rapidly growing popularity and scale of data-parallel workloads demand a corresponding increase in raw computational power of Graphics Processing Units (GPUs). As single-GPU platforms struggle to satisfy these performance demands, multi-GPU platforms have started to dominate the high-performance computing world. The advent of such systems raises a number of design challenges, including the GPU microarchitecture, multi-GPU interconnect fabric, runtime libraries, and associated programming models. The research community currently lacks a publicly available and comprehensive multi-GPU simulation framework to evaluate next- generation multi-GPU system designs. In this work, we present MGPUSim, a cycle-accurate, extensively validated, multi-GPU simulator, based on AMD's Graphics Core Next 3 (GCN3) instruction set architecture. MGPUSim comes with in-built support for multi-threaded execution to enable fast, parallelized, and accurate simulation. In terms of performance accuracy, MGPUSim differs by only 5.5% on average from the actual GPU hardware. We also achieve a 3.5x and a 2.5x average speedup running functional emulation and detailed timing simulation, respectively, on a 4-core CPU, while delivering the same accuracy as serial simulation. We illustrate the flexibility and capability of the simulator through two concrete design studies. In the first, we propose the Locality API, an API extension that allows the GPU programmer to both avoid the complexity of multi-GPU programming, while precisely controlling data placement in the multi-GPU memory. In the second design study, we propose Progressive Page Splitting Migration (PASI), a customized multi-GPU memory management system enabling the hardware to progressively improve data placement. For a discrete 4-GPU system, we observe that the Locality API can speed up the system by 1.6x (geometric mean), and PASI can improve the system performance by 2.6x (geometric mean) across all benchmarks, compared to a unified 4-GPU platform.

Reversible Data Hiding in Encrypted 2D Vector Graphics Based on Reversible Mapping Model for Real Numbers

Abstract: Currently, much attention has been paid to reversible data hiding (RDH) in an encrypted domain due to the popular deployment of cloud storage. However, nearly all existing RDH schemes in the encrypted domain are proposed for raster images, and very little work has been done to 2D vector graphics, which are represented in real numbers. In this paper, a reversible mapping model for real numbers is first built. It maps the points in $R^{n}$ to $2^{s}$ non-intersecting subsets in $R^{n}$ , which guarantees that $s$ bits can be embedded into each real number. Based on the model, an RDH scheme in encrypted 2D vector graphics is put forward. In the scheme, a user encrypts 2D engineering graphics and stores them in the cloud, and then the cloud service provider can perform information hiding, extraction, and even recover the encrypted 2D vector graphics. For the authorized user, it can acquire the recovered 2D vector graphics from the cloud and obtain their original versions after decryption. For an unauthorized user, he can only acquire the encrypted 2D vector graphics with a hidden message, and only approximate 2D vector graphics can be obtained even if he knows the decryption key but does not know the hiding key. The experimental results and analysis show that it can strike a good balance between security, distortion, and capacity. It provides a new paradigm for RDH in the encrypted domain for the data represented in real numbers.

Accelerating parameter inference with graphics processing units

Abstract: Gravitational wave Bayesian parameter inference involves repeated comparisons of gravitational wave data to generic candidate predictions. Even with algorithmically efficient methods such as RIFT or reduced-order quadrature, the time needed to perform these calculations and the overall computational cost can be significant compared to the minutes to hours needed to achieve the goals of low-latency multimessenger astronomy. By translating some elements of the RIFT algorithm to operate on graphics processing units, we demonstrate substantial performance improvements, enabling dramatically reduced overall cost and latency.

Identifying natural images and computer generated graphics based on binary similarity measures of PRNU

Abstract: Aiming at the identification of natural images and computer generated graphics, an image source pipeline forensics method based on binary similarity measures of PRNU (photo response non-uniformity) is proposed. As PRNU is a unique attribute of natural images, binary similarity measures of PRNU are used to represent the differences between natural images and computer generated graphics. Binary Kullback-Leibler distance, binary minimum histogram distance, binary absolute histogram distance and binary mutual entropy are calculated from PRNU in RGB three channels. With a total of 36 dimensions of features, LIBSVM is used for classification. Experimental results and analysis indicate that it can achieve an average identification accuracy of 99.83%, and the capability of identifying natural images and computer generated graphics is balanced. Meanwhile, it is robust against JPEG compression, rotation and additive noise.

Synthesizing Facial Photometries and Corresponding Geometries Using Generative Adversarial Networks

Abstract: Artificial data synthesis is currently a well-studied topic with useful applications in data science, computer vision, graphics, and many other fields. Generating realistic data is especially challenging, since human perception is highly sensitive to non-realistic appearance. In recent times, new levels of realism have been achieved by advances in GAN training procedures and architectures. These successful models, however, are tuned mostly for use with regularly sampled data such as images, audio, and video. Despite the successful application of the architecture on these types of media, applying the same tools to geometric data poses a far greater challenge. The study of geometric deep learning is still a debated issue within the academic community, as the lack of intrinsic parametrization inherent to geometric objects prohibits the direct use of convolutional filters, a main building block of today’s machine learning systems.In this article, we propose a new method for generating realistic human facial geometries coupled with overlayed textures. We circumvent the parametrization issue by utilizing a specialized non-rigid alignment procedure, and imposing a global mapping from our data to the unit rectangle. This mapping enables the representation of our geometric data as regularly sampled 2D images. We further discuss how to design such a mapping to control the distortion and conserve area within the target image. By representing geometric textures and geometries as images, we are able to use advanced GAN methodologies to generate new plausible textures and geometries. We address the often-neglected topic of relationship between texture and geometry and propose different methods for fitting generated geometries to generated textures. In addition, we widen the scope of our discussion and offer a new method for training GAN models on partially corrupted data. Finally, we provide empirical evidence demonstrating our generative model’s ability to produce examples of new facial identities, independent from the training data, while maintaining a high level of realism—two traits that are often at odds.

A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning

Abstract: Detecting scene text of arbitrary shapes has been a challenging task over the past years. In this paper, we propose a novel segmentation-based text detector, namely SAST, which employs a context attended multi-task learning framework based on a Fully Convolutional Network (FCN) to learn various geometric properties for the reconstruction of polygonal representation of text regions. Taking sequential characteristics of text into consideration, a Context Attention Block is introduced to capture long-range dependencies of pixel information to obtain a more reliable segmentation. In post-processing, a Point-to-Quad assignment method is proposed to cluster pixels into text instances by integrating both high-level object knowledge and low-level pixel information in a single shot. Moreover, the polygonal representation of arbitrarily-shaped text can be extracted with the proposed geometric properties much more effectively. Experiments on several benchmarks, including ICDAR2015, ICDAR2017-MLT, SCUT-CTW1500, and Total-Text, demonstrate that SAST achieves better or comparable performance in terms of accuracy. Furthermore, the proposed algorithm runs at 27.63 FPS on SCUT-CTW1500 with a Hmean of 81.0% on a single NVIDIA Titan Xp graphics card, surpassing most of the existing segmentation-based methods.

A framework for fast automatic image cropping based on deep saliency map detection and gaussian filter

Abstract: Extracting robust visual saliency map and image cropping are fundamental problems in computer vision, graphics, and so on. It is not easy task to accurately detect and crop the entire salient objec...

Unveiling your keystrokes: A Cache-based Side-channel Attack on Graphics Libraries.

Abstract: Operating systems use shared memory to improve performance. However, as shown in recent studies, attackers can exploit CPU cache side-channels associated with shared memory to extract sensitive information. The attacks that were previously attempted typically only detect the presence of a certain operation and require significant manual analysis to identify and evaluate their effectiveness. Moreover, very few of them target graphics libraries which are commonly used, but difficult to attack. In this paper, we consider the execution time of shared libraries as the side-channel, and showcase a completely automated technique to discover and select exploitable side-channels on shared graphics libraries. In essence, we first collect the cache lines accessed by a victim process during different key presses offline, and then use machine learning to infer the best cache lines (e.g., easily measurable, robust to noise, high information leakage) for a flush and reload attack. We are able to discover effective strategies to classify what keys have been pressed. Using this approach, we not only preclude the need for manual analyses of code and traces — the automated system discovered many previously unknown sidechannels of the type we are interested in, but also achieve high precision in terms of inferring the sensitive information entered on desktop and Android platforms. We show that our approach infers the passwords with lowercase letters and numbers 10,000 1,000,000 times faster than random guessing. For a large fraction of PINs consisting of 4 to 6 digits, we are able to infer them within 20 and 80 guesses respectively. Finally, we suggest ways to mitigate these attacks.

Science–graphic art partnerships to increase research impact

Abstract: Graphics are becoming increasingly important for scientists to effectively communicate their findings to broad audiences, but most researchers lack expertise in visual media. We suggest collaboration between scientists and graphic designers as a way forward and discuss the results of a pilot project to test this type of collaboration.

Research on garment pattern design based on fractal graphics

Abstract: This paper firstly analyzes the basic principle of generating fractal art graphics and the artistic features of graphics and then uses scientific visualization method to innovate and improve the theoretical model used in this paper. The generation principle and graphic characteristics of fractal graphics of complex dynamic system and L-system are mainly analyzed, and two kinds of art graphics—flower art graphics and geometric art graphics—have been developed. On this basis, the generated artistic figures are designed for the second time and then applied to the design of clothing patterns. By using MATLAB programming software to generate art graphics conforming to a specific style, combined with image processing software Photoshop to process and redesign the generated graphics, these art graphics can assist the design of clothing printing patterns and make patterns applicable for clothing fabrics. Finally, the fractal pattern theory is applied to silk scarves design and clothing fabric design through digital printing technology, which can fully reflect the practicability and superiority of clothing pattern design based on the fractal theory. Based on the experimental result, it shows that it is completely feasible to design clothing fabric printing patterns based on fractal theory, and the unusual artistic patterns designed have very considerable practical value. In addition, this method encourages creativity in the garment pattern design process and accelerates new design generation.

A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning

Abstract: Detecting scene text of arbitrary shapes has been a challenging task over the past years. In this paper, we propose a novel segmentation-based text detector, namely SAST, which employs a context attended multi-task learning framework based on a Fully Convolutional Network (FCN) to learn various geometric properties for the reconstruction of polygonal representation of text regions. Taking sequential characteristics of text into consideration, a Context Attention Block is introduced to capture long-range dependencies of pixel information to obtain a more reliable segmentation. In post-processing, a Point-to-Quad assignment method is proposed to cluster pixels into text instances by integrating both high-level object knowledge and low-level pixel information in a single shot. Moreover, the polygonal representation of arbitrarily-shaped text can be extracted with the proposed geometric properties much more effectively. Experiments on several benchmarks, including ICDAR2015, ICDAR2017-MLT, SCUT-CTW1500, and Total-Text, demonstrate that SAST achieves better or comparable performance in terms of accuracy. Furthermore, the proposed algorithm runs at 27.63 FPS on SCUT-CTW1500 with a Hmean of 81.0% on a single NVIDIA Titan Xp graphics card, surpassing most of the existing segmentation-based methods.

Effects of Dark Mode on Visual Fatigue and Acuity in Optical See-Through Head-Mounted Displays

Abstract: Light-on-dark color schemes, so-called “Dark Mode,” are becoming more and more popular over a wide range of display technologies and application fields. Many people who have to look at computer screens for hours at a time, such as computer programmers and computer graphics artists, indicate a preference for switching colors on a computer screen from dark text on a light background to light text on a dark background due to perceived advantages related to visual comfort and acuity, specifically when working in low-light environments. In this paper, we investigate the effects of dark mode color schemes in the field of optical see-through head-mounted displays (OST-HMDs), where the characteristic “additive” light model implies that bright graphics are visible but dark graphics are transparent. We describe a human-subject study in which we evaluated a normal and inverted color mode in front of different physical backgrounds and among different lighting conditions. Our results show that dark mode graphics on OST-HMDs have significant benefits for visual acuity, fatigue, and usability, while user preferences depend largely on the lighting in the physical environment. We discuss the implications of these effects on user interfaces and applications.

RoboGraphics: Dynamic Tactile Graphics Powered by Mobile Robots

Abstract: Tactile graphics are a common way to present information to people with vision impairments. Tactile graphics can be used to explore a broad range of static visual content but aren't well suited to representing animation or interactivity. We introduce a new approach to creating dynamic tactile graphics that combines a touch screen tablet, static tactile overlays, and small mobile robots. We introduce a prototype system called RoboGraphics and several proof-of-concept applications. We evaluated our prototype with seven participants with varying levels of vision, comparing the RoboGraphics approach to a flat screen, audio-tactile interface. Our results show that dynamic tactile graphics can help visually impaired participants explore data quickly and accurately.

Graphical Data Analysis with R

Abstract: Setting the Scene Graphics in action Introduction What is graphical data analysis (GDA)? Using this book, the R code in it, and the book's webpage Brief Review of the Literature and Background Materials Literature review Interactive graphics Other graphics software Websites Datasets Statistical texts Examining Continuous Variables Introduction What features might continuous variables have? Looking for features Comparing distributions by subgroups What plots are there for individual continuous variables? Plot options Modelling and testing for continuous variables Displaying Categorical Data Introduction What features might categorical variables have? Nominal data-no fixed category order Ordinal data-fixed category order Discrete data-counts and integers Formats, factors, estimates, and barcharts Modelling and testing for categorical variables Looking for Structure: Dependency Relationships and Associations Introduction What features might be visible in scatterplots? Looking at pairs of continuous variables Adding models: lines and smooths Comparing groups within scatterplots Scatterplot matrices for looking at many pairs of variables Scatterplot options Modelling and testing for relationships between variables Investigating Multivariate Continuous Data Introduction What is a parallel coordinate plot (pcp)? Features you can see with parallel coordinate plots Interpreting clustering results Parallel coordinate plots and time series Parallel coordinate plots for indices Options for parallel coordinate plots Modelling and testing for multivariate continuous data Parallel coordinate plots and comparing model results Studying Multivariate Categorical Data Introduction Data on the sinking of the Titanic What is a mosaicplot? Different mosaicplots for different questions of interest Which mosaicplot is the right one? Additional options Modelling and testing for multivariate categorical data Getting an Overview Introduction Many individual displays Multivariate overviews Multivariate overviews for categorical variables Graphics by group Modelling and testing for overviews Graphics and Data Quality: How Good Are the Data? Introduction Missing values Outliers Modelling and testing for data quality Comparisons, Comparisons, Comparisons Introduction Making comparisons Making visual comparisons Comparing group effects graphically Comparing rates visually Graphics for comparing many subsets Graphics principles for comparisons Modelling and testing for comparisons Graphics for Time Series Introduction Graphics for a single time series Multiple series Special features of time series Alternative graphics for time series R classes and packages for time series Modelling and testing time series Ensemble Graphics and Case Studies Introduction What is an ensemble of graphics? Combining different views-a case study example Case studies Some Notes on Graphics with R Graphics systems in R Loading datasets and packages for graphical analysis Graphics conventions in statistics What is a graphic anyway? Options for all graphics Some R graphics advice and coding tips Other graphics Large datasets Perfecting graphics Summary Data analysis and graphics Key features of GDA Strengths and weaknesses of GDA Recommendations for GDA References General Index Datasets Index

GIF2Video: Color Dequantization and Temporal Interpolation of GIF images

Abstract: Graphics Interchange Format (GIF) is a highly portable graphics format that is ubiquitous on the Internet. Despite their small sizes, GIF images often contain undesirable visual artifacts such as flat color regions, false contours, color shift, and dotted patterns. In this paper, we propose GIF2Video, the first learning-based method for enhancing the visual quality of GIFs in the wild. We focus on the challenging task of GIF restoration by recovering information lost in the three steps of GIF creation: frame sampling, color quantization, and color dithering. We first propose a novel CNN architecture for color dequantization. It is built upon a compositional architecture for multi-step color correction, with a comprehensive loss function designed to handle large quantization errors. We then adapt the SuperSlomo network for temporal interpolation of GIF frames. We introduce two large datasets, namely GIF-Faces and GIF-Moments, for both training and evaluation. Experimental results show that our method can significantly improve the visual quality of GIFs, and outperforms direct baseline and state-of-the-art approaches.

Graphics-aware Power Governing for Mobile Devices

Abstract: Graphics increasingly play a key role in modern mobile devices. The graphics pipeline requires a close relationship between the CPU and the GPU to ensure energy efficiency and the user's quality of experience (QoE). Our preliminary analysis showed that the current techniques employed to achieve energy efficiency in the Android graphics pipeline are not optimized especially in the frame generation process. In this paper, we aim to improve the energy efficiency of the Android graphics pipeline without degrading the user's QoE. To achieve this goal, we studied the internals of the Android graphics pipeline and observed the energy inefficiency in the existing governing framework of the CPU and GPU. Based on the findings, we propose three techniques for addressing energy inefficiency: (1) aggressively capping the maximum CPU frequency, (2) lowering the CPU frequency by raising the GPU minimum frequency, and (3) allocating the frame rendering-related threads in the energy-efficient CPU cores. These techniques are integrated into a single governing framework, called the GFX Governor, and implemented in the newest Android-based smartphones. Experimental results show that without hampering the user's QoE the average energy consumption of Nexus 6P, Pixel XL, and Pixel 2 XL is reduced at the device level by 24.2%, 18.6%, and 13.7%, respectively, for the 60 chosen applications. We also analyzed the efficacy of the proposed technique in comparison with the state-of-the-art Energy-Aware Scheduling (EAS) implemented in the latest smartphone.

A system for generating complex physically accurate sensor images for automotive applications

Abstract: We describe an open-source simulator that creates sensor irradiance and sensor images of typical automotive scenes in urban settings. The purpose of the system is to support camera design and testing for automotive applications. The user can specify scene parameters (e.g., scene type, road type, traffic density, time of day) to assemble a large number of random scenes from graphics assets stored in a database. The sensor irradiance is generated using quantitative computer graphics methods, and the sensor images are created using image systems sensor simulation. The synthetic sensor images have pixel level annotations; hence, they can be used to train and evaluate neural networks for imaging tasks, such as object detection and classification. The end-to-end simulation system supports quantitative assessment, from scene to camera to network accuracy, for automotive applications.

Design and analysis of an effective graphics collaborative editing system

Abstract: With the rapid development of computer-supported cooperative work (CSCW) technology, graphical collaborative editing plays an increasingly important role in CSCW. The most important technique in graphics co-editing is the consistency of graphics co-editing, which mainly includes causality consistency, consistency of results, and consistency of intention. Most of the previous research was abstract and ineffective, lacking theoretical depth and scalability. However, because the algorithm proposed in this paper can solve the contradictions in the consistency of graphical collaborative editing, the research in this paper has particularity, and the results will be proven by the experiment described in the paper. In order to solve the consistency conflict problem of graphic collaborative editing, the common graphics collaborative editing algorithm (CGCE algorithm) is proposed. It is proposed not only to perfect and expand the definition of graphics collaborative editing but also to merge with HTML5 Canvas, WebSocket, jQuery, Node.js and other network programming languages and technologies. The graphic collaborative editing based on the design and implementation of this paper can effectively solve the consistency conflict problem of many users during the collaborative editing of graphics, which ensures that the graphics of each graphical collaborative editing interface is consistent and the collaborative work can achieve the desired effect.

HaraliCU: GPU-Powered Haralick Feature Extraction on Medical Images Exploiting the Full Dynamics of Gray-Scale Levels

Abstract: Image texture extraction and analysis are fundamental steps in Computer Vision. In particular, considering the biomedical field, quantitative imaging methods are increasingly gaining importance since they convey scientifically and clinically relevant information for prediction, prognosis, and treatment response assessment. In this context, radiomic approaches are fostering large-scale studies that can have a significant impact in the clinical practice. In this work, we focus on Haralick features, the most common and clinically relevant descriptors. These features are based on the Gray-Level Co-occurrence Matrix (GLCM), whose computation is considerably intensive on images characterized by a high bit-depth (e.g., 16 bits), as in the case of medical images that convey detailed visual information. We propose here HaraliCU, an efficient strategy for the computation of the GLCM and the extraction of an exhaustive set of the Haralick features. HaraliCU was conceived to exploit the parallel computation capabilities of modern Graphics Processing Units (GPUs), allowing us to achieve up to \(\sim \!20\times \) speed-up with respect to the corresponding C++ coded sequential version. Our GPU-powered solution highlights the promising capabilities of GPUs in the clinical research.

Deep learning-based damage detection of miter gates using synthetic imagery from computer graphics

Abstract: Structural inspections of large, difficult-to-access infrastructure like dams and bridges are often time-consuming, laborious and unsafe. In the United States, federal and state agencies responsible for managing such infrastructure assets are investigating the use of unmanned aerial vehicles (UAV) to allow for remote data acquisition. Processing the large amounts of data acquired by the UAV remains a challenging task. Over the past four years, researchers have been investigating deep learning methods for automated damage detection through image classification and more recently, the use of semantic segmentation where each pixel in the image is given a certain label. For such algorithms to work effectively, deep neural networks need to be trained on large datasets of labelled images. The generation of these labels for semantic segmentation is a very tedious process as it requires each pixel in the image to be labelled. This paper investigates the use of computer graphics to automatically generate synthetic imagery for the purposes of training deep learning algorithms for vision-based damage detection using semantic segmentation. The significant advantage of this is the automatic generation of precise semantic labels due to the implicit information in the developed graphics models. Parametric noise-based graphics texture models are created for defects such as cracks and corrosion and for other features such as vegetation growth, and dirt. The parameterization of the texture models allows for generation of a range of different surface conditions, thereby providing increased flexibility over data generation. To demonstrate the benefits of the proposed methodology for synthetic data generation a virtual environment of inland navigation infrastructure including miter gates and tainter gate dams is created. The developed texture models are applied to the virtual environment to produce a photo-realistic model. Synthetic image data is then rendered from the developed model and used to demonstrate the efficacy for training deep learning-based semantic segmentation algorithms for damage detection.1

Emerald: graphics modeling for SoC systems

Abstract: Mobile systems-on-chips (SoCs) have become ubiquitous computing platforms, and, in recent years, they have become increasingly heterogeneous and complex. A typical SoC includes CPUs, graphics processor units (GPUs), image processors, video encoders/decoders, AI engines, digital signal processors (DSPs) and 2D engines among others [33], [70], [71]. One of the most significant SoC units in terms of both off-chip memory bandwidth and SoC die area is the GPU. In this paper, we present Emerald, a simulator that builds on existing tools to provide a unified model for graphics and GPGPU applications. Emerald enables OpenGL (v4.5) and OpenGL ES (v3.2) shaders to run on GPGPU-Sim's timing model and is integrated with gem5 and Android to simulate full SoCs. Emerald thus provides a platform for studying system-level SoC interactions while including the impact of graphics. We present two case studies using Emerald. First, we use Emerald's full-system mode to highlight the importance of system-wide interactions by studying and analyzing memory organization and scheduling schemes for SoC systems. Second, we use Emerald's standalone mode to evaluate a novel mechanism for balancing the graphics shading work assigned to each GPU core.