Showing papers in "IEEE Computer Graphics and Applications in 2022"

Visual Analytics for Human-Centered Machine Learning

Abstract: We introduce a new research area in visual analytics (VA) aiming to bridge existing gaps between methods of interactive machine learning (ML) and eXplainable Artificial Intelligence (XAI), on one side, and human minds, on the other side. The gaps are, first, a conceptual mismatch between ML/XAI outputs and human mental models and ways of reasoning, and second, a mismatch between the information quantity and level of detail and human capabilities to perceive and understand. A grand challenge is to adapt ML and XAI to human goals, concepts, values, and ways of thinking. Complementing the current efforts in XAI towards solving this challenge, VA can contribute by exploiting the potential of visualization as an effective way of communicating information to humans and a strong trigger of human abstractive perception and thinking. We propose a cross-disciplinary research framework and formulate research directions for VA.

Visual Analytics for Human-Centered Machine Learning

Abstract: We introduce a new research area in visual analytics (VA) aiming to bridge existing gaps between methods of interactive machine learning (ML) and eXplainable Artificial Intelligence (XAI), on one side, and human minds, on the other side. The gaps are, first, a conceptual mismatch between ML/XAI outputs and human mental models and ways of reasoning, and second, a mismatch between the information quantity and level of detail and human capabilities to perceive and understand. A grand challenge is to adapt ML and XAI to human goals, concepts, values, and ways of thinking. Complementing the current efforts in XAI towards solving this challenge, VA can contribute by exploiting the potential of visualization as an effective way of communicating information to humans and a strong trigger of human abstractive perception and thinking. We propose a cross-disciplinary research framework and formulate research directions for VA.

MAGES 4.0: Accelerating the World’s Transition to VR Training and Democratizing the Authoring of the Medical Metaverse

Abstract: In this work, we propose MAGES 4.0, a novel software development kit to accelerate the creation of collaborative medical training applications in virtual/augmented reality (VR/AR). Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different VR/AR as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master–apprentice medical training model. Our platform incorporates, in a nutshell, the following novelties: 1) 5G edge-cloud remote rendering and physics dissection layer, 2) realistic real-time simulation of organic tissues as soft-bodies under 10 ms, 3) a highly realistic cutting and tearing algorithm, 4) neural network assessment for user profiling and, 5) a VR recorder to record and replay or debrief the training simulation from any perspective.

Situated Visual Analysis and Live Monitoring for Manufacturing

Abstract: Modern machines continuously log status reports over long periods of time, which are valuable data to optimize working routines. Data visualization is a commonly used tool to gain insights into these data, mostly in retrospective (e.g., to determine causal dependencies between the faults of different machines). We present an approach to bring such visual analyses to the shop floor to support reacting to faults in real time. This approach combines spatio-temporal analyses of time series using a handheld touch device with augmented reality for live monitoring. Important information augments machines directly in their real-world context, and detailed logs of current and historical events are displayed on the handheld device. In collaboration with an industry partner, we designed and tested our approach on a live production line to obtain feedback from operators. We compare our approach for monitoring and analysis with existing solutions that are currently deployed.

Situated Visual Analysis and Live Monitoring for Manufacturing

Abstract: Modern machines continuously log status reports over long periods of time, which are valuable data to optimize working routines. Data visualization is a commonly used tool to gain insights into these data, mostly in retrospective (e.g., to determine causal dependencies between the faults of different machines). We present an approach to bring such visual analyses to the shop floor to support reacting to faults in real time. This approach combines spatio-temporal analyses of time series using a handheld touch device with augmented reality for live monitoring. Important information augments machines directly in their real-world context, and detailed logs of current and historical events are displayed on the handheld device. In collaboration with an industry partner, we designed and tested our approach on a live production line to obtain feedback from operators. We compare our approach for monitoring and analysis with existing solutions that are currently deployed.

Visualization for Architecture, Engineering, and Construction: Shaping the Future of Our Built World

Abstract: Our built world is one of the most important factors for a livable future, accounting for massive impact on resource and energy use, as well as climate change, but also the social and economic aspects that come with population growth. The architecture, engineering, and construction industry is facing the challenge that it needs to substantially increase its productivity, let alone the quality of buildings of the future. In this article, we discuss these challenges in more detail, focusing on how digitization can facilitate this transformation of the industry, and link them to opportunities for visualization and augmented reality research. We illustrate solution strategies for advanced building systems based on wood and fiber.

VisHikers’ Guide to Evaluation: Competing Considerations in Study Design

Abstract: In this Viewpoint article, we describe the persistent tensions between various camps on the “right” way to conduct evaluations in visualization. Visualization as a field is the amalgamation of cognitive and perceptual sciences and computer graphics, among others. As a result, the relatively disjointed lineages in visualization understandably approach the topic of evaluation very differently. It is both a blessing and a curse to our field. It is a blessing, because the collaboration of diverse perspectives is the breeding ground of innovation. Yet it is a curse, because as a community, we have yet to resolve an appreciation for differing perspectives on the topic of evaluation. We explicate these differing expectations and conventions to appreciate the spectrum of evaluation design decisions. We describe some guiding questions that researchers may consider when designing evaluations to navigate differing readers’ evaluation expectations.

Visualization for Architecture, Engineering, and Construction: Shaping the Future of Our Built World

Abstract: Our built world is one of the most important factors for a livable future, accounting for massive impact on resource and energy use, as well as climate change, but also the social and economic aspects that come with population growth. The architecture, engineering, and construction industry is facing the challenge that it needs to substantially increase its productivity, let alone the quality of buildings of the future. In this article, we discuss these challenges in more detail, focusing on how digitization can facilitate this transformation of the industry, and link them to opportunities for visualization and augmented reality research. We illustrate solution strategies for advanced building systems based on wood and fiber.

VisHikers’ Guide to Evaluation: Competing Considerations in Study Design

Abstract: In this Viewpoint article, we describe the persistent tensions between various camps on the "right" way to conduct evaluations in visualization. Visualization as a field is the amalgamation of cognitive and perceptual sciences and computer graphics, among others. As a result, the relatively disjointed lineages in visualization understandably approach the topic of evaluation very differently. It is both a blessing and a curse to our field. It is a blessing, because the collaboration of diverse perspectives is the breeding ground of innovation. Yet it is a curse, because as a community, we have yet to resolve an appreciation for differing perspectives on the topic of evaluation. We explicate these differing expectations and conventions to appreciate the spectrum of evaluation design decisions. We describe some guiding questions that researchers may consider when designing evaluations to navigate differing readers' evaluation expectations.

Identifying Deception as a Critical Component of Visualization Literacy

Abstract: The ability to recognize misleading data visualizations is a key aspect of visualization literacy. In this article, we argue that learning to successfully identify a deceptive graphic requires strategies that deliberately force learners to take an active role in the visualization process. We describe a series of experiments where three groups of learners were shown various deceptive graphics and asked to answer a series of questions. Three different interventions were analyzed to compare the educational effectiveness of the strategies used to engage learners into the process of identifying deceptive visualizations. Our results suggest that the ability to identify deceptive visualizations must be explicitly taught as a core element of visualization literacy. Although both traditional and self-learning approaches are beneficial, the more active the intervention, the higher its educational effectiveness.

BitConduite: Exploratory Visual Analysis of Entity Activity on the Bitcoin Network

Abstract: We present BitConduite, a visual analytics approach for explorative analysis of financial activity within the Bitcoin network, offering a view on transactions aggregated by entities, i.e., by individuals, companies, or other groups actively using Bitcoin. BitConduite makes Bitcoin data accessible to nontechnical experts through a guided workflow around entities analyzed according to several activity metrics. Analyses can be conducted at different scales, from large groups of entities down to single entities. BitConduite also enables analysts to cluster entities to identify groups of similar activities as well as to explore characteristics and temporal patterns of transactions. To assess the value of our approach, we collected feedback from domain experts.

VisLitE: Visualization Literacy and Evaluation

Abstract: With the widespread advent of visualization techniques to convey complex data, visualization literacy (VL) is growing in importance. Two noteworthy facets of literacy are user understanding and the discovery of visual patterns with the help of graphical representations. The research literature on VL provides useful guidance and opportunities for further studies in this field. This introduction summarizes and presents research on VL that examines how well users understand basic and advanced data representations. To the best of our knowledge, this is the first tutorial article on interactive VL. We describe evaluation categories of existing relevant research into unique subject groups that facilitate and inform comparisons of literacy literature and provide a starting point for interested readers. In addition, the introduction also provides an overview of the various evaluation techniques used in this field of research and their challenging nature. Our introduction provides researchers with unexplored directions that may lead to future work. This starting point serves as a valuable resource for beginners interested in the topic of VL.

Deep Synthesis of Cloud Lighting

Abstract: Current appearance models for the sky are able to represent clear-sky illumination to a high degree of accuracy. However, these models all lack a common feature of real skies: clouds. These are an essential component for many applications which rely on realistic skies, such as image editing and synthesis. While clouds can be added to existing sky models through rendering, this is hard to achieve due to the difficulties of representing clouds and the complexities of volumetric light transport. In this work, an alternative approach to this problem is proposed whereby clouds are synthesized using a learned data-driven representation. This leverages a captured collection of high dynamic range cloudy sky imagery, and combines this dataset with clear-sky models to produce plausible cloud appearance from a coarse representation of cloud positions. This representation is artist controllable, allowing for novel cloudscapes to be rapidly synthesized, and used for lighting virtual environments.

SUBPLEX: A Visual Analytics Approach to Understand Local Model Explanations at the Subpopulation Level

Abstract: Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts, such as transport control, financial activities, and medical diagnosis. While local explanation techniques are popular methods to interpret ML models on a single instance, they do not scale to the understanding of a model’s behavior on the whole dataset. In this article, we outline the challenges and needs of visually analyzing local explanations and propose SUBPLEX, a visual analytics approach to help users understand local explanations with subpopulation visual analysis. SUBPLEX provides steerable clustering and projection visualization techniques that allow users to derive interpretable subpopulations of local explanations with users’ expertise. We evaluate our approach through two use cases and experts’ feedback.

Is the Perceived Comfort With CG Characters Increasing With Their Novelty?

Abstract: Realistic characters from movies and games can cause strangeness and involuntary feelings in viewers, an effect known as the uncanny valley (UV). This article revisits the central UV hypothesis, proposed by Masahiro Mori in 1970, to evaluate its impact on people's perception of characters created using computer graphics (CG). More precisely, our goal is to answer the following questions: 1) Are people feeling more comfortable with more recent CG characters than the older ones? 2) Does charisma or familiarity with virtual humans correlate with perceived comfort? To answer these questions, we first replicated an experiment from 2012 and compared the perception concerning CG characters then and now, and then we included images of more recent CG characters in our analysis. Our results indicate that the perceived comfort increased over time when comparing the characters of 2012 and 2020. However, it did not change significantly for the characters of 2012. In addition, we found a correlation between perceived charisma and familiarity, at all levels of realism, and between charisma and comfort. Interestingly, more charisma was perceived in videos than in images. In addition, unrealistic characters were also perceived as more charismatic.

Supporting Visual Exploration of Iterative Job Scheduling

Abstract: We consider the general problem known as job shop scheduling, in which multiple jobs consist of sequential operations that need to be executed or served by appropriate machines having limited capacities. For example, train journeys (jobs) consist of moves and stops (operations) to be served by rail tracks and stations (machines). A schedule is an assignment of the job operations to machines and times where and when they will be executed. The developers of computational methods for job scheduling need tools enabling them to explore how their methods work. At a high level of generality, we define the system of pertinent exploration tasks and a combination of visualizations capable of supporting the tasks. We provide general descriptions of the purposes, contents, visual encoding, properties, and interactive facilities of the visualizations and illustrate them with images from an example implementation in air traffic management. We justify the design of the visualizations based on the tasks, principles of creating visualizations for pattern discovery, and scalability requirements. The outcomes of our research are sufficiently general to be of use in a variety of applications.

VisLitE: Visualization Literacy and Evaluation

Abstract: With the widespread advent of visualization techniques to convey complex data, visualization literacy (VL) is growing in importance. Two noteworthy facets of literacy are user understanding and the discovery of visual patterns with the help of graphical representations. The research literature on VL provides useful guidance and opportunities for further studies in this field. This introduction summarizes and presents research on VL that examines how well users understand basic and advanced data representations. To the best of our knowledge, this is the first tutorial article on interactive VL. We describe evaluation categories of existing relevant research into unique subject groups that facilitate and inform comparisons of literacy literature and provide a starting point for interested readers. In addition, the introduction also provides an overview of the various evaluation techniques used in this field of research and their challenging nature. Our introduction provides researchers with unexplored directions that may lead to future work. This starting point serves as a valuable resource for beginners interested in the topic of VL.

DETOXER: A Visual Debugging Tool With Multiscope Explanations for Temporal Multilabel Classification

Abstract: In many applications, developed deep-learning models need to be iteratively debugged and refined to improve the model efficiency over time. Debugging some models, such as temporal multilabel classification (TMLC) where each data point can simultaneously belong to multiple classes, can be especially more challenging due to the complexity of the analysis and instances that need to be reviewed. In this article, focusing on video activity recognition as an application of TMLC, we propose DETOXER, an interactive visual debugging system to support finding different error types and scopes through providing multiscope explanations.

Visualizing Internal Sustainability Efforts in Big Companies

Abstract: Internal sustainability efforts (ISE) refer to a wide range of internal corporate policies focused on employees. They promote, for example, work-life balance, gender equality, and a harassment-free working environment. At times, however, companies fail to keep their promises by not publicizing truthful reports on these practices, or by overlooking employees voices on how these practices are implemented. To partly fix that, we developed a deep-learning (DL) framework that scored fourth fifths of the S&P 500 companies in terms of six ISEs, and a web-based system that engages users in a learning and reflection process about these ISEs. We evaluated the system in two crowdsourced studies with 421 participants, and compared our treemap visualization with a baseline textual representation. We found that our interactive treemap increased by up to 7% our participants opinion change about ISEs, demonstrating its potential in machine-learning (ML) driven visualizations.

Supporting Visual Exploration of Iterative Job Scheduling

Abstract: We consider the general problem known as job shop scheduling, in which multiple jobs consist of sequential operations that need to be executed or served by appropriate machines having limited capacities. For example, train journeys (jobs) consist of moves and stops (operations) to be served by rail tracks and stations (machines). A schedule is an assignment of the job operations to machines and times where and when they will be executed. The developers of computational methods for job scheduling need tools enabling them to explore how their methods work. At a high level of generality, we define the system of pertinent exploration tasks and a combination of visualizations capable of supporting the tasks. We provide general descriptions of the purposes, contents, visual encoding, properties, and interactive facilities of the visualizations and illustrate them with images from an example implementation in air traffic management. We justify the design of the visualizations based on the tasks, principles of creating visualizations for pattern discovery, and scalability requirements. The outcomes of our research are sufficiently general to be of use in a variety of applications.

<i>DETOXER:</i> A Visual Debugging Tool With Multiscope Explanations for Temporal Multilabel Classification

Abstract: In many applications, developed deep-learning models need to be iteratively debugged and refined to improve the model efficiency over time. Debugging some models, such as temporal multilabel classification (TMLC) where each data point can simultaneously belong to multiple classes, can be especially more challenging due to the complexity of the analysis and instances that need to be reviewed. In this article, focusing on video activity recognition as an application of TMLC, we propose DETOXER , an interactive visual debugging system to support finding different error types and scopes through providing multiscope explanations.

Stress Visualization for Interface Optimization of a Hybrid Component using Surface Tensor Spines

Abstract: In lightweight construction, engineers focus on designing and optimizing lightweight components without compromising their strength and durability. In this process, materials such as polymers are commonly considered for a hybrid construction, or even used as a complete replacement. In this work, we focus on a hybrid component design combining metal and carbon fiber reinforced polymer parts. Here, engineers seek to optimize the interface connection between a polymer and a metal part through the placement of load transmission elements in a mechanical millimetric mesoscale level. To assist engineers in the placement and design process, we extend tensor spines, a 3-D tensor-based visualization technique, to surfaces. This is accomplished by combining texture-based techniques with tensor data. Moreover, we apply a parametrization based on a remeshing process to provide visual guidance during the placement. Finally, we demonstrate and discuss real test cases to validate the benefit of our approach.

DLA-VPS: Deep-Learning-Assisted Visual Parameter Space Analysis of Cosmological Simulations

Abstract: Cosmologists often build a mathematics simulation model to study the observed universe. However, running a high-fidelity simulation is time consuming and thus can inconvenience the analysis. This is especially so when the analysis involves trying out a large number of simulation input parameter configurations. Therefore, selecting an input parameter configuration that can meet the needs of an analysis task has become an important part of the analysis process. In this work, we propose an interactive visual system that efficiently helps users understand the parameter space related to their cosmological data. Our system utilizes a GAN-based surrogate model to reconstruct the simulation outputs without running the expensive simulation. We also extract information learned by the deep neural-network-based surrogate models to facilitate the parameter space exploration. We demonstrate the effectiveness of our system via multiple case studies. These case study results demonstrate valuable simulation input parameter configuration and subregion analyses.

Situated VR: Toward a Congruent Hybrid Reality Without Experiential Artifacts

Abstract: The vision of extended reality (XR) systems is living in a world where real and virtual elements seamlessly and contextually augment experiences of ourselves and the worlds we inhabit. While this integration promises exciting opportunities for the future of XR, it comes with the risk of experiential distortions and feelings of dissociation, especially related to virtual reality (VR). When transitioning from a virtual world to the real world, users report of experiential structures that linger on, as sort of after images, causing disruptions in their daily life. In this work, we define these atypical experiences as experiential artifacts (EAs) and present preliminary results from an informal survey conducted online with 76 VR users to highlight different types of artifacts and their durations. To avoid disruptions caused by these artifacts and simultaneously increase the user’s sense of presence, we propose the idea of situated VR, which blends the real and virtual in novel ways that can reduce incongruencies between the two worlds. We discuss the implications of EAs, and through examples from our own work in building hybrid experiences, we demonstrate the potential and relevance of situated VR in the design of a future, more immersive, artifact-free hybrid reality.

Visual Analytics Platform for Centralized COVID-19 Digital Contact Tracing

Abstract: The COVID-19 pandemic and its dramatic worldwide impact has required global multidisciplinary actions to mitigate its effects. Mobile phone activity-based digital contact tracing (DCT) via Bluetooth low energy technology has been considered a powerful pandemic monitoring tool, yet it sparked a controversial debate about privacy risks for people. In order to explore the potential benefits of a DCT system in the context of occupational risk prevention, this article presents the potential of visual analytics methods to summarize and extract relevant information from complex DCT data collected during a long-term experiment at our research center. Visual tools were combined with quantitative metrics to provide insights into contact patterns among volunteers. Results showed that crucial actors, such as participants acting as bridges between groups could be easily identified—ultimately allowing for making more informed management decisions aimed at containing the potential spread of a disease.

<i>SUBPLEX</i>: A Visual Analytics Approach to Understand Local Model Explanations at the Subpopulation Level

Abstract: Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts, such as transport control, financial activities, and medical diagnosis. While local explanation techniques are popular methods to interpret ML models on a single instance, they do not scale to the understanding of a model’s behavior on the whole dataset. In this article, we outline the challenges and needs of visually analyzing local explanations and propose SUBPLEX , a visual analytics approach to help users understand local explanations with subpopulation visual analysis. SUBPLEX provides steerable clustering and projection visualization techniques that allow users to derive interpretable subpopulations of local explanations with users’ expertise. We evaluate our approach through two use cases and experts’ feedback.

Learning Perceptual Aesthetics of 3-D Shapes From Multiple Views

Abstract: The quantification of 3-D shape aesthetics has so far focused on specific shape features and manually defined criteria such as the curvature and the rule of thirds. In this article, we built a model of 3-D shape aesthetics directly from human aesthetics preference data and show it to be well aligned with human perception of aesthetics. To build this model, we first crowdsource a large number of human aesthetics preferences by showing shapes in pairs in an online study and then use the same to build a 3-D shape multiview-based deep neural network architecture to allow us to learn a measure of 3-D shape aesthetics. In comparison to previous approaches, we do not use any predefined notions of aesthetics to build our model. Our algorithmically computed measure of shape aesthetics is beneficial to a range of applications in graphics such as search, visualization, and scene composition.

Bulsarapp: Interactive Visual Analysis for Surname Trend Exploration

Abstract: The study of surnames for a given population, together with their distribution and spatial patterns identification, has been a long-standing problem in the fields of human biology, public health, and social sciences. The ancestry inferred from surname information can be a useful means to understand the dynamics of human populations. This knowledge allows us to characterize geographically the ethnicity of populations, and to understand the complex relationships between identity, migration, and health issues in a demographic view. However, in most cases, a detailed geolocalization of this data can be a daunting task. We propose a visual analytic tool that summarizes the heterogeneous surname and geographic information collected from Argentinean electoral rolls. This tool allows a massive data analysis, and facilitates interdisciplinary studies about population dynamics related to ancestry, migration, and health. It also offers an easy-to-use interface that allows interactive exploration of isonymy and surname origins, their distribution, and spatial trends in a high population density context.

Role of Intricate Pottery Visualization in Ceramic Manufacturing

Abstract: Layered manufacturing, the underlying technology of 3-D printing, has made rapid strides over the last 30 years. We discuss layered manufacturing from the artist's perspective, especially for intricate ceramic pottery. We contend that opportunities exist for applying visualization to the foremost problems plaguing layered manufacturing. Virtual pottery involves meeting two conflicting constraints: rapid visualization during modeling and accurate rapid prototyping during manufacturing. Artists simultaneously need both low polygon shape representation for interactive visualization and adequate representation for generating accurate printable models. Artists also face the additional complexities of adding surface details that cannot be achieved by hand and handling materials like clay used in the manufacturing of real pottery. Illustrated by a system we have developed that uses sound resonance patterns to create volumetric textures for virtual pottery, we show how visualization helps address both these problem areas.