We present a model of visualization design based on algebraic considerations of the visualization process. The model helps characterize visual encodings, guide their design, evaluate their effectiveness, and highlight their shortcomings. The model has three components: the underlying mathematical structure of the data or object being visualized, the concrete representation of the data in a computer, and (to the extent possible) a mathematical description of how humans perceive the visualization. Because we believe the value of our model lies in its practical application, we propose three general principles for good visualization design. We work through a collection of examples where our model helps explain the known properties of existing visualizations methods, both good and not-so-good, as well as suggesting some novel methods. We describe how to use the model alongside experimental user studies, since it can help frame experiment outcomes in an actionable manner. Exploring the implications and applications of our model and its design principles should provide many directions for future visualization research.

An Algebraic Process for Visualization Design

Augmented reality technologies allow people to view and interact with virtual objects that appear alongside physical objects in the real world. For augmented reality applications to be effective, users must be able to accurately perceive the intended real world location of virtual objects. However, when creating augmented reality applications, developers are faced with a variety of design decisions that may affect user perceptions regarding the real world depth of virtual objects. In this paper, we conducted two experiments using a perceptual matching task to understand how shading, cast shadows, aerial perspective, texture, dimensionality (i.e., 2D vs. 3D shapes) and billboarding affected participant perceptions of virtual object depth relative to real world targets. The results of these studies quantify trade-offs across virtual object designs to inform the development of applications that take advantage of users' visual abilities to better blend the physical and virtual world.

Designing for Depth Perceptions in Augmented Reality

Cerebral vascular images obtained through angiography are used by neurosurgeons for diagnosis, surgical planning, and intraoperative guidance. The intricate branching of the vessels and furcations, however, make the task of understanding the spatial three-dimensional layout of these images challenging. In this paper, we present empirical studies on the effect of different perceptual cues (fog, pseudo-chromadepth, kinetic depth, and depicting edges) both individually and in combination on the depth perception of cerebral vascular volumes and compare these to the cue of stereopsis. Two experiments with novices and one experiment with experts were performed. The results with novices showed that the pseudo-chromadepth and fog cues were stronger cues than that of stereopsis. Furthermore, the addition of the stereopsis cue to the other cues did not improve relative depth perception in cerebral vascular volumes. In contrast to novices, the experts also performed well with the edge cue. In terms of both novice and expert subjects, pseudo-chromadepth and fog allow for the best relative depth perception. By using such cues to improve depth perception of cerebral vasculature, we may improve diagnosis, surgical planning, and intraoperative guidance.

An Evaluation of Depth Enhancing Perceptual Cues for Vascular Volume Visualization in Neurosurgery

This survey provides an overview of perceptually motivated techniques for the visualization of medical image data, including physics-based lighting techniques as well as illustrative rendering that incorporate spatial depth and shape cues. Additionally, we discuss evaluations that were conducted in order to study the perceptual effects of these visualization techniques as compared to conventional techniques. These evaluations assessed depth and shape perception with depth judgment, orientation matching, and related tasks. This overview of existing techniques and their evaluation serves as a basis for defining the evaluation process of medical visualizations and to discuss a research agenda.

/pdf/a-survey-of-perceptually-motivated-3d-visualization-of-29le6d65wl.pdf

A survey of perceptually motivated 3D visualization of medical image data

Otologic Skills Training.

Visualizing complex volume data usually renders selected parts of the volume semitransparently to see inner structures of the volume or provide a context. This presents a challenge for volume rendering methods to produce images with unambiguous depth-ordering perception. Existing methods use visual cues such as halos and shadows to enhance depth perception. Along with other limitations, these methods introduce redundant information and require additional overhead. This paper presents a new approach to enhancing depth-ordering perception of volume rendered images without using additional visual cues. We set up an energy function based on quantitative perception models to measure the quality of the images in terms of the effectiveness of depth-ordering and transparency perception as well as the faithfulness of the information revealed. Guided by the function, we use a conjugate gradient method to iteratively and judiciously enhance the results. Our method can complement existing systems for enhancing volume rendering results. The experimental results demonstrate the usefulness and effectiveness of our approach.

Perceptually-Based Depth-Ordering Enhancement for Direct Volume Rendering

Proper lighting in rendering is essential for visualizing 3D objects, but most visualization software tools still employ simple lighting models. The advent of hardware-accelerated advanced lighting suggests that volume visualization can be truly usable for clinical work. Researchers studied how volume rendering incorporating global illumination impacted perception of bone surface features captured by x-ray computed-tomography scanners for clinical monitoring of rheumatoid arthritis patients. The results, evaluated by clinical researchers familiar with the disease and medical-image interpretation, indicate that interactive visualization with global illumination helped the researchers derive more accurate interpretations of the image data. With clinical needs and the recent advancement of volume visualization technology, this study is timely and points the way for further research.

/pdf/using-global-illumination-in-volume-visualization-of-1w50mo6i6k.pdf

Using global illumination in volume visualization of rheumatoid arthritis CT data.

With the advances in dual medical imaging, the requirements for multimodal and multifield volume visualization begin to emerge. One of the challenges in multimodal visualization is how to simplify the process of generating informative pictures from complementary data. In this paper we present an automatic technique that makes use of dual modality information, such as CT and PET, to produce effective focus+context volume visualization. With volume ray casting, per-ray visibility histograms summarize the contribution of samples along each ray to the final image. By quantifying visibility for the region of interest, indicated by the PET data, occluding tissues can be made just transparent enough to give a clear view of the features in that region while preserving some context. Unlike most previous methods relying on costly-preprocessing and tedious manual tuning, our technique achieves comparable and better results based on on-the-fly processing that still enables interactive visualization. Our work thus offers a powerful visualization technique for examining multimodal volume data. We demonstrate the technique with scenarios for the detection and diagnosis of cancer and other pathologies.

/pdf/visibility-guided-multimodal-volume-visualization-1a0h7jqt86.pdf

Visibility guided multimodal volume visualization

Volume rendering is an effective method for visualizing 3D data. However, it's still difficult to obtain an effective image, especially when there are complicated structures which may cause underexposure problems. Adjusting light sources and parameters adds computational cost, without alleviating the underexposure phenomenon. This paper presents the novel idea of applying lightness anchoring theory for volume visualization enhancement. An anchoring hypothesis, the Highest-Luminance-As-White rule, is adjusted to adapt our volume rendered image. After employing the lightness anchored optimization, underexposed areas can be revealed while still preserving the local depth relationship.

https://dl.acm.org/doi/pdf/10.1145/3095140.3095160

Enhancing volume visualization with lightness anchoring theory

Multimodal volume data commonly found in medical imaging applications present both opportunities and challenges to segmentation and visualization tasks. This paper presents a user directed volume segmentation system. Through a spreadsheets interface, the user can interactively examine and refine segmentation results obtained from automatic clustering. In addition, the user can isolate or highlight a feature of interest in a volume based on different modalities, and see the corresponding segmented results. Our system is easy to use since the preliminary segmentation results are organized and presented to the user in a relation-aware fashion based on the spatial relations between the segmented regions. We demonstrate this system using two multimodal datasets.

Lin Zheng

Papers

Perceptually-Based Depth-Ordering Enhancement for Direct Volume Rendering

Using global illumination in volume visualization of rheumatoid arthritis CT data.

Visibility guided multimodal volume visualization

Enhancing volume visualization with lightness anchoring theory

Relation-aware spreadsheets for multimodal volume segmentation and visualization