UC Davis
IDAV Publications
Title
Future Challenges for Ensemble Visualization
Permalink
https://escholarship.org/uc/item/9v9799q1
Journal
IEEE Computer Graphics and Applications, 34
Authors
Obermaier, Harald
Joy, Kenneth I.
Publication Date
2014
Peer reviewed
eScholarship.org Powered by the California Digital Library
University of California
Future Challenges for Ensemble Visualization
H. Obermaier and K. I. Joy
University of California, Davis
Introduction
How often have you executed an algorithm only to
find that getting reasonable results means changing
parameters and restarting? How much time did you
spend on finding the ”correct” parame t er s? Imag-
ine going through the sam e ordeal with unbelievably
complex simulation models used for the prediction
of physical phenomena. Scientist s have fought this
battle for many years and have long been sick of the
”sitting, waiting, and restarting” process. Luckily,
increased availability of vast computation resources
and new computation strategies o↵er a solution to
this dilemma: Scientists can now run several alterna-
tive parameter settings or simulation models in paral-
lel, creating an ’ensemble’ of p os si bl e outcomes for a
given event of interest (see Figure 1). In our conver-
sations with simulation scientists and visualization
researchers, the visual analysis of ensemble data has
repeatedly come up as one of th e most important
new areas of visualization and we expect it to have
a wide impact on our field in the next few years.
The challenge is to develop expr e ssi ve visualizations
of properties of this set of solutions, the ensemble, to
support scientists in this challenging parameter-space
exploration task.
The idea to move away from visualizing a single
concrete solution to analyzing a family of outcomes
is not entirely novel to the field of scientific visualiza-
tion: Since t he mid 1990s, visualization researchers
have developed methods to visualize uncertainty and
errors in data. However, t he re is a key di↵erence be-
tween uncertain data and ensemble data – uncer tai n
data encodes the probability di st r i bu t i ons of values
throughout a data set , allowing the identification of
a ”most likely” or ”most common” output, while con-
taining no information about how di↵erent outcomes
Figure 1: A set of 15 stirring simulations, showing
velocity magnitudes for di↵erent fluid properties. Vi-
sual comparison of ens emble members in a list vi e w
is difficult.
relate. Ensembles, on the other hand, present us with
concrete dist r ib u ti on s of data, where each out come
can be uniquely associated with a specific run or set
of simulation parameters.
This discrete character together with the ability to
relate outputs to specific inputs is what makes en-
sembles so valuable to domain experts. Our chal-
lenge is to develop visualization techniques and tools
to extract and highlight commonalities, di↵erences,
and trends in the set of ensemble members and to
allow scientists to discover conceptual drawbacks or
1
the value of simulation models or specific parameter
choices.
Visualization of Ensemble Data
Recently, researchers have started to look into the
visualization of ensemble data. The few existing ap-
proaches can be classified as
• Feature-Based: Features are extracted from indi-
vidual ensemble members and compared across
the ensemble.
• Location-Based: Ensemble comparison is per-
formed at fixed locations in the data set.
Due to the existence of a variety of prediction mod-
els, weather and cli mat e research is one of t h e central
driving forces behind the creation of simulation en-
sembles. Predictions of climate events rely on a large
number of external i nfl ue nc es (p ar amet e rs ) and are
generally associated with a certain probability of oc-
currence. This is because there is not only one possi-
ble outcome to existing prediction mod el s, but rather
a spectrum of possibilities. Sanyal et al. [5] approach
the vi su ali z ati on of ensembles of numerical weather
simulations by extracting sets of isocontour lines and
designing glyphs that illu st r at e local variances. The
so-called spaghetti plots created by rendering sets of
isocontour lines allow feature-bas ed comparison be-
tween members of the ensemble and give a basic im-
pression of how ensembles agree for a given scalar
value. A sample image of their technique is shown in
Figure 2. In three dimensions, slicing can help cre at e
an impression of di↵erences in isosur fac es , as demon-
strated by Alabi et al. [1] and shown in F i gure 3.
As you can see, even simple tasks such as rendering
spaghetti plots becomes a visualization challenge in
three dimensions.
Location-based methods have largely been used
to compute statistical properties of the ensemble
throughout the domain. Common statistical mea-
sures then provide insights into outliers, agre eme nt
or disagreement between members of the ensemble.
Means and variances of scalar quantities in c l i mat e
simulations have been employed by Potter et al. [4],
Figure 2: A spaghetti plot of perturbation pressure
isocontours along with uncertainty glyphs showing
deviation. Ensembles are particularl y relevant in
weather and climate prediction. Image courtesy of
Sanyal et al. [5].
see Figure 4. Notions of variances can also be em-
ployed to detect agreement and disagreement in en-
sembles for arbitrary flow simulations [3]. Figure 5
shows an example of CFD variance-based coloring
and trends querying. This provides a more conci se
representation of a set of flow simulations, than pro-
vided by list-based visualizations as shown in F i gu re
1. Ensemble visualizations can be made especially ex-
pressive if (some) ground truth data is available. This
allows for an estimation of predictive uncertainty of
the ensemble and can aid in identifying outliers in the
set of ensemble members [2], see Figure 6. Gosink et
al. also propose a visualization of parameter sensiti v -
ity, which is a key component of efficient parameter
space analysis.
What the Future Holds
We see th e need for visualization research to aid sim-
ulation scientists in the parameter space expl or at i on
task and to support accurate an d robust decision
2
Figure 3: Rendering of sliced isosurfaces from four
ensemble members. Feat u re -b ase d comparison in 3D
is challenging and can su↵er from occlusion and visual
complexity. Image courtesy of Alabi et al. [1].
making in a complex simulation environment. The
need for e↵ective visual analysis tools in this area has
the potential to open and extend a large variety of
research directions and application scenarios. In our
correspondence with domain scientists we have iden-
tified several key req u ir em ents of e↵ect i ve ensemble
visualization tools along with conceptual and techni-
cal issues. These challenges include
• Conceptual - Finding the Answer: Can we help
domain experts in finding a ”most likely/best
prediction” made by the ensemble?
• Conceptual - Parameter Space: How can we re-
late insights gained by ensemble visualization
with locations in parameter space?
• Conceptual - Perception: How can we visualize
such a multitude of data in a precise and eas y-
to-understand way?
• Mathema tical - Features: What are statistical or
geometric feature definitions that are relevant in
the context of ensembles?
• Mathema tical - Metrics: How can we compare
members of the ensemble or their features?
Figure 4: Mean and standard deviation shown as
a combination of isocontouring and col or -map pi n g.
Correlations of the two variables are conveyed im-
plicitly. Image c our t esy of Potter et al. [4].
• Algorithmic - Data Complexity: How do we han-
dle the im men se increases in memory require-
ment and dat a complexity?
• Algorithmic - Exploration: How do we enable
goal-driven exploration and analysis of parame-
ter space and parameter sensitivity?
One direction we regard as especially challeng-
ing is the visualization an d exploration of multi-
dimensional parameter spaces. In this direction
we are investigating how techniques from high-
dimensional data visualization can help in making the
connection between ensemble and parameter-space
analysis. Specifically, the question whether and how
recent work in computational steering, paramete r-
space exploration (e.g., Waser et al. [6]) and multi-
variate analysis may be applied to complex ensemble
visualization problems remains to be answered.
Conclusions
Providing domain scientists with visualization solu-
tions for ensemble dat a will be a key factor in improv-
ing analysis performance in complex simulation envi-
ronments. Solving the inhere nt visualization chal-
lenges will inc re ase the speed with which scientists
3
Figure 5: Simulation agreement and disagreement in
an ensemble of CF D simulations is visualized through
color mapped transport variance. Trends are identi-
fied as pathline clusters. A large number of ensemble
members may lead to cluttered pathline renderings.
Image courtesy of Hummel et al. [3].
can explore, adapt, and validate simulation models.
We expect the visualization community to engage in
solving this challenging task, and thereby improve the
robustness and reliability of simulation-based predic-
tion and decision-making.
Acknowledgements
This work was supported in part by the NSF
(IIS 0916289 and IIS 1018097), and the Office of
ASCR, Office of Science, through DOE SciDAC con-
tract D E- FC02-06ER25780, and contract DE-FC02-
12ER26072 (SDAV).
References
[1] O . S. Alabi, X. Wu, J. M. Harter, M. Phadke,
L. Pinto, H. Petersen, S. Bass, M. Keifer,
S. Zhong, C. Healey, and R. M. Taylor II. Com -
parative visualization of ensembles using ensem-
ble surface slicing. In Proc. SPIE 8294, Visual-
ization and Data Analysis 2012, pages 82940U–
82940U–12, 2012.
[2] L. G osi nk , K. Bensema, T. Pulsipher, H. Ober-
maier, M. Henry, H. Childs, and K. I. Joy. Char -
acterizing and v i su al i zi ng predictive uncertainty
Figure 6: Evaluating predictive uncertainty of a sin-
gle ensemble member through accuracy classification
with respect to a probabilistic ground truth model.
Ground truth data provides a basis for more complex
analysis and classification of ensemble data. Visualiz-
ing complex classifications in 3D proves challenging.
Image courtesy of Gosink et al. [2].
in numerical ensembles through Bayesian model
averaging. IEEE Transactions on Visualization
and Computer Graphics, 19(12), 2013.
[3] M . Hummel, H. Obermaier, C. Gart h , an d K. I.
Joy. Comparative visual analysis of Lagrangian
transport in CFD ensembles. IEEE Transactions
on Visualization and Computer Graphics, 19(12),
2013.
[4] K . Potter, A. Wilson, P.-T. Bremer, D. Williams,
C. D ou t ri au x , V. Pascucci, and C. R. Johnson .
Ensemble-vi s: A framework for the statistical vi-
sualization of ensemble data. In IEEE Workshop
on Knowledge Discovery from Climate Data: Pre-
diction, Extremes., pages 233–240, 2009.
[5] J . Sanyal, S. Zhang, J. Dyer, A. Mercer, P. Am-
burn, and R. Moor h ead . Noodles: A tool for vi-
sualization of numerical weather model ensemble
uncertainty. IEEE Transactions on Visualization
and Computer Graphics, 16(6):1421–1430, 2010.
[6] J . Waser, H. Ribi˘ci´c, R. Fuchs, C. Hirsch,
B. Schindler, G. Bl¨oschl, and M. Gr¨oller. Nodes
on ropes: A compr eh en si ve data and control flow
for steering ensemble simulations. Visualization
and Computer Graphics, IEEE Transactions on,
17(12):1872–1881, 2011.
4
![Figure 2: A spaghetti plot of perturbation pressure isocontours along with uncertainty glyphs showing deviation. Ensembles are particularly relevant in weather and climate prediction. Image courtesy of Sanyal et al. [5].](/figures/figure-2-a-spaghetti-plot-of-perturbation-pressure-362lkqim.webp)
![Figure 5: Simulation agreement and disagreement in an ensemble of CFD simulations is visualized through color mapped transport variance. Trends are identified as pathline clusters. A large number of ensemble members may lead to cluttered pathline renderings. Image courtesy of Hummel et al. [3].](/figures/figure-5-simulation-agreement-and-disagreement-in-an-1n67vubu.webp)
![Figure 6: Evaluating predictive uncertainty of a single ensemble member through accuracy classification with respect to a probabilistic ground truth model. Ground truth data provides a basis for more complex analysis and classification of ensemble data. Visualizing complex classifications in 3D proves challenging. Image courtesy of Gosink et al. [2].](/figures/figure-6-evaluating-predictive-uncertainty-of-a-single-39hej4gx.webp)
![Figure 4: Mean and standard deviation shown as a combination of isocontouring and color-mapping. Correlations of the two variables are conveyed implicitly. Image courtesy of Potter et al. [4].](/figures/figure-4-mean-and-standard-deviation-shown-as-a-combination-2bgwwace.webp)
![Figure 3: Rendering of sliced isosurfaces from four ensemble members. Feature-based comparison in 3D is challenging and can su↵er from occlusion and visual complexity. Image courtesy of Alabi et al. [1].](/figures/figure-3-rendering-of-sliced-isosurfaces-from-four-ensemble-3t7hl4vg.webp)