Beyond the Surface

Evaluating Perception and Accuracy in 3D Printed Data Visualizations

Tyler Wiederich

September 29, 2025

Overview

• Motivation and Background
• Roadmap
• Projects
  1. 3D Bar Charts
  2. Experiential Learning
  3. 3D Heat Maps
• Contribution

Experiential learning serves a dual purpose by letting Stat 218 students interact with the study, thus letting us get a large sample size

Motivation

Motivation

Modern statistical graphics exist largely within the confines of 2D projections.

• Research articles
• Textbooks
• Documentation / reports
• News articles

Why?

• Easy to make / understand
• Low cost to produce
• Many available software options

Source: www.1011now.com

Source: www.klkntv.com

Source: Black Hills Energy

Motivation

The rise in computer graphics also included the pioneering of 3D graphical renderings ^{1 2}. Many software programs include options for creating such charts, including:

• R
• Python
• SAS
• MATLAB
• SPSS
• etc.

Figure 1: 3D scatterplot of the Palmer Penguins dataset from Horst, Hill, and Gorman (2020)

Mention Excel in the list of software

The first digital 3d software was Sketchpad, created by Ivan Sutherland as part of his PhD in 1963.

1. Armstrong (2016)

2. Murdoch and Adler (2024)

(a) Microsoft® Excel 95

(b) Microsoft ® Excel for Mac, Version 16

Figure 2: Two 3D charts created with Microsoft® Excel

Other types of 3d charts include area, line, bar, and pie

Motivation

To date, there is no widespread usage of 3D statistical graphics outside of 2D projections. They are generally considered to be items of interest, without a formal role in data interpretations.

Figure 3: 3D visualization of ___ in DSCI 210 at Winona State University, 2019.

There are many instances of data physicalization, but often utilized for unique artistic displays.

Background

Evaluation of 3D Charts

There are many possible metrics to evaluate the qualities of a chart ^{1 2 3 4 5}.

• Accuracy

• Response times

• Pattern recognition

• Memorability

• Preference

1. Vanderplas, Cook, and Hofmann (2020)

2. VanderPlas and Hofmann (2017)

3. Buja et al. (2009)

4. Cleveland and McGill (1984)

5. Peña, Ragan, and Harrison (2020)

Types of 3D charts

3D charts can be grouped into one of two categories.

Decorative 3D

• 2D charts converted drawn to have artificial axis
• Considered to be “chart junk”¹
• Static renderings

Informational 3D

• All axes convey information
• Intuitive use cases
• Static or interactive renderings

1. Tufte (2001)

Data beyond a screen

Many studies on 2D and 3D charts have one key limitation: they are limited to 3D charts presented on 2D surfaces. With modern technology, we are able to create 3D charts in our 3D world via 3D-printing. The rising popularity and decreasing cost of 3D printers makes this option more feasible to implement in the communication of data.

Figure 4: 3D-printed representation of 3D printer sales. Source: Wall Street Journal (link to print)

• Kraus et al. (2020) uses VR

Why study 3D-printed charts?

There are several reasons to explore this medium of statistical graphics.

• Interactions are more natural than digitized counterparts
• Increased recall of information ^{1 2}
• Realistic use cases for informational/educational purposes

1. Stusak, Schwarz, and Butz (2015)

2. Stusak, Hobe, and Butz (2016)

Research Objectives

Data physicalization of statistical graphics has not been widely studied. In our research, we are broadly studying the following questions:

• Do 3D-printed charts follow patterns found in previous studies that compared different types of charts?

• Are 3D-printed charts better or worse than their digitized counterparts (2D and 3D renderings) regarding numerical accuracies of stimuli ratio comparisons?

• How do students participating in our experiments reflect on the scientific process, both as a participant and when viewing the study through the eyes of a researcher?

I believe only there are only a couple of studies that focus on the memorability of statistical graphics

Research Overview

Project 1

• Study of 3D-printed bar charts
• Pilot study within the Statistics department

Project 2

• Incorporate the experiment of Project 1 into the curriculum of Stat 218
• Give students a series of reflections related to the experiment, allowing us to follow students from participants to consumers of scientific knowledge

Project 3

• Same format as Project 2, but replacing the 3D bar charts with 3D heat maps

 

 

 

Roadmap

3D Bar Charts

• ✅ Lit Review
• ✅ Design
• ✅ Data Collection
• ✅ Analysis
• ✅ Write-up

Experiential Learning

• ☑️ Lit Review
• ✅ Design
• ➡️ Data Collection
  • ✅ 3D Bar Charts
  • ➡️ 3D Heat Maps
• Analysis
• Write-up

3D Heat Maps

• ✅ Lit Review
• ✅ Design
• ➡️ Data Collection
• Analysis
• Write-up

Project 1: 3D Bar Charts

3D Bar Charts

(a) Figure from 1978 Handbook of Agricultural Charts. Enlargements / [u.s. Department of Agriculture] (1978)

(b) Figure from 1978 Handbook of Agricultural Charts. Enlargements / [u.s. Department of Agriculture] (1978)

(c) Figure from Modley (1952)

Figure 5: Three examples of 3D bar charts found in publications.

Background

The relationship between 2D and 3D bar charts presented on paper or computer screens has been widely studied.

• Accuracy tends to be worse than 2D equivalents, although not always with statistical significance (Zacks et al. 1998; Siegrist 1996; Melody Carswell, Frankenberger, and Bernhard 1991; Stewart, Cipolla, and Best 2009).

• More time is needed to answer questions than 2D equivalents (Siegrist 1996; Fischer 2000; Stewart, Cipolla, and Best 2009).

• 2D charts chosen more often when used for numerical estimations, although 3D is chosen more often when presenting results to others (Levy et al. 1996; Fisher, Dempsey, and Marousky 1997; Stewart, Cipolla, and Best 2009).

Overview of Project 1

While early testing of statistical graphics started in the early 20th century ^{1 2}, a major study was conducted by Cleveland and McGill (1984) to provide guidance on better visualization practices. In their study, numerical estimations of stimuli ratios showed differences in various methods of data representation.

In our study, we partially replicate Cleveland and McGill’s study, including the additional factor of chart medium.

1. Croxton and Stein (1932)

2. Vanderplas, Cook, and Hofmann (2020)

Figure 6: Examples of EPTs listed in Figure 1 of Cleveland and McGill (1984).

Cleveland and McGill (1984)

Stimuli

• Target stimuli followed \(s_i=10 \times 10^{(i-1)/12},\quad i=1,\dots, 10\)
• Ratios of target stimuli: 0.178, 0.261, 0.383, 0.464 (twice), 0.562, 0.681 (twice), 0.825 (twice)

All non-target stimuli were chosen at random, subject to constraints.

(a) Stimuli Values

(b) Ratios

Figure 7: Stimuli and ratios used by Cleveland and McGill (1984).

Cleveland and McGill (1984)

Chart Types

Figure 8: Chart types used by Cleveland and McGill (1984)

• Type 1: Position along a common scale
• Type 2: Position along a common scale
• Type 3: Position along a common scale
• Type 4: Length
• Type 5: Length

Cleveland and McGill (1984)

Experimental Design

• Treatments
  • Ratio (10)
  • Graph type (5)
• Responses
  • Correctly identify smaller value of stimuli pair
  • \(\log_2(|\text{Error}|+1/8)\)
• Procedure
  • 5 practice graphs followed by 50 graphs in an identical randomized order for each participant

Project 1

Our first project partially replicates and expands on the first experiment by Cleveland and McGill (1984).

Component	Our Research	Cleveland and McGill
# of ratios	7 ratios	10 ratios (7 unique)
# of chart types	2	5
# of media types	3 (2D and 3D digitized, 3D printed)	1 (2D on paper)
# of charts/participant	15	50

To get 15 graphs per participant, we used incomplete blocking of 5 on the seven ratios. The type of chart was randomly allotted so that number of Type 1 and Type 3 charts were approximately the same.

After the pilot study, a static 3D chart was included into the study.

Project 1 Chart Examples

Figure 9: Charts used in bar chart pilot study.

Project 1 Results

Figure 10: Results from pilot study conducted for the 3D bar chart experiment.

Using a generalized additive model (GAM), the only significant effect was either replicating Type 1 or Type 3. In the pilot study, we had 38 participants, so it is likely that the experiment was under-powered.

• Integration of the experiment into Stat 218 has provided many more responses, but results have not yet been analyzed.

Project 2: Experiential Learning

Experiential Learning

In a dual purpose role, a large sample was obtained for the two previous projects by incorporating the experiments as an experiential learning opportunity for Stat 218 students. This is a six stage project that follows students from participants to consumers of scientific knowledge.

Informed Consent

• Obtain consent
• Confirm age of majority (≥19)

Pre-experiment

• How does the scientific process differ between researchers and the general population

Experiment

• Provide completion code to show proof of experiment participation

Experiential Learning

Post Experiment

• What was the purpose of experiment?
• What were the hypotheses?
• What are the sources of error?
• What were the variables?
• What elements of experimental design were present?

Abstract

• What components of the experiment are now clearer?

Presentation

• How did the information you gained from each stage differ?
• What was emphasized in the presentation that wasn’t emphasized in the the abstract?
• What critiques do you have of the study?
• Did you prefer the abstract or presentation?

Figure 11: Word bigram from the abstract reflection module. Students answered the question: “What components of the experiment are clearer now than they were as a participant? What questions do you still have for the experimenter? Write 3-5 sentences reflecting on the abstract.”

Project 3: 3D Heat Maps

3D Heat Maps

(a) Figure from for Educational Statistics et al. (1977)

(b) Figure from “3D Population Density of the US - HomeArea.com” (n.d.)

Figure 12: Examples of 3D heat maps.

Background

Unlike decorative 3D elements, direct comparisons of dimensionality is more limited when the third dimension coveys information.

• Accuracy is worse for volumes than for areas (Croxton and Stein 1932).

• 3D charts can sometimes better encode information than 2D charts (Barfield and Robless 1989).

• 3D heat maps have lower error rates than 2D heat maps in virtual reality (Kraus et al. 2020).

• Kraus et al. (2020) talks about more time needed for 3D charts
• Also, Kraus et al. (2020) mentioned 2D was better for overview tasks

Overview of Project 3

We maintain the same objective when adding information to the third dimension: does numerical accuracy of ratio estimations differ between dimensionality and projections of chart types. However, it is important to note that direct translations of 2D and 3D heatmaps require different visual cues.

(a) Flatter height scale

(b) Stretched height scale

Figure 13: Two figures representing the volcano dataset (R Core Team 2024) using different height scaling factors. Both figures use the same color scale, but there is no “correct” translation of color into the measurable height.

2D heat maps use color, 3D heat maps use height and can possibly use color as well.

Stimuli Construction

The design of the 3D heat map experiment uses the method of constant stimuli: ratios are estimated with respect to one stimuli height that remains the same.

Setting 50 as the constant and 90 as the maximum, a sequence of stimuli are chosen by equally partitioning the ratios between \(50/50=1\) and \(50/90\approx0.556\). The same ratios are used when setting 50 as the maximum in the stimuli pair.

(a) Stimuli values

(b) Ratios of stimuli pairs

Figure 14: Stimuli used in 3D heat map experiment. All values are paired with the stimuli magnitude of 50.

Media Types

(a) 2D heatmap

(b) 3D Digital heatmap

(c) 3D-Printed heatmap

Figure 15: Media types using dataset 1 in the 3D heatmap experiment.

Procedure

Treatment Design

• 9 stimuli pairs
• 3 media types
• 2 heat map datasets

Experiment Design

1. Create incomplete blocks on stimuli pairs (choose 4 from 9)
2. Randomly assign participant to block on experiment initialization
3. Present media type x dataset in randomized order (whole-plot)
4. Present stimuli pair in randomized order (split-plot)

Contribution to literature

Contribution to literature

Nearly all studies involving statistical graphics use paper-printed or digitized charts, resulting in a knowledge gap in statistical graphics presented in tangible formats. Our contributions are as follows:

1. Explore the relationship of physical dimensionality on bar charts / heat maps using numerical accuracy of ratio estimations.
2. Provide recommendations on the construction of 3D-printed statistical graphics.
3. Explore how students interact with the process of scientific investigation using a “hands-on” statistical study.

Exceptions include tactile charts, virtual reality, and artistic data physicalization. However, the interaction with these differs vastly with our 3D printed charts.

Questions?

References

1978 Handbook of Agricultural Charts. Enlargements / [u.s. Department of Agriculture]. 1978. District of Columbia: United States. Department of Agriculture; U.S. Dept. of Agriculture ; Distributed by Science; Education Administration.

“3D Population Density of the US - HomeArea.com.” n.d. https://www.homearea.com/featured/3d-population-density/#3128000. Accessed September 17, 2025.

Armstrong, Helen. 2016. Digital Design Theory : Readings from the Field. Design Briefs. New York, New York: Princeton Architectural Press.

Barfield, Woodrow, and Robert Robless. 1989. “The Effects of Two- or Three-Dimensional Graphics on the Problem-Solving Performance of Experienced and Novice Decision Makers.” Behaviour & Information Technology 8 (5): 369–85. https://doi.org/10.1080/01449298908914567.

Buja, Andreas, Dianne Cook, Heike Hofmann, Michael Lawrence, Eun-Kyung Lee, Deborah F. Swayne, and Hadley Wlckham. 2009. “Statistical Inference for Exploratory Data Analysis and Model Diagnostics.” Philosophical Transactions: Mathematical, Physical and Engineering Sciences 367 (1906): 4361–83. https://www.jstor.org/stable/40485732.

Cleveland, William S., and Robert McGill. 1984. “Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods.” Journal of the American Statistical Association 79 (387): 531–54. https://doi.org/10.1080/01621459.1984.10478080.

Croxton, Frederick E., and Harold Stein. 1932. “Graphic Comparisons by Bars, Squares, Circles, and Cubes.” Journal of the American Statistical Association 27 (177): 54–60. https://doi.org/10.1080/01621459.1932.10503227.

Fischer, Martin H. 2000. “Do Irrelevant Depth Cues Affect the Comprehension of Bar Graphs?” Applied Cognitive Psychology 14 (2): 151–62. https://doi.org/10.1002/(SICI)1099-0720(200003/04)14:2<151::AID-ACP629>3.0.CO;2-Z.

Fisher, Samuel H., John V. Dempsey, and Robert T. Marousky. 1997. “Data Visualization: Preference and Use of Two-Dimensional and Three-Dimensional Graphs.” Social Science Computer Review 15 (3): 256–63. https://doi.org/10.1177/089443939701500303.

for Educational Statistics, National Center, National Center for Education Statistics, United States. Office of Educational Research, Improvement. Center for Education Statistics, and Institute of Education Sciences (U.S.). 1977. The Condition of Education. 1975-1979: DHEW Publication. U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics.

Horst, Allison Marie, Alison Presmanes Hill, and Kristen B Gorman. 2020. Palmerpenguins: Palmer Archipelago (Antarctica) Penguin Data. https://doi.org/10.5281/zenodo.3960218.

Kraus, Matthias, Katrin Angerbauer, Juri Buchmüller, Daniel Schweitzer, Daniel A. Keim, Michael Sedlmair, and Johannes Fuchs. 2020. “CHI ’20: CHI Conference on Human Factors in Computing Systems.” In, 1–14. Honolulu HI USA: ACM. https://doi.org/10.1145/3313831.3376675.

Levy, Ellen, Jeff Zacks, Barbara Tversky, and Diane Schiano. 1996. “The SIGCHI Conference.” In, 42–49. Vancouver, British Columbia, Canada: ACM Press. https://doi.org/10.1145/238386.238400.

Melody Carswell, C., Sylvia Frankenberger, and Donald Bernhard. 1991. “Graphing in Depth: Perspectives on the Use of Three-Dimensional Graphs to Represent Lower-Dimensional Data.” Behaviour & Information Technology 10 (6): 459–74. https://doi.org/10.1080/01449299108924304.

Modley, Rudolf. 1952. Pictographs and Graphs : How to Make and Use Them. New York: Harper.

Murdoch, Duncan, and Daniel Adler. 2024. Rgl: 3D Visualization Using OpenGL. https://CRAN.R-project.org/package=rgl.

Peña, Alyssa, Eric Ragan, and Lane Harrison. 2020. “Memorability of Enhanced Informational Graphics.” Interdisciplinary Journal of Signage and Wayfinding 4 (1). https://doi.org/10.15763/issn.2470-9670.2020.v4.i1.a54.

R Core Team. 2024. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Siegrist, Michael. 1996. “The Use or Misuse of Three-Dimensional Graphs to Represent Lower-Dimensional Data.” Behaviour & Information Technology 15 (2): 96–100. https://doi.org/10.1080/014492996120300.

Stewart, Brandie M., Jessica M. Cipolla, and Lisa A. Best. 2009. “Extraneous Information and Graph Comprehension: Implications for Effective Design Choices.” Edited by M. Henderson. Campus-Wide Information Systems 26 (3): 191–200. https://doi.org/10.1108/10650740910967375.

Stusak, Simon, Moritz Hobe, and Andreas Butz. 2016. “If Your Mind Can Grasp It, Your Hands Will Help.” In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, 92–99. Eindhoven Netherlands: ACM. https://doi.org/10.1145/2839462.2839476.

Stusak, Simon, Jeannette Schwarz, and Andreas Butz. 2015. “Evaluating the Memorability of Physical Visualizations.” In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 3247–50. Seoul Republic of Korea: ACM. https://doi.org/10.1145/2702123.2702248.

Tufte, Edward R. 2001. The Visual Display of Quantitative Information. 2nd ed. Cheshire, Conn: Graphics Press.

Vanderplas, Susan, Dianne Cook, and Heike Hofmann. 2020. “Testing Statistical Charts: What Makes a Good Graph?” Annual Review of Statistics and Its Application 7 (1): 61–88. https://doi.org/10.1146/annurev-statistics-031219-041252.

VanderPlas, Susan, and Heike Hofmann. 2017. “Clusters Beat Trend!? Testing Feature Hierarchy in Statistical Graphics.” Journal of Computational and Graphical Statistics 26 (2): 231–42. https://www.jstor.org/stable/44861949.

Zacks, Jeff, Ellen Levy, Barbara Tversky, and Diane J. Schiano. 1998. “Reading Bar Graphs: Effects of Extraneous Depth Cues and Graphical Context.” Journal of Experimental Psychology: Applied 4 (2): 119–38. https://doi.org/10.1037/1076-898X.4.2.119.

Appendix

Project 1

Using a linear mixed model approach, a suitable model is:

\[ \log_2(|\text{Error}|+ 1/8)=\mu+R_i+T_j+M(R)_{k(i)} + \gamma_l + \omega_{lm} + \epsilon_{ijklmn} \]

where

• \(R_i\) is the effect of ratio i
• \(T_j\) is the effect of type j
• \(M(R)_{k(i)}\) is the effect of Media k nested in Ratio i
• \(\gamma_l\sim N(0,\sigma^2_{kit})\)
• \(\omega_{lm}\sim N(0,\sigma^2_{sub-kit})\)
• \(\epsilon_{ijkl}\sim N(0,\sigma^2_{e})\)

Project 1

Figure 16: Responses of Project 1

Project 1

Figure 17: Responses of Project 1 using log2(|Error| + 1/8)

Project 1

With estimates from the pilot study, a power analysis can be conducted. Note that 18 out of 21 total kits were used in the analysis of the pilot study.

• Power > 0.8 for ratio with 1 subject / kit

• Power > 0.8 for plot(ratio) with 2 subjects / kit

• Power > 0.8 for chart type with 17 subjects / kit

• \(n=1\) per kit: Power > 0.8 for ratio of 46.4 between 3dd and 3dp

• \(n=10\) per kit: Power > 0.8 for 5/21 plot(ratio) comparisons

• \(n=20\) per kit: Power > 0.8 for 11/21 plot(ratio) comparisons

• \(n=40\) per kit: Power > 0.8 for 13/21 plot(ratio) comparisons

Power is low when the ratio is 17.8 and 56.2. Also note that \(n=40\) per kit equates to \(840\) total subjects across 21 kits, which is reasonable given the number of Stat 218 students enrolled across multiple semesters.

Project 3

What is the benefit of using the same ratio twice?

Figure 18: Scatterplot of ratios and difference of each stimuli pairs.

Since color does not have a direct translation to height, it is a good idea to see if ratio estimations are different depending on the physical different of the bars on the heat map.

Using the method of constant stimuli, stimuli pairs have a larger difference for smaller ratios.

• If there is a difference in ratio estimations between stimuli pairs at the same ratio, then we show that the physical difference is also an important factor, meaning that future studies need to take this into consideration