Portal de Periódicos da CAPES

Data Visualization in the Neurosciences: Overcoming the Curse of Dimensionality

2012; Cell Press; Volume: 74; Issue: 4 Linguagem: Inglês

10.1016/j.neuron.2012.05.001

1097-4199

Elena A. Allen, Erik B. Erhardt, Vince D. Calhoun,

Neural and Behavioral Psychology Studies

In publications, presentations, and popular media, scientific results are predominantly communicated through graphs. But are these figures clear and honest or misleading? We examine current practices in data visualization and discuss improvements, advocating design choices which reveal data rather than hide it. In publications, presentations, and popular media, scientific results are predominantly communicated through graphs. But are these figures clear and honest or misleading? We examine current practices in data visualization and discuss improvements, advocating design choices which reveal data rather than hide it. Visualizations are vital tools for neuroscientists of every discipline, affording the ability to reveal relationships in large data sets and communicate information to a broad audience. But with the great power of graphs, one might say, comes great responsibility. Graphs can be fundamentally misleading about underlying data, and design choices can skew viewers' perceptions, leading them toward incorrect conclusions (Jones, 2006Jones G.E. How to Lie with Charts.Second Edition. LaPuerta, Santa Monica, CA2006Google Scholar). For example, recent studies suggest that results rendered on aesthetically pleasing brain images are perceived as more persuasive and credible than identical information presented in other formats (Keehner et al., 2011Keehner M. Mayberry L. Fischer M.H. Psychon. Bull. Rev. 2011; 18: 422-428Crossref PubMed Scopus (46) Google Scholar, McCabe and Castel, 2008McCabe D.P. Castel A.D. Cognition. 2008; 107: 343-352Crossref PubMed Scopus (475) Google Scholar). Beyond the attractiveness of displays, readers may also be misled by the frequent errors that plague scientific figures (Cleveland, 1984Cleveland W.S. Am. Stat. 1984; 38: 261-269Google Scholar) or a lack of sufficient information. In the words of statistician and graphic design expert Howard Wainer, effective data visualization must “remind us that the data being displayed do contain some uncertainty” and “characterize the size of that uncertainty as it pertains to the inferences we have in mind” (Wainer, 1996Wainer H. Am. Stat. 1996; 50: 101-111Google Scholar). It is our impression that such descriptions (along with more basic elements) are often lacking from published figures. In this NeuroView, we perform a survey of figures from leading neuroscience journals with an eye toward clarity and the portrayal of uncertainty. Based on survey results, we discuss methods to improve graphics (particularly for large data sets in which visualization poses a challenge) and propose a set of figure guidelines in the form of a checklist (Table 1). We hope these recommendations, compiled from a number of excellent resources on data visualization (Lane and Sándor, 2009Lane D.M. Sándor A. Psychol. Methods. 2009; 14: 239-257Crossref PubMed Scopus (40) Google Scholar, Tufte, 2001Tufte E. The Visual Display of Quantitative Information. Graphics Press, Cheshire, CT2001Google Scholar, Wainer, 1996Wainer H. Am. Stat. 1996; 50: 101-111Google Scholar), may be used by both internal and external reviewers to help evaluate figures for clarity and completeness.Table 1When Evaluating a Figure for Clarity and Completeness, Consider the Following Questions Open table in a new tab We sampled 288 articles published in 2010 from six neuroscience journals (Frontiers in Systems Neuroscience, Human Brain Mapping, Journal of Neuroscience, Nature Neuroscience, NeuroImage, and Neuron) and examined the 1,451 figures therein. We surveyed four basic features that were applicable to nearly all graphs and addressed Wainer's points above. The survey asked the following questions: (1) Is the dependent variable or quantity of interest labeled? (2) Is the scale of the dependent variable indicated? (3) Where applicable, is a measure of uncertainty displayed? (4) Is the type of uncertainty (e.g., standard error bars or confidence intervals) defined in the figure or accompanying legend? Examples of these graphical features are shown in Figure 1A for two-dimensional (2D) and 3D data sets. Survey results, shown in Figure 1B, overwhelmingly suggest that graphical displays become less informative as the dimensions and complexity of data sets increase. Compared to graphs of 2D data, 3D displays provide poorer descriptions of the outcome of interest and rarely provide an indication of uncertainty. Only 43% of 3D graphics label the dependent variable (meaning that if you were asked, “What is being plotted here?” you would be able to answer less than half of the time) and only 20% portray the uncertainty of reported effects. Even for 2D data, the proportion of graphs displaying uncertainty is lower when explanatory variables are continuous (and typically take on many values) than when they are categorical (and typically represent a few conditions; Figure 1C). Of 2D figures that do indicate uncertainty, nearly 30% fail to define the type of uncertainty or variability being portrayed. Given the plurality of interpretations connoted by an error bar (e.g., a standard deviation [SD] of the sample, a standard error of the mean [SEM], a range, a parametric confidence interval [CI] of the mean, a bootstrap CI, a Bayesian probability interval, a prediction interval, etc.), it is unclear how including it without a proper label would offer readers any further understanding of the data; in contrast, the poor labeling or omission of error bars has been shown to encourage misinterpretation (Cumming and Finch, 2005Cumming G. Finch S. Am. Psychol. 2005; 60: 170-180Crossref PubMed Scopus (937) Google Scholar, Vaux, 2004Vaux D.L. Nature. 2004; 428: 799Crossref PubMed Scopus (16) Google Scholar, Wainer, 1996Wainer H. Am. Stat. 1996; 50: 101-111Google Scholar). A breakdown of results by journal (see supplementary analysis at http://mialab.mrn.org/datavis) further highlights the issue of data dimensionality in visualization: journals with lower proportions of 2D and 3D graphical features are those that primarily publish neuroimaging and systems-level findings, in which results are often distilled from very large data sets using a hierarchy of models. That the so-called “curse of dimensionality” extends to the realm of data visualization is not surprising. Dependent variables are more difficult to label when they represent abstract parameter estimates rather than directly measured quantities; uncertainty is more challenging to render when data sets require error surfaces rather than error bars. However, these results are undesirable. As data sets become more complex, displays should become increasingly informative, elucidating relationships that would be inaccessible from tables or summary statistics. In the next section, we provide examples of creating more informative displays for simple and complex data sets by making design choices that reveal data, rather than hide it. Consider a simple experiment in which a researcher investigates the effect of different conditions on a single response variable. Having collected 50 samples of the response variable under each condition 1, 2, and 3, how should the researcher visualize the data to best inform themselves and their audience of the results? Figure 2 provides three possible designs. In panel A, a bar plot displays the sample mean and SEM under each condition. With no distributional information provided, the data density is quite low and the same information could be provided in a single sentence, e.g., “Mean response ± SEM for conditions 1, 2, and 3 were 4.9 ± 0.4, 5.0 ± 0.4, and 5.2 ± 0.4, respectively.” Panel B offers some improvement, with box plots displaying the range and quartiles of each sample. This design reveals that response variables may take on both positive and negative values (hidden in panel A) and that condition 2 may be right skewed. Distributional differences are better understood in panel C when using violin plots to display kernel density estimates (smoothed histograms) of each data set (Hintze and Nelson, 1998Hintze J.L. Nelson R.D. Am. Stat. 1998; 52: 181-184Google Scholar). Violin plots make the skew in condition 2 more apparent and reveal that responses in condition 3 are bimodal (hidden in panels A and B). Although the additional distributional information in panel C does not change our initial inference that sample means are similar between conditions, we are certainly not likely to make the misinterpretation that condition has no effect on the response. Distributional differences also suggest that assumptions of the ANOVA (or other parametric models) may not be met and that the mean may not be the most interesting quantity to investigate. This example is not meant to imply that bar plots should always be avoided in favor of more complex designs. Bar plots have numerous merits: they are easy to generate, straightforward to comprehend, and can efficiently contrast a large number of conditions in a small space. They are particularly effective for displaying frequencies or proportions (as in Figure 1), in which binary data samples are transformed into a height that intuitively reflects the fraction of “successes.” Yet, bar plots are also commonly used in scenarios in which the distance from zero is not meaningful and in which distributional information would be of great benefit to readers. In roughly the same amount of space required by a bar plot, one can portray the full shape of distributions and overlay descriptive statistics, inferential statistics related to hypothesis testing, or even individual data points, creating a so-called “bean plot” (Kampstra, 2008Kampstra, P. (2008). Journal of Statistical Software 28, Code Snippet 1, 1–9.Google Scholar). By increasing the amount of information available to the viewers, we allow them to assess the appropriateness of related statistical analyses and make their own inferences. In Figure 3, we apply the guiding principle of “show more, hide less” to high-dimensional electroencephalographic (EEG) and functional magnetic resonance imaging (fMRI) data sets. We portray the results using a common design (panel A) and a modified design (panel B), in which each change is arrived at by following the guidelines in Table 1. Figures 3Aa and 3Ba present data from an EEG visual flanker task. Subjects were asked to indicate the direction of a visual target which appeared shortly after the presentation of flanking distracters. For each participant, multichannel EEG time series were decomposed using independent component analysis, and a single component best matching the expected frontocentral topography for a performance monitoring process was selected for further analysis (Eichele et al., 2010Eichele H. Juvodden H.T. Ullsperger M. Eichele T. Front Hum Neurosci. 2010; 4: 65PubMed Google Scholar). Here, we ask how the extracted event-related potential (ERP) differs according to the subject's response (i.e., correct or incorrect). Panel A provides a typical portrayal of results, in which mean ERPs are displayed for each condition. As Table 1 recommends, the axes are labeled, variable units are indicated, and experimental conditions are distinguished by line color with direct annotation on the plot. While this panel is clear, it is not complete: there is no portrayal of uncertainty. In panel B, we add 95% confidence bands around the average ERPs. The confidence bands are made slightly transparent to highlight overlap between conditions and to maintain the visual prominence of the means. Confidence intervals clarify that there is greater uncertainty in the error response than the correct response (because subjects make few errors) and that there is insufficient evidence to conclude a response difference after ∼800 ms. In panel B, we also add verbal descriptions and additional annotation to the graphic (Lane and Sándor, 2009Lane D.M. Sándor A. Psychol. Methods. 2009; 14: 239-257Crossref PubMed Scopus (40) Google Scholar, Tufte, 2001Tufte E. The Visual Display of Quantitative Information. Graphics Press, Cheshire, CT2001Google Scholar). Labels indicate that the timeline is relative to the presentation of the target stimulus and specify our null and alternative hypotheses as well as the alpha level (type I error rate) chosen to determine statistical significance. Integrating descriptions into the figure (rather than the legend) discourages misinterpretation and permits readers to understand the display more quickly. Of course, annotation must be used judiciously and should not overwhelm or detract from the data visualization itself. Figures 3Ab and 3Bb portray results from an auditory oddball event-related fMRI experiment. Participants responded to target tones presented within a series of standard tones and novel sounds. Blood oxygenation level-dependent (BOLD) time series at each brain voxel were regressed onto activation models for the target, novel, and standard stimuli (Kiehl et al., 2001Kiehl K.A. Laurens K.R. Duty T.L. Forster B.B. Liddle P.F. Psychophysiology. 2001; 38: 133-142Crossref PubMed Google Scholar). Here, we ask what brain regions might be involved in the novelty processing of auditory stimuli and compare beta parameters between novel and standard conditions. Panel A presents voxelwise differences between beta coefficients using a widely reproduced design: functional-imaging results are thresholded based on statistical significance and overlaid on a high-resolution structural image. Following Table 1, the variable of interest is labeled, the color map is sensible for the data and is mapped with symmetric endpoints, and annotation clearly indicates the directionality of the contrast (i.e., “Novel–Standard”). This design provides excellent spatial localization for functional effects but is not without problems. The display does not portray uncertainty and has a remarkably low data-ink ratio due to the prominent (nondata) structural image and sparsity of actual data (Habeck and Moeller, 2011Habeck C. Moeller J.R. Brain Connect. 2011; 1: 99-103Crossref PubMed Scopus (16) Google Scholar). More crucially, the design encourages authors to hide results not passing a somewhat arbitrary statistical threshold. Given numerous correction methods and little consensus on the appropriate family-wise type I error rate (Lieberman and Cunningham, 2009Lieberman M.D. Cunningham W.A. Soc. Cogn. Affect. Neurosci. 2009; 4: 423-428Crossref PubMed Scopus (1005) Google Scholar), authors may arrive at a “convenient” threshold to reveal visually appealing and easily explained results. This design reduces a rich and complex data set to little more than a dichotomous representation (i.e., “significant or not?”) that suffers from all the limitations of all-or-none hypothesis testing (Harlow et al., 1997Harlow L.L. Mulaik S.A. Steiger J.H. What If There Were No Significance Tests?. Lawrence Erlbaum, Mahwah, NJ1997Google Scholar). Rather than threshold results, we suggest a dual-coding approach to represent uncertainty (Hengl, 2003Hengl, T. (2003). Proceedings of the 7th International Conference on GeoComputation, 8.Google Scholar). As shown in panel B, differences in beta estimates are mapped to color hue, and associated paired t statistics (providing a measure of uncertainty) are mapped to color transparency. Compared to panel A, no information is lost. Transparency is sufficient to determine structural boundaries and statistical significance is indicated with contours. However, substantial information is gained. The quality of the data is now apparent: large and consistent differences in betas are wholly localized to gray matter, while white matter and ventricular regions exhibit very small or very uncertain differences. In addition, isolated blobs of differential activation in panel A are now seen as the peaks of larger contiguous activations (often with bilateral homologs) that failed to meet significance criteria. The modified display also reveals regions in lateral parietal cortex, medial prefrontal cortex, and posterior cingulate cortex with reduced activation to novel stimuli compared to standard tones. These brain areas coincide with the so-called “default-mode network,” a system preferentially active when subjects engage in internal rather than external processes (Buckner et al., 2008Buckner R.L. Andrews-Hanna J.R. Schacter D.L. Ann. N Y Acad. Sci. 2008; 1124: 1-38Crossref PubMed Scopus (6979) Google Scholar). We hope to impress upon the reader the wealth of findings that can be revealed simply by unhiding data. To encourage the use of this approach, we provide sample MATLAB scripts for hue and transparency coding on our website (http://mialab.mrn.org/datavis). Along with increased annotation, panel B also displays the beta parameters for individual subjects, averaged over clusters of voxels passing significance (Figures 3Bb1 and 3Bb2). The 2D plots remove dependence on color mapping (which is more difficult for viewers to decode than position along an axis; Cleveland and McGill, 1985Cleveland W.S. McGill R. Science. 1985; 229: 828-833Crossref PubMed Scopus (375) Google Scholar) and allow us to access the data in greater detail. Scatter plots indicate the beta estimates for each condition (rather than just the difference), reveal the degree of variability across subjects (and the absence of outliers), and validate our “paired” statistical approach, because beta values covary across conditions. A single figure may portray experimental data painstakingly collected over months or even years. Rather than use standard designs such as bar plots and thresholded maps that hide these data, we, as authors, peer reviewers, and editors, can establish new standards for visualizations that reveal data and inform readers.