Advanced Data Visualization - A Critical BI Component

As one of the industry-renowned data visualization experts Edward Tufte once said, “The world is complex, dynamic, multidimensional; the paper is static, flat. How are we to represent the rich visual world of experience and measurement on mere flatland?” Indeed, there’s just too much information out there for all categories of knowledge workers to visualize it effectively. More often than not, traditional reports using tabs, rows, and columns do not paint the whole picture or, even worse, lead an analyst to a wrong conclusion. Firms need to use data visualization because information workers:

  • Cannot see a pattern without data visualization. Simply seeing numbers on a grid often does not convey the whole story — and in the worst case, it can even lead to a wrong conclusion. This is best demonstrated by Anscombe’s quartet where four seemingly similar groups of x/y coordinates reveal very different patterns when represented in a graph.
  • Cannot fit all of the necessary data points onto a single screen. Even with the smallest reasonably readable font, single-line spacing, and no grid, one cannot realistically fit more than a few thousand data points on a single page or screen using numerical information only. When using advanced data visualization techniques, one can fit tens of thousands (an order-of-magnitude difference) of data points onto a single screen. In his book The Visual Display of Quantitative Information, Edward Tufte gives an example of more than 21,000 data points effectively displayed on a US map that fits onto a single screen.
  • Cannot effectively show deep and broad data sets on a single screen. Fitting in and analyzing hundreds or thousands of columns of attributes (dimensions in BI speak) is an enormous challenge. Imagine a typical drug trial conducted by a pharmaceutical company where each patient has thousands of attributes: physical, psychological, genetic, behavioral, etc. Analysts looking for patterns, dependencies, and correlations typically need to run the data through complex statistical models before they can find a pattern or correlation. Building such models and running them through millions of rows of data can be time-consuming and can tax even the most advanced software and hardware resources. But in a technique often used in the pharma industry, reducing each data point in a column to a single pixel and color-coding pixels according to their value ranges can let an analyst relatively easily visualize and identify a pattern and then quickly zoom in to research the details. 
How is advanced data visualization (ADV) is different from earlier generations of data visualizations? Many corporations have effectively used — and will continue to use — traditional business graphics, such as bar charts and pie charts. At the next level, modern technologies have enabled the use of more dynamic and interactive business graphics, such as real-time dashboards and charts that update automatically as the data changes. (These new technologies also include new types of displays that make the high resolution necessary for ADV possible.) Now, through ADV, potential exists for nontraditional and more visually rich approaches, especially in regard to more complex (i.e., thousands of dimensions or attributes) or larger (i.e., billions of rows) data sets, to reveal insights not possible through conventional means. Forrester differentiates ADV from static graphs and charts along six capabilities, as follows:
  1. Dynamic data content. 
  2. Visual querying. 
  3. Multiple-dimension, linked visualization. 
  4. Animated visualization.
  5. Personalization. 
  6. Business-actionable alerts. 
Navigating the ADV landscape requires evaluating significantly more features than the six key ADV capabilities described in the previous section. In our latest research, Forrester identified numerous functional and technical capabilities businesses need to architect, design, build, and implement ADV applications. Forrester recommends starting your evaluation of ADV platforms by defining your requirements for the following functionality:
  • Types of graphs, charts and other visualizations. 
  • Tufte’s microcharts. 
  • Cockpit gauges. 
  • Visual query.
  • Visual exploration. 
  • Geospatial representations. 
  • Modes of interaction.
  • Storyboarding fit for client and boardroom-level presentations.
  • Data latency.
  • Data granularity based on your requirements. 
Then make sure that the ADV platform meshes with your technical architecture. Technical architecture is a key differentiator for ADV tool capabilities. Forrester identified eight categories of ADV technical architecture capabilities through posing the following questions:
  • What analytical engines does the ADV platform support? How does it access and process data?
  • Is there an intermediate storage platform? 
  • How is the in-memory data model managed?
  • What types of data can the ADV platform analyze?
  • Does the ADV platform support write-backs?
  • What platform/technology is the ADV output based on?
  • What, if any, ADV-specific programming language is used?
  • What are the ADV platform’s integration capabilities?

Last but not least, as you venture down the ADV road, Forrester recommends paying at least equal (if not more) attention to ADV best practices as you do to technology. In our other research, Forrester has identified multiple such practices including screen layouts, data-to-ink ratios, appropriate use of text and labels, using similar sequencing of objects, using parallel scales, minimizing the use of color, showing causality, and many more.

Read more about our ADV research and a detailed evaluation of 15 top ADV vendors here. And always remember: a picture speaks a thousand words!

Comments

I'm curious why Forrester

I'm curious why Forrester recommends minimizing the use of color. Color can be one of the most powerful ways to represent data.

Also, as I'm sure you know, treemaps can display upwards of 1 million data points or more on a single screen. With monitors increasing in size & resolution, the amount of data points that can be represented on a single screen continues to climb.

Trevor, sure, colors can tell

Trevor, sure, colors can tell a very powerful story, unless you are color blind. And that's 8-10% of world male population (7% in the US). So your story may be falling on deaf ears - i mean color blind eyes :-)

Yes, good point about tree maps.

Cheers!

Color blindness can be corrected for

Only 0.00001% of the population are totally color blind (monochromatic color blindness). The rest of people who are color blind see _some_ colors.

The advantage of interactive data visualizations over print is that you can change the color scheme. For instance, for those with red-green color blindness, instead of using a red-green color scheme, they can use a blue-yellow color scheme. The only problem is if the data visualization doesn't allow people to change color schemes. But this is a user interface problem, not a problem with using color.

For some visualizations, you can also use iconography or other elements to add duplicate information so the visualization degrades gracefully for color blind people. This allows those who can see the colors to take advantage of the preattentive processing of colors in our brains, while allowing those with color blindness to use attentive processing.

Color is the most powerful visualization attribute; removing it limits visualizations too much. Thus, the advice shouldn't be to avoid using color, but to ensure that you provide alternatives for color blind people.

Any thoughts on Stephen Few's

Any thoughts on Stephen Few's observations about your ADV forrester report. Please see http://www.perceptualedge.com/blog/?p=1277

Prasad, thanks for reaching

Prasad, thanks for reaching out. FYI, it is against Forrster policy to comment on other analyst firms. It is also against Forrester policy to have vendors influence our Waves. Our inclusion criteria are transparent and objective and are clearly stated in the report. Our evaluation methodology with clearly defined scales is also transparent (see the detailed model attached to the report), so we can stand by the results as 100% objective, not based on subjective opinions, but rather simple facts. If you have questions or comments about any aspect of our research reports, I'll be very happy to provide feedback. Best!

Dude, you have really great

Dude, you have really great info about Advanced Data Visualization, Thanks for sharing regarding A Critical BI Component; Great post.

Patricia Hall - www.mobileapplicationdevelopmentservices.com