Advanced Data Visualization - A Critical BI Component

As one of the industry-renowned data visualization experts Edward Tufte once said, “The world is complex, dynamic, multidimensional; the paper is static, flat. How are we to represent the rich visual world of experience and measurement on mere flatland?” Indeed, there’s just too much information out there for all categories of knowledge workers to visualize it effectively. More often than not, traditional reports using tabs, rows, and columns do not paint the whole picture or, even worse, lead an analyst to a wrong conclusion. Firms need to use data visualization because information workers:

  • Cannot see a pattern without data visualization. Simply seeing numbers on a grid often does not convey the whole story — and in the worst case, it can even lead to a wrong conclusion. This is best demonstrated by Anscombe’s quartet where four seemingly similar groups of x/y coordinates reveal very different patterns when represented in a graph.
  • Cannot fit all of the necessary data points onto a single screen. Even with the smallest reasonably readable font, single-line spacing, and no grid, one cannot realistically fit more than a few thousand data points on a single page or screen using numerical information only. When using advanced data visualization techniques, one can fit tens of thousands (an order-of-magnitude difference) of data points onto a single screen. In his book The Visual Display of Quantitative Information, Edward Tufte gives an example of more than 21,000 data points effectively displayed on a US map that fits onto a single screen.
Read more

Use Cases For Specific BI Tools

I get the following question very often. What are the best practices for creating an enterprise reporting policy as to when to use what reporting tool/application? Alas, as with everything else in business intelligence, the answer is not that easy. The old days of developers versus power users versus casual users are gone. The world is way more complex these days. In order to create such a policy, you need to consider the following dimensions:

  •  Report/analysis type
    • Historical (what happened)
    • Operational (what is happening now)
    • Analytical (why did it happen)
    • Predictive (what might happen)
    • Prescriptive (what should I do about it)
    • Exploratory (what's out there that I don't know about)
  • Interaction types
    • Looking at static report output only
    • Lightly interacting with canned reports (sorting, filtering)
    • Fully interacting with canned reports (pivoting, drilling)
    • Assembling existing report, visualizations, and metrics into customized dashboards
    • Full report authoring capabilities
  • User types
    • Internal
    • External (customers, partners)
  • Data latency
    • Real time
    • Near-real time
    • Batch
  • Report latency, as in need the report:
    • Now
    • Tomorrow
    • In a few days
    • In a few weeks
  • Decision types
    • Strategic (a few complex decisions/reports per month)
    • Tactical (many less-complex decisions/reports per month)
    • Operational (many complex/simple decisions/reports per day)
Read more

Self-Service BI

Traditional BI approaches and technologies — even when using the latest technology, best practices, and architectures — almost always have a serious side effect: a constant backlog of BI requests. Enterprises where IT addresses more than 20% of BI requirements will continue to see the snowball effect of an ever-growing BI requests backlog. Why? Because:

  • BI requirements change faster than an IT-centric support model can keep up. Even with by-the-book BI applications, firms still struggle to turn BI applications on a dime to meet frequently changing business requirements. Enterprises can expect a life span of at least several years out of enterprise resource planning (ERP), customer relationship management (CRM), human resources (HR), and financial applications, but a BI application can become outdated the day it is rolled out. Even within implementation times of just a few weeks, the world may have changed completely due to a sudden mergers and acquisitions (M&A) event, a new competitive threat, new management structure, or new regulatory reporting requirements.
Read more

Everything You Wanted To Know But Were Afraid To Ask About BI

How does an enterprise — especially a large, global one with multiple product lines and multiple enterprise resource planning (ERP) applications — make sense of operations, logistics, and finances? There’s just too much information for any one person to process. It’s business intelligence (BI) to the rescue! But what is BI, and how does BI differ from reporting and management information systems (MIS)? What is the business impact, and what are the costs versus the benefits? What is the appropriate strategy for implementing BI and achieving continued BI success? Our new report will give business and IT executives an understanding of the four critical phases of strategizing around BI to achieve business goals — or “everything you wanted to know but were afraid to ask” about BI. Here’s a sneak preview of the kinds of topics the report covers and the kinds of BI questions one needs to ask in order to build an effective and efficient enterprise BI environment:

  1. Prepare For Your BI Program
    1. The future of BI is all about agility. IT no longer has exclusive control of BI platforms, tools, and applications; business users demand more empowerment (or make empowered changes without IT involvement), and previously unshakable pillars of the BI foundation such as relational databases are quickly being supplemented with alternative BI platforms. It’s no longer business as usual. Ask yourself:
      1. What are the main business and IT trends driving BI?
      2. What are the latest BI technologies that I need to know about?
      3. What’s out there beyond traditional BI?
Read more

Data Discovery And Exploration - IBM Acquires Vivisimo

Today IBM announced its plans to acquire Vivisimo - an enterprise search vendor with big data capabilities. Our research shows that only 1% to 5% of all enterprise data is in a structured, modeled format that fits neatly into enterprise data warehouses (EDWs) and data marts. The rest of enterprise data (and we are not even talking about external data such as social media data, for example) may not be organized into structures that easily fit into relational or multidimensional databases. There’s also a chicken-and-the-egg syndrome going on here. Before you can put your data into a structure, such as a database, you need to understand what’s out there and what structures do or may exist. But in order for you to explore the data in the first place, traditional data integration technologies require some structures to even start the exploration (tables, columns, etc). So how do you explore something without a structure, without a model, and without preconceived notions? That’s where big data exploration and discovery technologies such as Hadoop and Vivisimo come into play. (There are many others vendors in this space as well, including Oracle Endeca, Attivio, and Saffron Technology. While these vendors may not directly compete with Vivisimo and all use different approaches and architectures, the final objective - data discovery - is often the same.) Data exploration and discovery was one of our top 2012 business intelligence predictions. However, it’s only a first step in the full cycle of business intelligence and

Read more

To Be (To Cloud) Or Not To Be (Not To Cloud) BI

My colleagues and I have just completed yet another engagement with a large client — one of dozens recently — who was facing a to be or not to be decision: whether to move its BI platform and applications to the cloud. It’s a very typical question that our clients are asking these days, mainly for the following two reasons:

  1. In many cases, their current on-premises BI solutions are too inflexible to support the business now, much less in the future.
  2. The relative success of cloud-based CRM (SFDC and others) solutions may indicate that cloud offers a better alternative.

These clients put these two statements together and make the reasonable assumption that cloud BI will solve many of the current BI challenges that cloud-based CRM solved. Reasonable? Yes. Correct? Not so fast — the only correct answer is “It depends.”

Let’s take a couple of steps back. First, let’s define applications or packaged solutions vs. platforms (because BI requires both).

Packaged solutions

  • Subscribe to a solution-like CRM
  • Provide standard business functions to all customers (which makes it different from “hosting;” see below)
  • Difficult to tailor to specific needs
  • Usually are used synonymously (but incorrectly, see below) with software-as-a-service (SaaS)

 Platforms for building solutions

  • Subscribe to tools and resources to build solutions like CRM
  • Provide standard technical functions to developers
  • Contain limited, if any, business application functionality
  • Usually labeled either as platform-as-a-service (PaaS) or infrastructure-as-a-service (IaaS).
Read more

What Is ADV And Why Do We Need It?

As one of the industry-renowned data visualization experts Edward Tufte once said, “The world is complex, dynamic, multidimensional; the paper is static, flat. How are we to represent the rich visual world of experience and measurement on mere flatland?” There’s indeed just too much information out there to be effectively analyzed by all categories of knowledge workers. More often than not, traditional tabular row-and-column reports do not paint the whole picture or — even worse — can lead an analyst to a wrong conclusion. There are multiple reasons to use data visualization; the three main ones are that one:

  • Cannot see a pattern without data visualization. Simply seeing numbers on a grid often does not tell the whole story; in the worst case, it can even lead one to a wrong conclusion. This is best demonstrated by Anscombe’s quartet, where four seemingly similar groups of x and y coordinates reveal very different patterns when represented in a graph.
  • Cannot fit all of the necessary data points onto a single screen. Even with the smallest reasonably readable font, single line spacing, and no grid, one cannot realistically fit more than a few thousand data points using numerical information only. When using advanced data visualization techniques, one can fit tens of thousands data points onto a single screen — a difference of an order of magnitude. In The Visual Display of Quantitative Information, Edward Tufte gives an example of more than 21,000 data points effectively displayed on a US map that fits onto a single screen.
Read more

Top 10 Business Intelligence Predictions For 2012

Demands by users of business intelligence (BI) applications to "just get it done" are turning typical BI relationships, such as business/IT alignment and the roles that traditional and next-generation BI technologies play, upside down. As business users demand more control over BI applications, IT is losing its once-exclusive control over BI platforms, tools, and applications. It's no longer business as usual: For example, organizations are supplementing previously unshakable pillars of BI, such as tightly controlled relational databases, with alternative platforms. Forrester recommends that business and IT professionals responsible for BI understand and start embracing some of the latest BI trends — or risk falling behind.

Traditional BI approaches often fall short for the two following reasons (among many others):

  • BI hasn't fully empowered information workers, who still largely depend on IT
  • BI platforms, tools and applications aren't agile enough
Read more

BI In The Cloud: Separating Facts From Fiction

“… and they lived happily ever after.” This is the typical ending of most Hollywood movies, which is why I am not a big fan. I much prefer European or independent movies that leave it up to the viewer to draw their own conclusions. It’s just so much more realistic. Keep this in mind, please, as you read this blog, because its only purpose is to present my point of view on what’s happening in the cloud BI market, not to predict where it’s going. I’ll leave that up to your comments — just like your own thoughts and feelings after a good, thoughtful European or indie movie.

Market definition

First of all, let’s define the market. Unfortunately, the terms SaaS and cloud are often used synonymously and therefore, alas, incorrectly.

  • SaaS is just a licensing structure. Many vendors (open source, for example) offer SaaS software subscription models, which has nothing to do with cloud-based hosting.
  • Cloud, in my humble opinion, is all about multitenant software hosted on public or private clouds. It’s not about cloud hosting of traditional software innately architected for single tenancy.
Read more

Oracle Leapfrogs BI Competitors By Acquiring Endeca

This is a very smart move by Oracle. Until the Siebel and Hyperion acquisitions, Oracle was not a leader in the BI and analytics space. Those acquisitions put them squarely in the top three together with IBM and SAP. However, until this morning, Oracle played mostly in the traditional BI space: reporting, querying, and analytics based on relational databases. But these mainstream relational databases are an awkward fit for BI. You can use them, but it requires lots of tuning and customization and constant optimization — which is difficult, time-consuming, and costly. Unfortunately, row-based RDBMSes like IBM DB2, Microsoft SQL Server, Oracle, and Sybase ASE were originally designed and architected for transaction processing, not reporting and analysis. In order to tune such a RDBMS for BI usage, specifically data warehousing, architects usually:

  • Denormalize data models to optimize reporting and analysis.
  • Build indexes to optimize queries.
  • Build aggregate tables to optimize summary queries.
  • Build OLAP cubes to further optimize analytic queries.
Read more