Next gen of metadata driven BI apps

We all struggle with complexity of designing, building and maintaining BI apps. Why? Among many other reasons, the simplest one is that there's just too many components involved. Just to name a few

  • Data sourcing
  • Data extraction
  • Data integration
  • Data cleansing
  • Data aggregation
  • Data modeling (star schemas, cubes)
  • Metrics management
  • Queries
  • Reports
  • Dashboards
  • Alerts
  • Delivery (portals, schedulers, emails, etc)

For years there were many attempts to automate some of these steps via metadata. So rather than than coding source to target SQL transformations or DDL for DW generation vendors came up with, what I know call "1st generation" metadata driven BI tools, such as

  • ETL tools where metadata auto-generated SQL scripts for data extraction, loading and transformation
  • BI tools where metadata auto-generated SQL for queries
  • Data modeling tools where metadata auto-generated logical data models and DDL for physical data models

But, the "2nd generation" metadata driven BI apps (note apps vs tools now) do much more. For example, they:

  • Use metadata to generate multi vendor apps (like BalancedInsight, Kalido and BIReady do), and having a single place where changes can be made
  • Use metadata to generate all three (ETL SQL, BI SQL, DW DDL, like Cognos, Wherescape, BIReady do), and having a single place where changes to all 3 can be made
  • Using metadata to generate report layouts (like Cognos does)
Read more

11 Meanings of Why-My-BI-Application-Is-Not-Useful

When a user of a BI application complains about the application not being useful - something that I hear way too often - what does that really mean? I can count at least 11 possible meanings, and potential reasons:

1. The data is not there, because

  • It's not in any operational sources, in which case the organization needs to implement a new app, a new process or get that data from an outside source
  • It is in an operational source, but not accessible via the BI application.

The data is there, but

2. It's not usable as is, because

  • There are no common definitions, common metadata
  • The data is of poor quality
  • The data model is wrong, or out of date

3. I can't find it, because I

  • Can't find the right report
  • Can't find the right metadata
  • Can't find the data
  • I don't have access rights to the data I am looking for

4. I don't know how to use my application, because I

  • Was not trained
  • Was trained, but the application is not intuitive, user friendly enough

5. I can't/don't have time do it myself - because I just need to run my business, not do BI !!! - and

  • I don't have support staff
  • I am low on IT priority list

6. It takes too long to

  • Create a report/query
  • Run/execute a report/query
Read more

How Do We Define a "BI Vendor"

My colleague, Holger Kisker, just posted a very insightful blog on the convergence of BI and BPM technologies. Yes, Holger, BPM vendors definitely have some BI capabilities. And so do some search vendors like Attivio, Endeca and Microsoft FAST Search. And so do some middleware vendors like TIBCO, Vitria and Software AG. And so do rules vendors like FairIsaac, PegaSystems. Should I go on? I have a list of hundreds of vendors that "say" they are a BI vendor.

But it’s not that simple. First of all, let’s define BI. In the last BI Wave we defined BI as “a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making”. To provide all these capabilities a vendor should have most of the necessary components such as data integration, data quality, master data management, metadata management, data warehousing, OLAP, reporting, querying, dashboarding, portal, and many, many others. In this broader sense only full BI stack vendors such as IBM, Oracle, SAP, Microsoft, SAS, TIBCO and Information Builders qualify.

Even if we define BI more narrowly as the reporting and analytics layer of the broader BI stack, we still want to include capabilities such as 11 ones we use to rate BI vendors in the BI Waves:

Read more