Over the last 25 years in the business I heard my share of BI horror stories: “we have over 20 different BI tools”, or “we have a few thousand reports in our BI application”. BI is very much a self fulfilling prophecy – “build it, and they will come”. As we popularize BI, and as technology becomes more scalable, more stable, more function rich and user-friendly - BI spreads like wildfire and often becomes uncontrollable.
I can’t help but to quote from one of my favorite books by a British author Jerome K. Jerome “Three Men In A Boat, To Say Nothing Of A Dog”. One of the reasons I love the book, in addition to it being one of the funniest novels I ever read, is that I can almost always find a very relevant humorous quote to just about any life or business situation. At the beginning of the book three British gentlemen are planning a vacation on a river boat. As they plan for how much food and supplies they should carry, they quickly realize that there isn’t a boat big enough to fit the dimensions of the Thames river to carry all that junk.
“How they pile the poor little craft mast-high with fine clothes and big houses; with useless servants, and a host of swell friends that do not care twopence for them, and that they do not care three ha'pence for; with expensive entertainments that nobody enjoys, with formalities and fashions, with pretence and ostentation, and with - oh, heaviest, maddest lumber of all! - the dread of what will my neighbour think, with luxuries that only cloy, with pleasures that bore, with empty show that, like the criminal's iron crown of yore, makes to bleed and swoon the aching head that wears it!
We all struggle with complexity of designing, building and maintaining BI apps. Why? Among many other reasons, the simplest one is that there's just too many components involved. Just to name a few
Data modeling (star schemas, cubes)
Delivery (portals, schedulers, emails, etc)
For years there were many attempts to automate some of these steps via metadata. So rather than than coding source to target SQL transformations or DDL for DW generation vendors came up with, what I know call "1st generation" metadata driven BI tools, such as
ETL tools where metadata auto-generated SQL scripts for data extraction, loading and transformation
BI tools where metadata auto-generated SQL for queries
Data modeling tools where metadata auto-generated logical data models and DDL for physical data models
But, the "2nd generation" metadata driven BI apps (note apps vs tools now) do much more. For example, they:
Use metadata to generate multi vendor apps (like BalancedInsight, Kalido and BIReady do), and having a single place where changes can be made
Use metadata to generate all three (ETL SQL, BI SQL, DW DDL, like Cognos, Wherescape, BIReady do), and having a single place where changes to all 3 can be made
Using metadata to generate report layouts (like Cognos does)
When a user of a BI application complains about the application not being useful - something that I hear way too often - what does that really mean? I can count at least 11 possible meanings, and potential reasons:
1. The data is not there, because
It's not in any operational sources, in which case the organization needs to implement a new app, a new process or get that data from an outside source
It is in an operational source, but not accessible via the BI application.
The data is there, but
2. It's not usable as is, because
There are no common definitions, common metadata
The data is of poor quality
The data model is wrong, or out of date
3. I can't find it, because I
Can't find the right report
Can't find the right metadata
Can't find the data
I don't have access rights to the data I am looking for
4. I don't know how to use my application, because I
Was not trained
Was trained, but the application is not intuitive, user friendly enough
5. I can't/don'thave time do it myself - because I just need to run my business, not do BI !!! - and
My colleague, Holger Kisker, just posted a very insightful blog on the convergence of BI and BPM technologies. Yes, Holger, BPM vendors definitely have some BI capabilities. And so do some search vendors like Attivio, Endeca and Microsoft FAST Search. And so do some middleware vendors like TIBCO, Vitria and Software AG. And so do rules vendors like FairIsaac, PegaSystems. Should I go on? I have a list of hundreds of vendors that "say" they are a BI vendor.
But it’s not that simple. First of all, let’s define BI. In the last BI Wave we defined BI as “a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making”. To provide all these capabilities a vendor should have most of the necessary components such as data integration, data quality, master data management, metadata management, data warehousing, OLAP, reporting, querying, dashboarding, portal, and many, many others. In this broader sense only full BI stack vendors such as IBM, Oracle, SAP, Microsoft, SAS, TIBCO and Information Builders qualify.
Even if we define BI more narrowly as the reporting and analytics layer of the broader BI stack, we still want to include capabilities such as 11 ones we use to rate BI vendors in the BI Waves:
The following question comes from many of our clients: what are some of the advantages and risks of implementing a vendor provided analytical logical data model at the start of any Business Intelligence, Data Warehousing or other Information Management initiatives? Some quick thoughts on pros and cons:
Leverage vendor knowledge from prior experience and other customers
May fill in the gaps in enterprise domain knowledge
Best if your IT dept does not have experienced data modelers
May sometimes serve as a project, initiative, solution accelerator
May sometimes break through a stalemate between stakeholders failing to agree on metrics, definitions
May sometimes require more customization effort, than building a model from scratch
May create difference of opinion arguments and potential road blocks from your own experienced data modelers
May reduce competitive advantage of business intelligence and analytics (since competitors may be using the same model)
Goes against “agile” BI principles that call for small, quick, tangible deliverables
Goes against top down performance management design and modeling best practices, where one does not start with a logical data model but rather
Defines departmental, line of business strategies
Links goals and objectives needed to fulfill these strategies
Defines metrics needed to measure the progress against goals and objectives
Defines strategic, tactical and operational decisions that need to be made based on metrics
Slowly but surely, with lots of criticism and skepticism, the business intelligence (BI) software-as-a-service (SaaS) market is gaining ground. It's a road full of peril — at least two BI SaaS startups have failed this year — but what software market segment has not seen its share of failures? Although I do not see a stampede to replace traditional BI applications with SaaS alternatives in the near future, BI SaaS does have a few legitimate use cases even today, such as complementary BI, in coexistence with traditional BI, BI workspaces, and BI for small and some midsize businesses.
In our latest BI SaaS research report we recommend the following structured approach to see if BI SaaS is right for you and if you are ready for BI SaaS:
Map your BI requirements and IT culture to one of five BI SaaS use cases
Evaluate and consider scenarios where BI SaaS may be a right or wrong fit for you
Select the BI SaaS vendor that fits your business, technical, and operational requirements, including your tolerance for risk
First we identified 5 following BI SaaS use cases.
Coexistence case: on-premises BI complemented with SaaS BI in enterprises
SaaS-centric case in enterprises: main BI application in enterprises committed to SaaS
SaaS-centric case in midmarket: main BI application in midsized businesses
Elasticity case: BI for companies with strong variations in activity from season to season
Power user flexibility case: BI workspaces are often considered necessary by power analysts
Our latest BI maturity survey results are in. We used exactly the same questions from our online BI maturity self assessment tool to survey over 200 Forrester clients. Now you can compare your own BI maturity level against your peers by using data from the survey.
In the self assessment tool and in the survey we ask over 30 questions in the following 6 categories
Data and technology
Our clients rated themselves on the scale of 1 to 5 (5, if they strongly agree with our statement or 1, if they strongly disagree). Here are the overall results. Keep in mind that these results do not evaluate BI maturity accross ALL business, but rather in businesses that are already pretty far ahead in their BI implementations (they are Forrester clients, they read our research reports, they talk to our research analysts):
I get many questions about the usage, pervasiveness, and adaption of mobile BI applications. What's a mobile BI application? Beyond a simple delivery of alerts, URLs, or actual reports via email - functionality that has existed for years - here are a few newer approaches to deliver BI on a mobile device:
The no brainer. In theory any mobile device equipped with a browser can access web based, thin client, HTML only BI applications. Yes, these BI apps will be mostly static, not interactive reports and dashboards. Navigation (scrolling, zooming, etc) will be quite awkward. But, this approach indeed requires no additional effort to deploy.
Customization. The next step up is to render each (or all) reports and dashboards to a format suitable to any mobile device in terms of screen size, usage of screen real estate, and mobile device specific navigation instrumentation. A variation of this approach is to create device specific navigation controls (thumb wheel or thumb button for Blackberries, up/down/left/right arrows for Palms, gestural manipulation for iPhone, etc). This obviously requires more development effort, but still no additional software.
I'd like to drill into some more details on my BI SaaS blog from September 2009. A key critical point to "what differentiates one BI SaaS vendor from another" discussion is what really constitutes multi-tenant architecture. Here are some initiall thoughts to stimulate the discussion:
DBMS. There's got to be back end, DBMS architecture that allows for one of the following:
Automatically generate a separate DBMS instance for each client
Use same DBMS instance for multiple clients, but automatically generate a set of unique tables for each client
Use same DBMS instance and tables for multiple clients, but automatically assign unique keys to to each client so that they can only update and retrieve their own rows
Application. Similar functionality has to exist in the application tier:
Automatically connect to the appropriate, client specific DBMS instance, or
Automatically use views that only point to client specific tables, or
Append "where" clause to each SQL statement to only retrieve client specific rows