I just came back from an exciting week in Orlando, FL, shuttling between SAP SAPPHIRE and IBM Cognos Forum conferences. Thank you, my friends at SAP and IBM for putting the two conferences right next to each other (time- and location-wise), and for saving me an extra trip!
Both conferences showed new and exciting products and both vendors are making great progress towards my vision of “next generation BI”: automated, pervasive, unified and limitless. I track about 20 different trends under these four categories, but there’s a particular one that is especially catching my attention these days. It went largely under covers at both conferences, and I was struggling with how to verbalize it, until my good friend and peer, Mark Albala, of http://www.info-sight-partners.com, put it in excellent terms for me in an email earlier today: it’s all about “pre-discovery” vs. “post-discovery” of data.
In my recent BI Belt Tightening For Tough Economic Times document I explored a few low-cost alternatives to traditional, mainstream, and typically relatively expensive Business Intelligence (BI) tools. While some of these alternatives indeed were a fraction of a cost of a characteristic large enterprise BI software license, there were even fewer truly zero cost options. But there were some. For example, you can:
Leverage and use no-cost bundled BI software already in-house.Small departments and workgroups may be able to leverage BI software that comes bundled at no additional cost with BI appliances, database management systems (DBMSes), and application licenses. You can consider using these few free licenses from Actuate, IBM Cognos, Information Builders, Jaspersoft, Microsoft, MicroStrategy, Panorama, Pentaho, and SAP Business Objects for additional functions such as testing, QA, and prototyping. While these few free licenses are just a drop in the bucket in a typical large enterprise BI license requirements, do look around and don’t waste money on BI products you may already have.
I always predicted that Open Source BI has to reach critical mass before it becomes a viable alternative for large enterprise BI platforms. All the individual components (a mixture of Open Source BI projects and commercial vendor wrappers around them) are slowly but surely catching up to their bigger closed source BI brothers. Talend and Kettle (a Pentaho led project) offer data integration components like ETL, Mondrian and Palo (SourceForge projects) have OLAP servers, BIRT (an Eclipse project), Actuate, Jaspersoft and Pentaho have impressive reporting components, Infobright innovates with columnar dbms well suited for BI, and productized offerings from consulting companies like European based Engineering Ingegneria Informatica – SpagoBI – offer some Open Source BI component integration.
However, even large closed source BI vendors that acquired multiple BI components over the years still struggle with full, seamless component integration. So what chance do Open Source BI projects and vendors with independent leadership structure and often varying priorities have for integrating highly critical BI components such as metadata, data access layers, GUI, common prompting/sorting/ranking/filtering approaches, drill-throughs from one product to another, etc? Today, close to none. However, a potential consolidation of such products and technologies under one roof can indeed create a highly needed critical mass and give these individual components a chance to grow into large enterprise quality BI solutions.
I had an amazing client experience the other day. I searched long and hard for a client with flawless, perfect, 100% efficient and effective BI environment and applications. My criteria were tough and that's why it took me so long (I've been searching for as long as I've been in the BI business, almost 30 years). These applications had to be plug & play, involve little or no manual setup, be 100% automated, incorporate all relevant data and content, and allow all end users to self service every single BI requirement. Imagine my utter and absolute amazement when I finally stumbled on one.
The most remarkable part was that this was a very typical large enterprise. It grew over many years by multiple acquisitions, and as a result had many separate and disconnected front and back office applications, running on various different platforms and architectures. Its senior management suffered from a typical myopic attitude, mostly based on immediate gratification, caused by compensation structure that rewarded only immediate tangible results, and did not put significant weight and emphasis on long term goals and plans. Sounds familiar? If you haven't worked for one of these enterprises, the color of the sky in your world is probably purple.
Many years ago as I started researching and analyzing the differences between major BI vendors, one criterion that I always used was whether these vendors ate their own dog food. In other words, did a vendor executive team use the same solutions for data collection, building metrics and dashboards to run their own companies that they also tried to sell to their clients? Those who did tended to score higher in my evaluations.
The same guiding principle is applicable to Forrester: you have to eat your own dog food in order to convince the clients to buy your products and services. Hence, our methodologies, such as Forrester Waves are completely open and transparent (thank you, Doug Henschen, for recognizing this in your recent blog), and we encourage our clients to challenge us on every point made in our Waves.
If you haven’t yet heard the latest news on the American political scene, let me fill you in: Illinois Governor Rod Blagojevich has been arrested on charges of conspiracy to commit fraud and soliciting bribery. Among the alleged offenses is that the Governor planned to sell the Senate seat of President-elect Barack Obama to the highest bidder, or, if no offers met his expectations, to take the seat for himself for personal gain. One is reminded of the remark, often attributed (perhaps incorrectly) to Mark Twain, that the United States has “the best politicians money can buy.”
I know I am in the right business. Over 25 years ago, when I was a junior programmer on Wall Street, I heard the CEO of Citibank, Walter Wriston, say during one of the company meetings that “information about a transaction was going to become more important than a transaction itself”. I pondered on his prediction of the impending information revolution and decided to get into the business. I have not felt sorry ever since.
That is until now. I saw a good portion of my savings plan evaporate, some friends loosing their jobs on Wall Street in droves, and out of control media predicting, what basically amounts to, the end of the world (well, at least economic and social structures) as we know it.
What went wrong? While I am obviously not qualified to comment on the disastrous chain of events and a failure at every single link of the entire credit value chain (yes, I am not going to mention unreasonable social programs, uneducated consumers, greedy bankers and investors, ineffective risk rating agencies, and government regulation paralysis – did I miss anyone?), I am somewhat qualified to partially blame failed Business Intelligence at some levels of the credit value chain.
I am appalled at what has been happening in the economy lately. Seems like we are moving from one crisis management to another. First it was the oil price increase crisis, now it’s the credit markets crisis, while the oil crisis seems to have disappeared. There are revolutionary approaches to solving these crises being thrown around very lightly and carelessly these days: nationalization of certain industries, redistribution of wealth and other extremist approaches. Haven’t we learned from history? Don’t we know by now that revolutions do not work? It’s been proven time and time again in Soviet Union, China, Cuba and many other nations that revolutions only lead to disasters: terror, holocausts, starvation, turning societies and social structures upside down, and people leading miserable existence. I know. I lived in one of those countries. I do not want to live in another country going through revolution.
Most modern large enterprise Business Intelligence (BI) tools are very robust and feature rich these days. Up until a few years ago BI users could blame vendors for most of their BI ills. This is getting harder and harder to do. Many of the BI tools, especially the ones reviewed in our latest BI Wave, are very function rich, robust, stable and scalable. However, while the tools have really improved for the better over the last 5, typical BI issues and challenges remain the same as when I first tackled them as a BI programmer over 25 years ago: silo’d implementations, incomplete data sets, dirty data, poor management and governance, heavy reliance on IT, and many more.
We are right now in the middle of running a BI survey, exploring these and other BI issues. While the results are still pouring in, the preliminary findings are 100% supportive of the evidence we’ve collected qualitatively and anecdotally over the past few years:
Not all data is available in BI applications
Data is less than 100% trustworthy
BI applications are somewhat difficult to learn, use and navigate
Most of the reports and dashboards are developed by IT, not end users
Consistently rated as one of the most popular features of Forrester Events, one-on-one meetings give you the opportunity to discuss the unique technology issues facing your organization with Forrester analysts. Business & Technology Leadership Forum attendees may schedule up to two 20-minute one-on-one meetings with the Forrester analysts of their choice, depending on availability. Registered attendees will be able to schedule one-on-one meetings starting on Monday September 15, 2008. Book early!