I always predicted that Open Source BI has to reach critical mass before it becomes a viable alternative for large enterprise BI platforms. All the individual components (a mixture of Open Source BI projects and commercial vendor wrappers around them) are slowly but surely catching up to their bigger closed source BI brothers. Talend and Kettle (a Pentaho led project) offer data integration components like ETL, Mondrian and Palo (SourceForge projects) have OLAP servers, BIRT (an Eclipse project), Actuate, Jaspersoft and Pentaho have impressive reporting components, Infobright innovates with columnar dbms well suited for BI, and productized offerings from consulting companies like European based Engineering Ingegneria Informatica – SpagoBI – offer some Open Source BI component integration.
However, even large closed source BI vendors that acquired multiple BI components over the years still struggle with full, seamless component integration. So what chance do Open Source BI projects and vendors with independent leadership structure and often varying priorities have for integrating highly critical BI components such as metadata, data access layers, GUI, common prompting/sorting/ranking/filtering approaches, drill-throughs from one product to another, etc? Today, close to none. However, a potential consolidation of such products and technologies under one roof can indeed create a highly needed critical mass and give these individual components a chance to grow into large enterprise quality BI solutions.
I had an amazing client experience the other day. I searched long and hard for a client with flawless, perfect, 100% efficient and effective BI environment and applications. My criteria were tough and that's why it took me so long (I've been searching for as long as I've been in the BI business, almost 30 years). These applications had to be plug & play, involve little or no manual setup, be 100% automated, incorporate all relevant data and content, and allow all end users to self service every single BI requirement. Imagine my utter and absolute amazement when I finally stumbled on one.
The most remarkable part was that this was a very typical large enterprise. It grew over many years by multiple acquisitions, and as a result had many separate and disconnected front and back office applications, running on various different platforms and architectures. Its senior management suffered from a typical myopic attitude, mostly based on immediate gratification, caused by compensation structure that rewarded only immediate tangible results, and did not put significant weight and emphasis on long term goals and plans. Sounds familiar? If you haven't worked for one of these enterprises, the color of the sky in your world is probably purple.
Many years ago as I started researching and analyzing the differences between major BI vendors, one criterion that I always used was whether these vendors ate their own dog food. In other words, did a vendor executive team use the same solutions for data collection, building metrics and dashboards to run their own companies that they also tried to sell to their clients? Those who did tended to score higher in my evaluations.
The same guiding principle is applicable to Forrester: you have to eat your own dog food in order to convince the clients to buy your products and services. Hence, our methodologies, such as Forrester Waves are completely open and transparent (thank you, Doug Henschen, for recognizing this in your recent blog), and we encourage our clients to challenge us on every point made in our Waves.
If you haven’t yet heard the latest news on the American political scene, let me fill you in: Illinois Governor Rod Blagojevich has been arrested on charges of conspiracy to commit fraud and soliciting bribery. Among the alleged offenses is that the Governor planned to sell the Senate seat of President-elect Barack Obama to the highest bidder, or, if no offers met his expectations, to take the seat for himself for personal gain. One is reminded of the remark, often attributed (perhaps incorrectly) to Mark Twain, that the United States has “the best politicians money can buy.”
I know I am in the right business. Over 25 years ago, when I was a junior programmer on Wall Street, I heard the CEO of Citibank, Walter Wriston, say during one of the company meetings that “information about a transaction was going to become more important than a transaction itself”. I pondered on his prediction of the impending information revolution and decided to get into the business. I have not felt sorry ever since.
That is until now. I saw a good portion of my savings plan evaporate, some friends loosing their jobs on Wall Street in droves, and out of control media predicting, what basically amounts to, the end of the world (well, at least economic and social structures) as we know it.
What went wrong? While I am obviously not qualified to comment on the disastrous chain of events and a failure at every single link of the entire credit value chain (yes, I am not going to mention unreasonable social programs, uneducated consumers, greedy bankers and investors, ineffective risk rating agencies, and government regulation paralysis – did I miss anyone?), I am somewhat qualified to partially blame failed Business Intelligence at some levels of the credit value chain.
I am appalled at what has been happening in the economy lately. Seems like we are moving from one crisis management to another. First it was the oil price increase crisis, now it’s the credit markets crisis, while the oil crisis seems to have disappeared. There are revolutionary approaches to solving these crises being thrown around very lightly and carelessly these days: nationalization of certain industries, redistribution of wealth and other extremist approaches. Haven’t we learned from history? Don’t we know by now that revolutions do not work? It’s been proven time and time again in Soviet Union, China, Cuba and many other nations that revolutions only lead to disasters: terror, holocausts, starvation, turning societies and social structures upside down, and people leading miserable existence. I know. I lived in one of those countries. I do not want to live in another country going through revolution.
Most modern large enterprise Business Intelligence (BI) tools are very robust and feature rich these days. Up until a few years ago BI users could blame vendors for most of their BI ills. This is getting harder and harder to do. Many of the BI tools, especially the ones reviewed in our latest BI Wave, are very function rich, robust, stable and scalable. However, while the tools have really improved for the better over the last 5, typical BI issues and challenges remain the same as when I first tackled them as a BI programmer over 25 years ago: silo’d implementations, incomplete data sets, dirty data, poor management and governance, heavy reliance on IT, and many more.
We are right now in the middle of running a BI survey, exploring these and other BI issues. While the results are still pouring in, the preliminary findings are 100% supportive of the evidence we’ve collected qualitatively and anecdotally over the past few years:
Not all data is available in BI applications
Data is less than 100% trustworthy
BI applications are somewhat difficult to learn, use and navigate
Most of the reports and dashboards are developed by IT, not end users
Consistently rated as one of the most popular features of Forrester Events, one-on-one meetings give you the opportunity to discuss the unique technology issues facing your organization with Forrester analysts. Business & Technology Leadership Forum attendees may schedule up to two 20-minute one-on-one meetings with the Forrester analysts of their choice, depending on availability. Registered attendees will be able to schedule one-on-one meetings starting on Monday September 15, 2008. Book early!
Ever since our latest BI Wave was published a couple of weeks ago, I keep hearing comments about why we have not included evaluation of Excel as a BI tool. For example, Rajan Chandras, one of the contributing editors to the Intelligent Enterprise, poses really good arguments in his recent blog on why, when and how Excel can and should be used as a BI tool. Excellent question, everyone!
When Business Objects got acquired by SAP earlier this year, it made a statement that it plans to continue to remain an open, heterogeneous BI vendor, treating all partners equally. Apparently, all partners are not created equal – and, as we suspected and long predicted, this Business Objects strategy does not extend to its own parent.
Well, the cat's finally out of the bag. Efforts are already underway at SAP to improve the existing connectivity between Business Objects products and SAP applications. The improved connectivity that may result from these efforts will be very much optimized for Business Objects products only. SAP states that "SAP customers who instead decide to move forward with non-SAP third party BI tools will not benefit from these types of improvements and enhancements."