Yet another rumor crossed the wires today about Business Objects getting ready for a takeover. These rumors have come up before, and it may be nothing else but a routine exercise that BO and other vendors go through to test the market periodically.
As I already blogged back in March '07, there's no denying however that the business intelligence (BI) and business performance solutions (BPS) markets are consolidating. Last year Microsoft bought ProClarity; this year Oracle bought Hyperion, Business Objects acquired Carthesis, SAP acquired Outlooksoft, and most recently Cognos acquired Applix.
I predict that within 2-3 years there will be five major BI vendors carving up the BI and BPS market: Microsoft, IBM, SAP, Oracle and HP. There'll be much M&A activity seen from these vendors in the near future.
Here's how I see it playing out:
HP will probably make the next big move by acquiring somebody like Business Objects, Cognos or SAS. HP is pushing into BI market very strongly: It repositioned NeoView as a strong data warehouse (DW) platform and bought Knightsbridge — leading BI boutique strategy and implementation consultancy. It'd make a lot of sense for HP to pick up BO or SAS, since in one transaction HP would get the entire BI and PM suite. Cognos would be the second choice for them, since Cognos does not have operational ETL and data quality offerings.
SAP told us repeatedly that they can't justify very large acquisitions — it doesn't fit their model. So Microstrategy, Actuate or Information Builders would be more obvious pick up choices for them.
I sincerely hope that Oracle is more than busy with Hyperion integration and more than set in the BI and BPS markets for a while. I don't see the next big move coming from them.
Today Cognos announced its intention to acquire Applix, Inc for a cash tender offer of $17.87 per share, or approximately $339 million.
This acquisition is primarily about performance management, and I quote my colleague, Paul Hamerman: “This continues a trend of rapid consolidation in the business performance solutions space, following acquisitions of pure plays by larger BI and ERP vendors. Applix has been a stellar performer as a flexible platform for customers to rapidly implement applications for planning, budgeting, forecasting and business performance analytics. Although Cognos has been a leader in performance management solutions, the Applix technology gives it an opportunity to refresh its offerings and more aggressively sell to the midmarket.”
The BI side of this acquisition is all about TM1 OLAP technology. Cognos’s Powerplay OLAP product has been trailing Microsoft SQLServer Analysis Services and Hyperion (recently acquired by Oracle) Essbase in market adoption. With TM1, a memory-based OLAP cube, Cognos achieves several objectives:
Leapfrogs Microsoft and Oracle with one of the fastest read/write OLAP technologies on the market.
Provides an enterprise grade in-memory OLAP alternative to fast-growing and increasingly popular QlikTech, which is more of a departmental solution.
Provides additional midmarket entry points for Cognos
The future of the rest of the Applix BI products — reporting, querying, dashboards, and data integration — is quite murky. Cognos will face the same challenge as Oracle now does with the Hyperion acquisition: Establishing a strategy and a road map of overlapping product portfolio integration or sunsetting/retirement.
I get many questions on dashboards and scorecards and the role these tools play in BI (Business Intelligence). If we use Forrester’s definition of BI — a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information — then we see that dashboards are just the tip of the BI iceberg. One cannot build “just a dashboard”, without considering, architecting and implementing many other necessary BI layers and components such as data integration (ETL, data quality, etc), analytics (OLAP), metrics management, and many supporting components such as collaboration, knowledge management, metadata and master data management, and others. So that’s the first key takeaway: do not be fooled by 2nd tier dashboard vendor claims that one can implement an enterprise wide dashboard easily and inexpensively.
Let’s start with definitions, since I see the terms dashboards and scorecards used interchangeably:
Dashboards are just one style of interactive user interface, designed to deliver historical, current, and predictive information typically represented by key performance indicators (KPIs) using visual cues to focus user attention on important conditions, trends and exceptions.
Scorecards are a type of a dashboard which link KPIs to goals, objectives and strategies. Many scorecards follow a certain methodology, such as Balanced Scorecard, Six Sigma, Capability Maturity Models, etc.
Other types of dashboards include Business Activity Monitoring (BAM) dashboards and visualizations of data / text mining operations.
There’s tug of war going on in the world of BI. On the one hand we have IT whose mission it is to manage and protect enterprise information assets, and on the other side there are end users who just want the data when they want it, and in the shape and form that they want it, without any limitations.
Traditional, mainstream BI vendors have catered primarily to IT target audience. These vendors will disagree, but take one look at their complex architectures, multiple layers and components, integration and support requirements, and you can’t help but agree that these are IT tools that can be used to create end user applications.
On the other hand I am seeing am emergence of smaller BI vendors that cater directly to the end users. They pitch simplicity, flexibility and little or no reliance on IT. True, these vendors do not have large enterprise functions like metadata, semantic layers, robust security and scalability, so I do not see them as enterprise-level, but rather departmental, focused solutions. Yet, the appeal to end users is undeniable.
Finding a compromise – satisfying all typical IT requirements, while empowering the end users - remains an elusive goal, and hence an opportunity for all BI vendors.
I’d like to hear what my colleagues out there think about the convergence of structured and unstructured data business intelligence. Here are the intersects as I see them. I see two types of BI paradigms emerging in the future:
Structured OLAP will continue to be just that – structured, as far as the process and UI are concerned. However, to become more effective, we will need to bring unstructured data into the analysis, in a way that is transparent to the end user. For example, as we are creating customer segmentation analysis for a marketing campaign, in addition to structured data such as customer demographics and prior buying behavior, we’d want to bring in comments hidden in customer email and voice mail requests. In an ideal environment, the OLAP engine will automatically match these emails to a customer dimension and quantify and qualify comments into star schema facts (number of requests) and dimensions (request types).
Combination of search and light-weight query used for ad-hoc research and analysis. Here, a familiar search text box should be the main UI, however, the engine should be smart enough to a) quantify and qualify unstructured results into facts and dimensions – a so called guided search, or b) recognize that the request is actually about data stored in a structured repository and automatically return search results via OLAP, cross tab or tabular report format.
For years I've been predicting that relational DBMS will run out of steam when it comes to effectively managing and manipulating very large, heterogeneous (structured + unstructured) data sets for business intelligence. First, RDBMS were never designed and optimized for unstructured data (not just XML, which is structured data in my definition, but truly unstructured text pages). Second, there's just too much overhead and cost in RDBMS for handling OLTP functions. The result: search index DBMS will be king in BI and DSS in the future.
Today’s announcements that Microsoft may be buying Yahoo came several weeks early. On May 17th I would’ve gone on the record at Forrester IT Forum in Nashville by saying the following, and I quote from my presentation paper: “DBMS/BI vendor may buy a search company, to address the trend of increasing importance of unstructured data in BI and to obtain an early leading position in the space. I know it should be Oracle or IBM, but it probably won’t, since these guys will never admit that their relational DBMS cannot do something. Microsoft is a more likely contender since they know they won’t leapfrog IBM or Oracle in relational DBMS and they could use this opportunity to stick one to Google too.”
I thought Microsoft would buy somebody like Fast Search, but I guess that was too small for them.
Remember George Costanza from a Seinfeld episode where he was pulling his hair out about “the two worlds colliding”? He was agonizing over the world of his girlfriends and the world of his friends that should never mix. In my world, process and data, separate disciplines until recently, are now “colliding”. While some of the vendors have already been toying with the convergence of both disciplines (IBM, Oracle, SAP), today’s announcement by Tibco that it will acquire a Spotfire, is the first transaction that will merge a pureplay middleware vendor with a pureplay BI vendor (a convergence that Forrester’s been predicting for almost a year, please see our Business Intelligence Meets BPM In The Information Workplace research document. But by acquiring Spotfire, Tibco has actually achieved more than one goal.
Being efficient is no longer enough. Enterprises can no longer stay competitive just by squeezing more efficiencies from operational applications, including workflow, business process management (BPM) and business rules engine (BRE) — business intelligence applications are needed to become more effective. For example, while workflow and rules are be used to efficiently process a customer credit application, Business Intelligence analytics are needed to effectively segment customer population and extend the credit offer to a much more targeted customer segment for a better response, cross-sell/up-sell ratios.
The actual convergence of process and data. The other slant is the natural interdependency of process and data from two angles: a) one needs data to feed and enrich the business process and process rules, and b) an event (an alert, for example) triggered by a data condition has to go into a process so that it can be followed up and acted on.
I’ve recently conducted research on the issues of VLDB (very large databases) and how it affects BI, since the challenges of reporting and analyzing Gb data sets are very different from reporting and analyzing multi Tb data sets. Among many other interesting findings and conclusions I uncovered the following approaches to handling VLDB challenges as they relate to BI:
Generic solutions by DBMS vendors:
Share everything vs. share nothing architecture
Caching, in-memory databases
Specialized file systems
Indexing (bit-map vs. B-tree)
BI-specific solutions by BI and other vendors:
ROLAP and reporting tool specific SQL optimization
Alternate DBMS (such as search indexes and vector DBMS)
I predict that for the foreseeable future, spreadsheets will remain the most widely used enterprise application. The widespread adoption isn't hard to understand — spreadsheets are powerful and flexible, yet intuitive and easy to use and learn. Plus, ad hoc applications and spreadsheet models isolate users from constant reliance on IT and incur low costs. Since the early days of BI, spreadsheets have played a natural and major role in the BI process/architecture, including:
I've been in the BI business for over 25 years so I've seen many ups and downs in the BI cycle. We are definitely in the "up" cycle these days. I see two main reasons for it:
Enterprises can no longer stay competitive just by squeezing more efficiencies from operational applications – business intelligence applications are needed to become more effective.
Digital data (structured and unstructured) volumes are growing at 30% a year, and will be reaching zetabyte sizes by year 2010 – that’s a number with 21 zeros! Solid BI implementations will be critical to successfully turn that data into useful information.
Does anyone have any comments on where we are in the BI cycle and what are some of the more recent key drivers?