Trends 2011 And Beyond: Business Intelligence

Forrester continues to see ever-increasing levels of interest in and adoption of business intelligence (BI) platforms, applications, and processes. But while BI maturity in enterprises continues to grow, and BI tools have become more function-rich and robust, the promise of efficient and effective BI solutions remains challenging at best and elusive at worst. Why? Two main reasons: First, BI is all about best practices and lessons learned, which only come with years of experience; and second, earlier-generation BI approaches cannot easily keep up with ever-changing business and regulatory requirements. In the attached research document, Forrester reviews the top best practices for BI and predicts what the next-generation BI technologies will be. We summarize all of this in a single über-trend and best practice: agility. IT and business pros should adopt Agile BI processes, technologies, and architectures to improve their chances of delivering successful BI initiatives.

Business intelligence (BI) software has emerged as a hot topic in the past few years; in 2011, most companies will again focus their software investment plans on BI. More than 49% of the companies that responded to our most recent Forrsights Software Survey have concrete plans to implement or expand their use of BI software within the next 24 months. But being interested in BI software and spending money to adopt BI tools and processes do not necessarily translate into successful implementations: Forrester’s most recent BI maturity survey indicated that enterprise BI maturity levels are still below average (2.75 on a scale of 5, a modest 6% increase over 2009). Why are BI maturity levels so low, given the amount of money firms spend on it? Three factors contribute to this rift and can lead to less-than-successful BI initiatives:

  1. Implementing BI requires using best practices and building upon lessons learned.
  2. BI technologies and processes have not kept pace with business realities.
  3. Centralization has not led to agile, streamlined BI implementations.

Why don't traditional BI approaches often work? It’s possible to model, build, and implement business processes that will not require new features or other major enhancements for six to 12 months — sometimes even longer. This is especially true for many stable and well-defined back-office applications, such as budgeting, planning, and accounts payable/accounts receivable. But what we at Forrester call untamed processes (I've even heard some vendors refer to these as dark or hidden processes) change much more frequently, because some steps in these processes are either partially or completely manual and thus depend on less predictable external factors like human intervention. This is especially true when these processes involve external constituents, such as customers, prospects, partners, and regulators, over whom you have little or no control. Untamed business processes create challenges for standard BI approaches because:

  • Untamed processes are often invisible.
  • Traditional SDLC doesn’t work well for automating untamed processes.
  • Untamed processes often require new approaches and technologies.

Something radical needs to happen to make BI implementations more successful so that they can effectively support untamed business processes. And while Forrester will never say that agility will cure all of BI’s current ills, it certainly provides the most important best practices and leverages a key capability of the underlying BI technology to help close the gap that earlier-generation BI processes and technologies create. Don’t misinterpret what we’re recommending here — it’s not just about the Agile software development methodology, which is nothing new. But Agile development by itself is not enough for BI, so Forrester also recommends adopting multiple best practices and next-generation technologies to make BI more flexible.

In addition to Agile software development methodology, we recommend starting by adapting your organizational structures and enterprise culture for agility. No technology or processes can address BI challenges if a company’s organizational structure and enterprise culture are not already on firm, agile ground (in the detailed research document we describe five of these best practices). Once the organization is aligned for agility, the next step is to consider and implement Agile BI processes (in the detailed research document we describe three of these best practices)

Last, but not least, while some of these best practices can stand on their own, others require the application of next-generation BI technologies. In the past, BI vendors and BI application developers focused on business and operational functionality and architectural robustness. In most cases, these features have become commoditized. BI vendors and developers now need to concentrate on next-generation technologies; Forrester categorizes these technologies as agile, and defines four major subcategories of agility:

  • Automated (six emerging technologies).
  • Pervasive (six emerging technologies).
  • Unified (six emerging technologies).
  • Limitless (three emerging technologies).

Please read the research report for more details and comment.


Baby and bathwater

People are quick to criticize IT and BI teams for being too slow and costly. Although I agree BI needs to speed up to keep pace with the growing velocity of business, sometimes business needs to slow down. BI consists of two diametrically opposed activites: top-down, metrics-driven reporting and dashboarding where you know in advance what things you want to monitor and bottom-up, ad hoc analysis to answer unanticipated questions.

The key is not to abandon the top-down environment in favor of the bottom-up environment, which is the popular thing to do these days. Everyone is gravitating to low-cost, in-memory, visual analysis environments because they're cheap, quick, and easy to use, while forgetting the downsides, which is propagation of analytic silos and lack of enterprise information consistency.

The key challenge in the next ten years is figuring out ways to balance top-down and bottom-up activities so they complement, not undermine, each other. It's fine to go fast as long as you don't swerve off the road and risk fatal injuries. Don't throw the baby out with the bathwater.

I agree wholeheartedly that BI needs to be more agile to keep with the increasing velocity of business. Yet, untamed processes equal business chaos. While BI needs to speed up, business needs to slow down and think before it jumps.

Let's focus on the users

Boris, thanks for the research summary.

Wayne, I like what you're saying and agree that we need to find ways for BI's top-down and bottom-up activities to complement one another. Each has an appropriate use-case and in many cases can and will be kept separate. Take the traditional BI analysts who will gravitate towards the bottom-up tools. Or, the bulk of the enterprise for whom the top-down dashboards are immensely valuable and appropriate. And occasionally for certain engaged users or power users, they'll have a foot in either camp. The value in separating the use-cases and the users, is that it allows tools to focus on simplicity - avoid feature creep. For vendors, it's difficult doing both.

And that get's back to the future of BI ... we've got to focus on the users, their tasks and really make things easy, self evident, and fun.