Forrester continues to see ever-increasing levels of interest in and adoption of business intelligence (BI) platforms, applications, and processes. But while BI maturity in enterprises continues to grow, and BI tools have become more function-rich and robust, the promise of efficient and effective BI solutions remains challenging at best and elusive at worst. Why? Two main reasons: First, BI is all about best practices and lessons learned, which only come with years of experience; and second, earlier-generation BI approaches cannot easily keep up with ever-changing business and regulatory requirements. In the attached research document, Forrester reviews the top best practices for BI and predicts what the next-generation BI technologies will be. We summarize all of this in a single über-trend and best practice: agility. IT and business pros should adopt Agile BI processes, technologies, and architectures to improve their chances of delivering successful BI initiatives.

Business intelligence (BI) software has emerged as a hot topic in the past few years; in 2011, most companies will again focus their software investment plans on BI. More than 49% of the companies that responded to our most recent Forrsights Software Survey have concrete plans to implement or expand their use of BI software within the next 24 months. But being interested in BI software and spending money to adopt BI tools and processes do not necessarily translate into successful implementations: Forrester’s most recent BI maturity survey indicated that enterprise BI maturity levels are still below average (2.75 on a scale of 5, a modest 6% increase over 2009). Why are BI maturity levels so low, given the amount of money firms spend on it? Three factors contribute to this rift and can lead to less-than-successful BI initiatives:

  1. Implementing BI requires using best practices and building upon lessons learned.
  2. BI technologies and processes have not kept pace with business realities.
  3. Centralization has not led to agile, streamlined BI implementations.

Why don't traditional BI approaches often work? It’s possible to model, build, and implement business processes that will not require new features or other major enhancements for six to 12 months — sometimes even longer. This is especially true for many stable and well-defined back-office applications, such as budgeting, planning, and accounts payable/accounts receivable. But what we at Forrester call untamed processes (I've even heard some vendors refer to these as dark or hidden processes) change much more frequently, because some steps in these processes are either partially or completely manual and thus depend on less predictable external factors like human intervention. This is especially true when these processes involve external constituents, such as customers, prospects, partners, and regulators, over whom you have little or no control. Untamed business processes create challenges for standard BI approaches because:

  • Untamed processes are often invisible.
  • Traditional SDLC doesn’t work well for automating untamed processes.
  • Untamed processes often require new approaches and technologies.

Something radical needs to happen to make BI implementations more successful so that they can effectively support untamed business processes. And while Forrester will never say that agility will cure all of BI’s current ills, it certainly provides the most important best practices and leverages a key capability of the underlying BI technology to help close the gap that earlier-generation BI processes and technologies create. Don’t misinterpret what we’re recommending here — it’s not just about the Agile software development methodology, which is nothing new. But Agile development by itself is not enough for BI, so Forrester also recommends adopting multiple best practices and next-generation technologies to make BI more flexible.

In addition to Agile software development methodology, we recommend starting by adapting your organizational structures and enterprise culture for agility. No technology or processes can address BI challenges if a company’s organizational structure and enterprise culture are not already on firm, agile ground (in the detailed research document we describe five of these best practices). Once the organization is aligned for agility, the next step is to consider and implement Agile BI processes (in the detailed research document we describe three of these best practices)

Last, but not least, while some of these best practices can stand on their own, others require the application of next-generation BI technologies. In the past, BI vendors and BI application developers focused on business and operational functionality and architectural robustness. In most cases, these features have become commoditized. BI vendors and developers now need to concentrate on next-generation technologies; Forrester categorizes these technologies as agile, and defines four major subcategories of agility:

  • Automated (six emerging technologies).
  • Pervasive (six emerging technologies).
  • Unified (six emerging technologies).
  • Limitless (three emerging technologies).

Please read the research report for more details and comment.