To BW Or Not To BW - 2011 Update

I get lots of questions from clients on whether they should consider (or continue to rely on) SAP BW for their data warehousing (DW) and business intelligence (BI) platform, tools, and applications. It’s a multidimensional (forgive the pun) decision. Jim Kobielus and I authored our original point of view on the subject soon after the SAP/BusinessObjects merger, so this is an updated view. In addition to what I’ll describe here, please also refer to all of the DW research by my colleague, Jim Kobielus.

First of all, split the evaluation and the decision into two parts: front end (BI) and back end (DW).

  • Back end – DW
    • Strengths:
      • Best for SAP-centric environments.
      • Agile tool that lets you control multiple layers (typically handled by different tools) such as ETL, DDL, metadata, SQL/MDX from a single administrative interface.
      • Unique BW accelerator appliance (via in-memory indexes).
    • Concerns:
      • Typically tough to incorporate non-SAP data. This is not an ETL issue. SAP will tell you that if you use Business Objects Data Integrator, this will become easier; it will not. It’s not a question of a good, open ETL tool, it’s a question of mapping apples to oranges and fitting non-SAP data into multiple BW layers of ODS, InfoCubes, BEx Queries, etc.
      • The price for agility is typically significant difficulty to tune and optimize (since many components are not under direct DBA control).
      • Based on these strengths and weaknesses:
        • Enterprises where most of the data comes from SAP sources and with relatively small data volumes do use BW as enterprise DW (EDW).
        • Enterprises where a significant percentage of data comes from non-SAP sources or with data volumes in multiple terabytes (where lots of tuning and optimization is required) use BW as an SAP-specific data mart that then feeds another EDW platform/environment.
  • Front end - BI
    • BW’s native BI tools (BEx, Visual Composer, etc.) are mostly in support-only mode in favor of Business Objects (BO) BI suite.
    • SAP’s putting on significant pressure and offering incentives for clients to migrate to BO.
    • We encourage our clients to do the same:
      • If the majority of the data comes from SAP sources.
      • If it’s a mix, we encourage them to broaden their BI vendor evaluations and consider IBM Cognos (if they already use IBM stack products like InfoSphere, WebSphere, etc.) or Microsoft (especially if they already use SharePoint) and others (while still considering BO, which is an open platform).
      • Additionally, when considering BO versus other BI vendors, a major decision point is that while each BO tool (Crystal, WebIntelligence, Explorer, Xcelcius, etc.) has market leading functionality, there’s little product-to-product integration. Basically, these are all separate products running on the same administrative platform and sharing the same data access layer (Universe), but you design, build, and support these products separately. Other BI vendors offer much more integrated platforms. The next release of BO (currently in “ramp up” stage — meaning you have to specifically request it), which should hit the shelves later this winter, will address some, but not all, product-to-product integration issues.

 Did I miss any other major decision points? Please comment and answer a short poll here or on the right side of this page.


BW in Agile BI scenarios

SAP BW is generally being presented with a negative rather than positive point of view with regards to agile BI. But, I fully agree with your POV, agility is definitively a key strength: it provides a very solid and mature metadata driven information layer that enables enterprise level agile scenario. One other underestimated strength that you are eluding, also important for agile BI scenarios, is the openness to third party BI platform/tools, including MS Sharepoint/MS BI, but also in memory data discovery platforms like Qliktech to empower lines of business with prototyping or self service scenarios. Regarding the capabilities with data volume, my feeling is that, although BW would struggle with multi-terabyte warehouse, it can handle reasonable data volume (by that, I mean, not high end, but upper middle end rather that low end) : many BI surveys have shown that current BW customers handle, in average, bigger data volumes than the users of most other BI platforms. And BW provides some scenarios like near line storage or in memory that not all BI platform can propose today. Lastly, with some customers, we've been looking at the new release of BO (4.0) and I feel it worth a deep dive to address some of the challenges you are referring to : first, integration of non SAP sources is better now that a data federation layer is fully integrated in the solution, so that you can mix and match SAP sources with others at different levels. Integration between the front-end tools is now much better too, at many levels (meta-data, "info objects" such as query or OLAP views, presentation layer). Although far from being perfect, I feel this is a game changer for BO customers, because they won't have any more to opt for an option (eg : OLAP versus ad hoc query) against another on a project by project basis, but rather mix those in a same project.

In most of the cases where I

In most of the cases where I saw BW doing miserable job it was a failure of developers and admins to design and tune the system, rather that the product itself. And this is where SAP and especially SAP Education failed in my opinion:
a) there are how-to use the product trainings, but not methodology and design workshops available.
b) most of the admins are trained in tuning the SAP NetWeaver (aka Basis) system for ECC (R/3), and then replicating wrong tuning in BW installations.

I blogged a bit more on that on SCN:

SAP BW is good enough, especially for the customers using SAP Business Suite (ERP, CRM etc), for which SAP provides out-of-the-box Business Content, i.e. predefined strucutres, transformations, queries and security roles.

History of BW is no doubt fascinating lecture and I like to talk about it as well. But why it helps to understand what BW is today, history says nothing about what BW is going to be tomorrow.

As an open platform BW supports:
1/ Metadata exchange based on OMG's XMI standard (although indeed BW has its own data model; and there is a belief that one who cracks this data model can read the mind of German engineer ;-),
2/ Universal DataConnection that allows to pull data from any source that provides JDBC driver,
3/ ETL tools (like BO DI, Informatica, IBM DS etc) as possible source system,
4/ MDX-based open analysis interface for connection of other BI tools (BObj, Panorama, Microstrategy, Cognos, MS Excel etc) that support ODBO, XML/A or SAP's OLAP BAPI.

As well BW exposes APIs for external scheduling tools, like Control-M, or APIs for Data Mining Workbenches, like the one from Oracle (before going for war ;-), supports NLS, like those from SAND or PBS; and allows SI partners to build and sell reference data models (aka Business Content) through BW platform.

BW is not only DW platform, but is in the heart of other SAP products, like Supply Chain Advanced Planning & Optimization (APO), books consolidation engine in Strategic Enterprise Management (SEM), or business planning in BPC. As well prior to BObj acquisition BW was a bolt-on on SAP CRM to provide Interactive Analysis through its virtual infoproviders.

I need to add one more comment. Whenever you advise your customer to use BW as a pass-through to their other DW, you should tell the customer as well that they have to pay additional license fee to SAP for moving data outside of SAP landscape.

Overall, I think 2011 will be really interesting for SAP BI/DW professionals:

Why is DI (DS) not big progress?

Hi Boris,
sorry but I don't get your judgement on the integration of Data Services (DS) into BW 7.3. Furthermore, I believe you still struggle to go beyond pure "table thinking". BW infoproviders can conceptually be considered as flat structures albeit the various physical flavours (like DSOs or infocubes). In that sense, it's easy to map a flat table (e.g. from a CSV file or another DB table) to such a structure. BW actually receives those flat structures as "data sources" and then subsequently enriches semantically them which make them become infoproviders and infoobjects.
Now, having DS as an additional way to feed into BW data sources provides (a) additional and ETL-standard connectivity and (b) an additional alternative to transforming the data before it is fed into BW. From my BW project experience I can tell you that I would have leveraged both benefits to a large extent in past projects.
Unfortunately, your comment simply indicates that you are continuing to nourish the old and blunt stereotype of "BW is for SAP data only". Sorry, but get an update! You will be surprised. Forrester customers will also be happy if they get updated and not outdated infos, esp. when labeling this blog as a "2011 update".

Alistair, thank you for the

Alistair, thank you for the insightful comments. I'll definitely continue to pursue more evaluations on this matter. Just an FYI, my analysis is typically comprised of 80% of what I hear from our clients and 20% of my own evaluations. So, in that sense I do think my recent comments reflect current market reality.

Boris, to my knowledge, the

Boris, to my knowledge, the tight DS integration has become available only with BW 7.3 and DS 4.0, both out only since Dec 2010. So, in this particular instance, I guess that the "80% portion" of your sources must be close to 0 as customers are unlikely to have tested it yet. Also in that sense, I'm surprised about your premature verdict.

Don't only focus on the tool

Agree with most of the above. In addition for most of the Major BI vendor now a days it's not so much the tool but how to use them. 80% of the cost of BI projects is spend on defining needs, specs and change management. Though a lot of the companies tend to spend hours on discussions on what tool is the best and has the best ROI…that’s not where the money can be earned. Focus on getting professionals mastering the skills of how get the real requirements out of the organization and capable of managing the change a companies has to go trough when moving towards the domain of Performance Management.

Good comment

Hi Mischa. Good comment. Obviously total TCO related to the product includes as well availability of skilled professionals (and their rates ;-) for implementation and support, quality and cost of support provided by the vendor, documentation is part of the product etc. Cheers.