Data Quality A Thankless Task — Until A Crisis Arises

By Rob Karel.

Quality data in the enterprise is like breathable air — you don't truly appreciate it until it's gone. Many companies don't even bother to ask whether the customer, product, asset, or any other data it captures is actually complete, valid, and useful. Other companies leave the responsibility of standardizing, cleansing, and aggregating data from source systems to their IT developers, perhaps leveraging transformation capabilities within extract, transform, and load (ETL) tools to automate this hygiene process.

Then there are those companies that have felt enough data quality-induced pain such as wasted marketing costs or low call center productivity, and have invested in data quality software that allows for the advanced definition and maintenance of rules to standardize, cleanse, enrich, match, and merge. Once an investment in data quality software is made, companies hopefully have invested also in staffing at least a handful of data quality stewards or business analysts. These data quality professionals (DQPs) can translate requirements and perspectives of quality from the business stakeholders to technical requirements that can be implemented within the DQ software.

Unfortunately, these DQPs are usually treated like the red-headed stepchild — they are expected to do their jobs with minimal executive support, funding, strategic priority, or recognition. It is assumed that because an investment in data quality software was made, there should no longer be a data quality issue. This is of course absurd: The technology is only as good as the data governance and business ownership of the data rules and definitions. If the business does not actively participate and take responsibility for the quality of data, the technology will be nothing more than yet another IT expense.

These underappreciated DQPs often spend a large percentage of their time trying to convince management to let them do what they were hired to do — improve the quality of data — but can't move the quality needle far enough through their own grassroots efforts. That is why it's ironic and sad that often a DQPs best friend may just be a data quality crisis. When senior executives learn that they have failed a compliance audit, a customer is suing for violation of their privacy preferences, or they have made a very bad strategic business decision based on faulty data, the DQP is suddenly very popular and subsequently receives plenty of priority, funding, and support.

Thankfully there are many organizations today that do put significant investment and priority in data quality, especially those in highly regulated industries like financial services and healthcare. But for the rest of the pack, data quality is still just a nice to have. And to the data quality professionals working for these laggard organizations: Keep up the good work, your time will come!

Sesame Street 2.0

Web 2.0 is hitting the preschool set, as children's TV favorite Sesame Street is now producing a weekly video podcast. Each video podcast is five minutes long and features content repurposed from the show’s broadcasts. The content is available as a download on the show’s Web site and via an RSS feed, as well as through iTunes (apparently to help keep the wee ones occupied and educated while they're being dragged around town on errands).

So even the Muppets are finding new ways of engaging their customers by distributing Web 2.0-type content through multiple channels. Hope they’re using a digital asset management system and a good solid taxonomy while they’re at it.

(And I wonder if I can get a copy of the clip with the guy dropping all those cakes.)

BI Market Consolidation Predictions

by Boris Evelson.

Yet another rumor crossed the wires today about Business Objects getting ready for a takeover. These rumors have come up before, and it may be nothing else but a routine exercise that BO and other vendors go through to test the market periodically.

As I already blogged back in March '07, there's no denying however that the business intelligence (BI) and business performance solutions (BPS) markets are consolidating. Last year Microsoft bought ProClarity; this year Oracle bought Hyperion, Business Objects acquired Carthesis, SAP acquired Outlooksoft, and most recently Cognos acquired Applix.

I predict that within 2-3 years there will be five major BI vendors carving up the BI and BPS market: Microsoft, IBM, SAP, Oracle and HP. There'll be much M&A activity seen from these vendors in the near future.

Here's how I see it playing out:

• HP will probably make the next big move by acquiring somebody like Business Objects, Cognos or SAS. HP is pushing into BI market very strongly: It repositioned NeoView as a strong data warehouse (DW) platform and bought Knightsbridge — leading BI boutique strategy and implementation consultancy. It'd make a lot of sense for HP to pick up BO or SAS, since in one transaction HP would get the entire BI and PM suite. Cognos would be the second choice for them, since Cognos does not have operational ETL and data quality offerings.
• SAP told us repeatedly that they can't justify very large acquisitions — it doesn't fit their model. So Microstrategy, Actuate or Information Builders would be more obvious pick up choices for them.
• I sincerely hope that Oracle is more than busy with Hyperion integration and more than set in the BI and BPS markets for a while. I don't see the next big move coming from them.
• While IBM seems a likely suitor for a large transaction, IBM told us on numerous occasions that they view BI and BPS as "applications" and IBM does not want to be in the application game. I do not see IBM doing a large BI/BPS acquisition — they are more likely to acquire small BI/BPS "tools" vendors (similar to their Alphablox acquisition) and keep slowly enriching their portfolio of business optimization products.
• I place Microsoft in similar category as IBM — it's very unlikely in my opinion that they will buy a large BI vendor. Microsoft will most likely continue to execute relatively smaller acquisitions such as ProClarity, Active Views, Great Plains and others. Microsoft has a unique SMB market position, a tremendous partner/reseller network, and increasingly popular SharePoint Server (a necessary BI portal and collaboration component). They do not need to surprise the market, they can afford to move at their own pace.

As I mentioned back in my March '07 blog, an orthogonal move could come from EMC or Sun, who have been information management players for years, with BI being a natural addition/extension. Notably absent from all the rumors is Teradata, which in our opinion has to diversify into more layers of the BI "stack" beyond data warehousing to keep its competitive position.

BI Dominoes Continue To Fall...

by Boris Evelson.

Today Cognos announced its intention to acquire Applix, Inc for a cash tender offer of $17.87 per share, or approximately $339 million.

This acquisition is primarily about performance management, and I quote my colleague, Paul Hamerman: “This continues a trend of rapid consolidation in the business performance solutions space, following acquisitions of pure plays by larger BI and ERP vendors. Applix has been a stellar performer as a flexible platform for customers to rapidly implement applications for planning, budgeting, forecasting and business performance analytics. Although Cognos has been a leader in performance management solutions, the Applix technology gives it an opportunity to refresh its offerings and more aggressively sell to the midmarket.”

The BI side of this acquisition is all about TM1 OLAP technology. Cognos’s Powerplay OLAP product has been trailing Microsoft SQLServer Analysis Services and Hyperion (recently acquired by Oracle) Essbase in market adoption. With TM1, a memory-based OLAP cube, Cognos achieves several objectives:

• Leapfrogs Microsoft and Oracle with one of the fastest read/write OLAP technologies on the market.
• Provides an enterprise grade in-memory OLAP alternative to fast-growing and increasingly popular QlikTech, which is more of a departmental solution.
• Provides additional midmarket entry points for Cognos

The future of the rest of the Applix BI products — reporting, querying, dashboards, and data integration — is quite murky. Cognos will face the same challenge as Oracle now does with the Hyperion acquisition: Establishing a strategy and a road map of overlapping product portfolio integration or sunsetting/retirement.