Without Data Management Standards – Anarchy!

Michele Goetz

 

When I posted a blog on Don’t Establish Data Management Standards (it was also on Information Management's website as Data Management Standards are a Barrier) I expected some resistance.  I mean, why post a blog and not have the courage to be provocative, right?  However, I have to say I was surprised at the level of resistance.  Although, I also have to point out that this blog was also one of the most syndicated and recommended I have had.  I will assume that there is a bit of an agreement with it as well as I didn't see any qualifiers in tweets that I was completely crazy.  Anyway, here are just a few dissenter comments:

“This article would be funny if it wasn't so sad...you can't do *anything* in IT (especially innovate) without standing on the shoulders of some standard.” – John O

“Show me data management without standards and good process to review and update them and I'll show you the mortgage crisis which developed during 2007.” – Jim F 

“This article is alarmingly naive, detrimental, and counterproductive. Let me count the ways…” – Cynthia H

"No control leads to caos... I would be amused to watch the reaction of the ISO engineer while reading this article :)." - Eduardo G  (I would too!)

After wiping the rotten tomatoes from my face from that, here are some points made that get to the nuance I was hoping to create a discussion on:

Read more

Don't Establish Data Management Standards

Michele Goetz

A recent survey of Enterprise Architects showed a lack of standards for data management.* Best practices has always been about the creation of standards for IT, which would lead us to think that lack of standards for data management is a gap.

Not so fast.

Standards can help control cost. Standards can help reduce complexity. But, in an age when a data management architecture needs to flex and meet the business need for agility, standards are a barrier. The emphasis on standards is what keeps IT in a mode of constant foundation building, playing the role of deli counter, and focused on cost management.

In contrast, when companies throw off the straight jacket of data management standards the are no longer challenged by the foundation. These organizations are challenged by ceilings. Top performing organizations, those that have had annual growth above 15%, are working to keep the dam open and letting more data in and managing more variety. They are pushing the envelope on the technology that is available.

Think about this. Overall, organizations have made similar data management technology purchases. What has separated top performers from the rest of organizations is by not being constrained. Top performers maximize and master the technology they invest in. They are now better positioned to do more, expand their architecture, and ultimately grow data value. For big data, they have or are getting ready to step out of the sandbox. Other organizations have not seen enough value to invest more. They are in the sand trap.

Standards can help structure decisions and strategy, but they should never be barriers to innovation.

 

*203 Enterprise Architecture Professionals, State of Enterprise Architecture Global Survey Month,2012

**Top performer organization analysis based on data from Forrsights Strategy Spotlight BI And Big Data, Q4 2012

Don't Establish Data Management Standards

Michele Goetz

A recent survey of Enterprise Architects showed a lack of standards for data management.* Best practices has always been about the creation of standards for IT, which would lead us to think that lack of standards for data management is a gap.

Not so fast.

Standards can help control cost. Standards can help reduce complexity. But, in an age when a data management architecture needs to flex and meet the business need for agility, standards are a barrier. The emphasis on standards is what keeps IT in a mode of constant foundation building, playing the role of deli counter, and focused on cost management.

In contrast, when companies throw off the straight jacket of data management standards the are no longer challenged by the foundation. These organizations are challenged by ceilings. Top performing organizations, those that have had annual growth above 15%, are working to keep the dam open and letting more data in and managing more variety. They are pushing the envelope on the technology that is available.

Think about this. Overall, organizations have made similar data management technology purchases. What has separated top performers from the rest of organizations is by not being constrained. Top performers maximize and master the technology they invest in. They are now better positioned to do more, expand their architecture, and ultimately grow data value. For big data, they have or are getting ready to step out of the sandbox. Other organizations have not seen enough value to invest more. They are in the sand trap.

Standards can help structure decisions and strategy, but they should never be barriers to innovation.

 

*203 Enterprise Architecture Professionals, State of Enterprise Architecture Global Survey Month,2012

**Top performer organization analysis based on data from Forrsights Strategy Spotlight BI And Big Data, Q4 2012

Data Quality Reboot Series For Big Data: Part 2 Persistence Vs. Disposable Quality

Michele Goetz

We last spoke about how to reboot our thinking on master data to provide a more flexible and useful structure when working with big data. In the structured data world, having a model to work from provides comfort. However, there is an element of comfort and control that has to be given up with big data, and that is our definition and the underlying premise for data quality.

Current thinking: Persistence of cleansed data.For years data quality efforts have focused on finding and correcting bad data. We used the word “cleansing” to represent the removal of what we didn’t want, exterminating it like it was an infestation of bugs or rats. Knowing what your data is, what it should look like, and how to transform it into submission defined the data quality handbook. Whole practices were stood up to track data quality issues, establish workflows and teams to clean the data, and then reports were produced to show what was done. Accomplishment was the progress and maintenance of the number of duplicates, complete records, last update, conformance to standards, etc. Our reports may also be tied to our personal goals. Now comes big data — how do we cleanse and tame that beast?

Reboot: Disposability of data quality transformation. The answer to the above question is, maybe you don’t. The nature of big data doesn’t allow itself to traditional data quality practices. The volume may be too large for processing. The volatility and velocity of data change too frequently to manage. The variety of data, both in scale and visibility, is ambiguous.

Read more