Big Data: The Big Divide Between IT and Business

 

I met with a group of clients recently on the evolution of data management and big data.  One retailer asked, “Are you seeing the business going to external sources to do Big Data?”

My first reaction was, “NO!” Yet, as I thought about it more and went back to my own roots as an analyst, the answer is most likely, “YES!”

Ignoring nomenclature, the reality is that the business is not only going to external sources for big data, but they have been doing it for years.  Think about it; organizations that have considered data a strategic tool have invested heavily in big data going back to when mainframes came into vogue.  More recently, banking, retail, consumer packaged goods, and logistics have marquis case studies on what sophisticated data use can do. 

Before Hadoop, before massive parallel processing, where did the business turn?  Many have had relationships with market research organizations, consultancies, and agencies to get them the sophisticated analysis that they need. 

Think about the fact, too, that at the beginning of social media, it was PR agencies that developed the first big data analysis and visualization of Twitter, LinkedIn, and Facebook influence.  In a past life, I worked at ComScore Networks, an aggregator and market research firm analyzing and trending online behavior.  When I joined, they had the largest and fastest growing private cloud to collect web traffic globally. Now, that was big data.

Today, the data paints a split picture.  When surveying IT across various surveys, social media and online analysis is a small percentage of business intelligence and analytics that is supported.  However, when we look to the marketing and strategy clients at Forrester, there is a completely opposite picture. 

Read more

Without Data Management Standards – Anarchy!

 

When I posted a blog on Don’t Establish Data Management Standards (it was also on Information Management's website as Data Management Standards are a Barrier) I expected some resistance.  I mean, why post a blog and not have the courage to be provocative, right?  However, I have to say I was surprised at the level of resistance.  Although, I also have to point out that this blog was also one of the most syndicated and recommended I have had.  I will assume that there is a bit of an agreement with it as well as I didn't see any qualifiers in tweets that I was completely crazy.  Anyway, here are just a few dissenter comments:

“This article would be funny if it wasn't so sad...you can't do *anything* in IT (especially innovate) without standing on the shoulders of some standard.” – John O

“Show me data management without standards and good process to review and update them and I'll show you the mortgage crisis which developed during 2007.” – Jim F 

“This article is alarmingly naive, detrimental, and counterproductive. Let me count the ways…” – Cynthia H

"No control leads to caos... I would be amused to watch the reaction of the ISO engineer while reading this article :)." - Eduardo G  (I would too!)

After wiping the rotten tomatoes from my face from that, here are some points made that get to the nuance I was hoping to create a discussion on:

Read more

Avoiding Big Data Sand Traps

Data management history has shown, it is not what you buy; it is how you are able to use it that makes a difference. According to survey results from the Q4 2012 Forrsights BI/Big Data Survey, this is a story that is again ringing true as big data changes the data management landscape. 

Overall . . .

  • Big technology adoption across various capabilities ranges from 8% to just over 25%.
  • Plans to implement big data technology across various capabilities is as high as 31%.
  • Pilot projects are the preferred method to get started.

However . . .

  • High-performing organizations (15%-plus annual growth) are expanding big data investments by one to two times in many big data areas compared with other organizations.

The key takeaway . . . 

  • For most organizations, big data projects aren't leaving the pilot stage and aren't failing to attain strong return on investment (ROI).
Read more

Don't Establish Data Management Standards

A recent survey of Enterprise Architects showed a lack of standards for data management.* Best practices has always been about the creation of standards for IT, which would lead us to think that lack of standards for data management is a gap.

Not so fast.

Standards can help control cost. Standards can help reduce complexity. But, in an age when a data management architecture needs to flex and meet the business need for agility, standards are a barrier. The emphasis on standards is what keeps IT in a mode of constant foundation building, playing the role of deli counter, and focused on cost management.

In contrast, when companies throw off the straight jacket of data management standards the are no longer challenged by the foundation. These organizations are challenged by ceilings. Top performing organizations, those that have had annual growth above 15%, are working to keep the dam open and letting more data in and managing more variety. They are pushing the envelope on the technology that is available.

Think about this. Overall, organizations have made similar data management technology purchases. What has separated top performers from the rest of organizations is by not being constrained. Top performers maximize and master the technology they invest in. They are now better positioned to do more, expand their architecture, and ultimately grow data value. For big data, they have or are getting ready to step out of the sandbox. Other organizations have not seen enough value to invest more. They are in the sand trap.

Standards can help structure decisions and strategy, but they should never be barriers to innovation.

 

*203 Enterprise Architecture Professionals, State of Enterprise Architecture Global Survey Month,2012

**Top performer organization analysis based on data from Forrsights Strategy Spotlight BI And Big Data, Q4 2012

Don't Establish Data Management Standards

A recent survey of Enterprise Architects showed a lack of standards for data management.* Best practices has always been about the creation of standards for IT, which would lead us to think that lack of standards for data management is a gap.

Not so fast.

Standards can help control cost. Standards can help reduce complexity. But, in an age when a data management architecture needs to flex and meet the business need for agility, standards are a barrier. The emphasis on standards is what keeps IT in a mode of constant foundation building, playing the role of deli counter, and focused on cost management.

In contrast, when companies throw off the straight jacket of data management standards the are no longer challenged by the foundation. These organizations are challenged by ceilings. Top performing organizations, those that have had annual growth above 15%, are working to keep the dam open and letting more data in and managing more variety. They are pushing the envelope on the technology that is available.

Think about this. Overall, organizations have made similar data management technology purchases. What has separated top performers from the rest of organizations is by not being constrained. Top performers maximize and master the technology they invest in. They are now better positioned to do more, expand their architecture, and ultimately grow data value. For big data, they have or are getting ready to step out of the sandbox. Other organizations have not seen enough value to invest more. They are in the sand trap.

Standards can help structure decisions and strategy, but they should never be barriers to innovation.

 

*203 Enterprise Architecture Professionals, State of Enterprise Architecture Global Survey Month,2012

**Top performer organization analysis based on data from Forrsights Strategy Spotlight BI And Big Data, Q4 2012

What Master Data Management Metrics Matter?

I recently had a client ask about MDM measurement for their customer master.  In many cases, the discussions I have about measurement is how to show that MDM has "solved world hunger" for the organization.  In fact, a lot of the research and content out there focused on just that.  Great to create a business case for investment.  Not so good in helping with the daily management of master data and data governance.  This client question is more practical, touching upon:

  • what about the data do you measure?
  • how do you calculate?
  • how frequently do you report and show trends?
  • how do you link the calculation to something the business understands?
Read more

The Great Divide: MDM and Data Quality Solution Selection

I just came back from a Product Information Management (PIM) event this week had had a lot of discussions about how to evaluate vendors and their solutions.  I also get a lot of inquiries on vendor selection and while a lot of the questions center around the functionality itself, how to evaluate is also a key point of discussion.  What peaked my interest on this subject is that IT and the Business have very different objectives in selecting a solution for MDM, PIM, and data quality.  In fact, it can often get contentious when IT and the Business don't agree on the best solution. 

General steps to purchase a solution seem pretty consistent: create a short list based on the Forrester Wave and research, conduct an RFI, narrow down to 2-3 vendors for an RFP, make a decision.  But, the devil seems to be in the details.  

  • Is a proof of concept required?
  • How do you make a decision when vendors solutions appear the same? Are they really the same?
  • How do you put pricing into context? Is lowest really better?
  • What is required to know before engaging with vendors to identify fit and differentiation? 
  • When does meeting business objectives win out over fit in IT skills and platform consistency?
Read more

Judgement Day for Data Quality

Joining in on the spirit of all the 2013 predictions, it seems that we shouldn't leave data quality out of the mix.  Data quality may not be as sexy as big data has been this past year.  The technology is mature and reliable.  The concept easy to understand.  It is also one of the few areas in data management that has a recognized and adopted framework to measure success.  (Read Malcolm Chisholm's blog on data quality dimensions) However, maturity shouldn't create complancency. Data quality still matters, a lot.

Yet, judgement day is here and data quality is at a cross roads.  It's maturity in both technology and practice is steeped in an old way of thinking about and managing data.  Data quality technology is firmly seated in the world of data warehousing and ETL.  While still a significant portion of an enterprise data managment landscape, the adoption and use in business critical applications and processes of in-memory, Hadoop, data virtualization, streams, etc means that more and more data is bypassing the traditional platform.

The options to manage data quality are expanding, but not necessarily in a way that ensures that data can be trusted or complies with data policies.  Where data quality tools have provided value is in the ability to have a workbench to centrally monitor, create and manage data quality processes and rules.  They created sanity where ETL spaghetti created chaos and uncertainty.  Today, this value proposition has diminished as data virtualization, Hadoop processes, and data appliances create and persist new data quality silos.  To this, these data quality silos often do not have the monitoring and measurement to govern data.  In the end, do we have data quality?  Or, are we back where we started from?

Read more

Ethical Use: Do You Have A Data Policy for That?

Security and privacy have always been at the core of data governance.  Typically, company policies, processes, and procedures have been designed to comply with these regulations to avoid fines and in some cases jail time.  Very internally focused.  However, companies now operate in a more external and connected fashion then ever before.

Let's consider this.  Two stories in the news have recently exposed an aspect of data governance that muddies the water on our definition of data ownership and responsibility.  After the tragedy at Sandy Hook Elementary School, the Journal News combined gun owner data with a map and released it to the public causing speculation and outcry that it provided criminals information to get the guns and put owners at risk.  A more recent posting of a similar nature, an MIT graduate student creates an interactive map that lets you find individuals across the US and Canada to help people feel a part of something bigger.  My first reaction was to think this was a better stalker tool than social media.

Why is this game changing for data governance and why should you care?  It begs us to ask, even if a regulation is not hanging over our head, what is the ethical use of data and what is the responsibility of businesses to use this data?

Technology is moving faster than policy and laws can be created to keep up with this change.  The owners of data more often than not will sit outside your corporate walls.  Data governance has to take into account not only the interests of the company, but also the interests of the data owners.  Data stewards have to be the trusted custodians of the data.  Companies have to consider policies that not only benefit the corporate welfare but also the interests of customer and partners or face reputational risk and potential loss of business.

Read more

Categories:

The Kiss of Death for Data Strategy

The number one question I get from clients regarding their data strategy and data governance is, “How do I create a business case?” 

This question is the kiss of death and here is why.

You created an IT strategy that has placed emphasis on helping to optimize IT data management efforts, lower total cost of ownership and reduce cost, and focused on technical requirements to develop the platform.  There may be a nod toward helping the business by highlighting the improvement in data quality, consistency, and management of access and security in broad vague terms.  The data strategy ended up looking more like an IT plan to execute data management. 

This leaves the business asking, “So what? What is in it for me?”

Rethink your approach and think like the business:

·      Change your data strategy to a business strategy.  Recognize the strategy, objectives, and capabilities the business is looking for related to key initiatives.  Your strategy should create a vision for how data will make these business needs a reality.

·      Stop searching for the business case.  The business case should already exist based on project requests at a line of business and executive level. Use the input to identify a strategy and solution that supports these requests.

·      Avoid “shiny object syndrome”.  As you keep up with emerging technology and trends, keep these new solutions and tools in context.  There are more data integration, database, data governance, and storage options than ever before and one size does not fit all.  Leverage your research to identify the right technology for business capabilities.

Read more