The First Rule Of Big Data — Don't Talk About Big Data

I’ll be chairing Big Data World Europe on September 19 in London; in advance of that event, here are a few thoughts.

Since late 2011, we’ve seen the big data noise level eclipse cloud and even BYOD, and we are seeing the backlash too (see Death By Big Data, to which I tweeted, “Yes, I suppose, ‘too much of anything is a bad thing’”). The number one thing clients want to know is, “What is my competition doing? Give me examples I can talk to my business about.” These questions reflect a curiosity on the part of IT and a “peeking under the hood to see what’s there” attitude.

My advice is to start the big data journey with your feet on the ground and your head around what it really is. Here are some “rules” I’ve been using with folks I talk to:

First rule of big data: don’t talk about big data. The old adage holds true here — those that can do big data do it, those that can’t talk <yup, I see the irony :-)>. I was on the phone with a VP of analytics who reflected that her IT people were constantly bringing new technologies to them like a dog with a bone. Her general reaction is, show me the bottom-line value. So what to do? Instead of talking to your business about big data, find ways to solve problems more affordably with data at greater scale. Now that’s “doing big data.”

Second rule of big data: use it to solve a problem you have first. I believe emerging technology unlocks explosions in business evolution. Business demand exists to solve a problem for a while before a technology comes along to unlock it, then a flurry of innovations establish new business norms. Big data seems no different, but firms try to skip past the first step, which is solving problems that exist. My advice is to find where you have a need that can’t be met because data scale makes it too expensive, and solve that problem with big data techniques and technologies. Do this a few times to get your feet wet.

Third rule of big data: value both techniques and technology. We define big data and techniques and technologies that make data at extreme scale more affordable. The first part of the definition, “techniques and technology” is to the point — big data is about new techniques as much as it’s about new technology. The most obvious example is MapReduce jobs running on Hadoop; you need to understand analytics and parallel computing techniques to be successful.

Fourth rule of big data: ensure you really need it. We started talking internally about this topic when I posted this blog for comment on our internal social channel. As a result, my colleague Boris Evelson came up with some important questions that get to the heart of our messaging around big data. Ensure you need new techniques and technologies — in many cases we are finding clients talking about big data, but needing something different like some good old-fashioned data governance. It’s not sexy, like big data, but might be the prescription.

So what’s the bottom line? I summed it up in another tweet, “The math of predictive analytics has been around for years, it’s the computers that have caught up.” The need for big data has been around for years. The technology has caught up with the need; now let’s enjoy the innovation ride as new business norms develop. And don’t forget it’s still just data.

Want to talk more about big data? Tell me what you think here, follow me @practicingea, or come to Big Data World Europe, 2012. You can also listen to Mike Gualtieri and I discuss this topic further here.



Big Data Rules

You could summarize the second and fourth rule just in one: Nobody buys bid data for get big data

Great advice Brian. With the

Great advice Brian. With the explosion of big data, companies are faced with data challenges in three different areas. First, you know the type of results you want from your data but it’s computationally difficult to obtain. Second, you know the questions to ask but struggle with the answers and need to do data mining to help find those answers. And third is in the area of data exploration where you need to reveal the unknowns and look through the data for patterns and hidden relationships. The open source HPCC Systems big data processing platform can help companies with these challenges by deriving insights from massive data sets quick and simple. Designed by data scientists, it is a complete integrated solution from data ingestion and data processing to data delivery. More info at

Brian Hopkins Blog

Agree that what Brian blogged makes sense. We are seeing customers desire to understand how to monetize Big Data and in the retail space particularly in the area of understanding customer sentiment using social media.

Big Data World Europe

I completely agree with what Brian has written here. As project director for Big Data World Europe which ran last week in London I was lucky enough to hear him expand on this theme during his chairman's 'tenure' at the show.
Big Data has become a real buzzword and it does take someone like Brian help us take a step back and reveal the true value of some of the much vaunted tools and strategies available in this space.
Whilst Big Data certainly isn't a silver bullet, it can have a huge impact on a business when made use of appropriately. Brian did a great job last week in helping our audience see that.