Blogging From the IBM Big Data Symposium - Big Is More Than Just Big

Just attended a Big Data symposium courtesy of IBM and thought I’d share a few insights, as probably many of you have heard the term but are not sure what it means to you.

No. 1: Big Data is about looking out of the front window when you drive, not the rearview mirror. What do I mean? The typical decision-making process goes something like this: capture some data, integrate it together, analyze the clean and integrated data, make some decisions, execute. By the time you decide and execute, the data may be too old and have cost you too much. It’s a bit like driving by looking out of your rearview mirror.

Big Data changes this paradigm by allowing you to iteratively sift through data at extreme scale in the wild and draw insights closer to real time. This is a very good thing, and companies that do it well will beat those that don’t.

No. 2: Big is not just big volume. The term “Big Data” is a misnomer and it is causing some confusion. Several of us here at Forrester have been saying for a while that it is about the four “V’s" of data at extreme scale - volume, velocity, variety and variability. I was relieved when IBM came up with three of them; variability being the one they left out.

Some of the most interesting examples we discussed centered on the last 3 V’s – we heard from a researcher who is collecting data on vital signs from prenatal babies and correlating changes in heart rates with early signs of infection. According to her, they collect 90 million data points per patient per day! What do you do with that stream of information? How do you use it to save lives? It is a Big Data Problem.

Interesting also is how Watson works, which we got a look at. Watson deals with the variability characteristic of Big Data – specifically the variability of meaning in natural language and how to use Big Data technology to solve them in less than 3 seconds. Factoid 1 – it took IBM 80 Teraflops of processing power to meet the 3-second SLA. Factoid 2 – Watson only required 1TB of data to do his magic. Of course, the TB was processed out of many PBs of source content using . . . Big Data technology as well.

So what’s our biggest challenge with Big Data? Not surprisingly, I don’t think it’s the technology. I think that the biggest problem is going to be getting our business partners to adapt their thinking to leverage the power available to them via this new technology.

I’ll be discussing this more at the Forrester IT Forum and would love to hear from you there, or on the Forrester EA Community

See you soon.


Is big data the Killer App for Parallel Computing?

Pointing out the fact that Big data is not only volume is a fantastic piece of insight.

Your note that "how can" is more important that "how to" is also spot on. Many major technologies such as massively parallel computing, the initial dotCom of web interactions, Expert Systems and AI, Internet Appliances, PDAs, etc. have all been touted at the "next big" that failed when there was no critical mass of understanding.

Most of those technologies have achieved broad acceptance in a second life, often as part of something more intutive, or perhaps just more entertaining. Do you see big data making less than a splash and eventually coming back in another reincarnation, or will it hit on the first try? And if it does, what's the Killer App that makes the lights go on?

Interesting thought

Will big data in it's current form fail under it's own weight, only to rise again as a part of something else that succeeds? I think not because it is the convergence of a mature technology (distributed computing), elastic infrastructure and the explosion of digitized data. None of these are going away. Will the label "big data" die? Maybe, but the idea will persist and the technology will evolve.

Pleased to see 3-Vs being adopted broadly

Quite pleased to see the three Vs being adopted so broadly since I first described them over ten years ago in a 2001 Gartner (then Meta Group) research article. (Happy to share it with any readers.) Interesting tho how so many others have laid claim to the concept and/or tweaked with with other redundant "V"s or ones that don't related to increasing data management challenges. -Doug Laney, Gartner