Big Data: The Worst Category Name Ever

Grant me a "crabby old guy" rant on big data. I continually hear people in our industry using the term big data as a product-category name -- and confusing everyone about the business value of big data solutions. Moreover, too many people now seem to think that Hadoop is big data, when Hadoop is just one of the several big-data solutions available -- and Hadoop isn't good for many big data scenarios.

Big data is a label for the trend toward processing dynamic (and therefore voluminous) data using in-memory architectures. This trend is being played out in 8 major scenarios that I can find. In each case, enterprises are struggling to understand how the various big data solutions will help generate revenue and profits, manage expenses, and service customers and citizens.

I posted a slide deck from a recent discussion of the big data trend on Slideshare. Here is the link: http://www.slideshare.net/johnrrymer/the-worst-category-name-ever. In that slide deck, you'll see this diagram listing the 8 big data scenarios we've seen in our work with clients here at Forrester Research.

Each of these scenarios poses specific architectural design, application development, and operational challenges. And each typically demands a particular technology solution. That's right; there's not one big-data product category but many categories. Big data must include complex event processing platforms, elastic caching platforms, and the various not-only SQL (NoSQL) databases. We have yet to see a one-size-fits-all suite or solution for all of these scenarios.

My hope is that we start zeroing in on the scenarios and the value to enterprises of the various big data solutions that address each scenario. By doing so, we'll unlock the value of big data a lot faster. And this old guy will have one less thing to crab about.

Comments

Can Big Data become an elephant riding a bicycle?

John, excellent points that I used to support my own write-up that points out the need for an holisitic approach to Big Data.

http://successfulworkplace.com/2012/10/28/big-data-must-not-be-an-elepha...

Without seeing Big Data for what it really is, organizations risk trying to put in the one-size-fits-all solution as you say, but also risk creating a solution heavily dependent on analytics and large data sets but missing the key pieces that bring value. It starts with defining the uses cases and ends with starting over to create better and better outcomes.

Excellent write-up, thanks!

What in the name of Big Data?

John,

Thanks for your rant which is right on point. Unfortunately, both sellers and buyers prefer their technology solutions to be presented in simple, easily digested packages, no matter how complex they really are and despite inefficiency and waste that results when vastly different products are classified with one name.

Creating the theme that "Big Data is the future of IT" is just an echo of "the Cloud is the future of IT." It's much more enticing to have to get on the "the Cloud" or into "Big Data" than consider "offsite hosting" or "content ops."

Big Bad Data

There's tremendous value in the amount of data trolled these days, but personally, if we hadn't seemingly lost the ability to generate quality data in the first instance, we might not be in such need of "big data solutions" today.

Part of the problem is

Part of the problem is analytics firms classify technical solutions in terms of the business problem they solve, and potentially group them accordingly, when what technical solutions actually do, when you look hard at them, is solve technical problems. E.g. you could (as a thumbsuck - I'm sure I've got some of this wrong) do this:

Fast data read - risk, mobile scale content delivery, predictive analysis, new queries, behavioural insights

Fast data write - mobile scale content creation

Transactional data read - slow/important processes

Transactional data write - OLTP, critical mobile scale content creation

As Joel Spolsky once said, all abstractions are leaky abstractions, and if you just try and abstract away the technical choices as packaged-up business solutions then you're bound to get frustrated! But I guess the point of Forrester/Gartner is to advertise products to C-level guys, and at the end of the day a lot of decisions get based on highly leaky abstractions :)

John Interesting write up but

John

Interesting write up but not sure I am there with you that "Big data must include complex event processing platforms, elastic caching platforms" Although these might be a real advantage I am sure PASSUR Aerospace (using big data to predict gate arrival times) would be still Big Data although I don't think it includes an elastic caching platform. I like your slide 5 (in your linked preso) since there are many sides to Big Data, and your 8 scenarios make sense but they are all part of Big Data.

If Hadoop et. al are not Big Data what are they?

-Mark