Big Data Predictions For 2013

William Shakespeare wrote that “What’s past is prologue.” Big data surely builds on our rich past of using data to understand our world, our customers, and ourselves. Now the world is flush and getting flusher in big data from cloud, mobile, and the Internet of things. What does it mean for enterprises? In a word: opportunity. Firms have taken to big data. Here are my four predictions for key enterprise big data themes in 2013:

  1. Firms will realize that “big data” means all of their data. Big data is the frontier of a firm’s ability to store, process, and access (SPA) all of the data it needs to operate effectively, make decisions, reduce risks, and create better customer experiences. The key word in the definition of big data is frontier. Many think that big data is only about data stored in Hadoop. Not true. Big data is not defined by how it is stored. It can and will continue to reside in all kinds of data architectures, including enterprise data warehouses, application databases, file systems, cloud storage, Hadoop, and others. By the way, some predict the end of the data warehouse — but that’s nonsense. If anything, all forms of data technology will evolve and be necessary to handle the frontier of big data. In 2013, all data is big data.
  2. The algorithm wars will begin. Big data is lazy when it just sits on a disk somewhere. Firms increasingly realize that it must use predictive and descriptive analytics to find nonobvious information to discover value in the data. Advanced analytics uses advanced statistical, data mining, and machine learning algorithms to dig deeper to find patterns that you can’t see using traditional BI tools, simple queries, or rules. Many of these algorithms have been around for years, but many firms will rediscover their power, combine them in new ways, and even initiate research efforts to find competitive new algorithms. Big data is the fuel. Algorithms are the engine to explore virgin data, seek out new meaning and models, and deliver more personalized, contextualized customer experiences. Look no further than Internet giants like Google that invest heavily in algorithms to power services such as Google Now. In 2013, CEOs will give their firms an imperative to to beef up their data science capabilities.
  3. Real-time architectures will swing to prominence. Firms that find predictive models in big data must put them to use. Firms will seek out streaming, event processing, and in-memory data technologies to provide real-time analytics and run predictive models. Mobile is a key driver, because hyperconnected consumers and employees will require architectures that can quickly process incoming data from all digital channels to make business decisions and deliver engaging customer experiences in real time. The result: In 2013, enterprise architects will step out of their ivory towers to once again focus on technology — real-time technology that is highly available, scalable, and performant.
  4. Naysayers will fall silent. Big data is not just a buzzword; it’s real. But it disrupts many who don’t see anything new in it or don’t see the tremendous opportunity firms have to harness it for competitive advantage. My prediction: Time magazine will name big data its 2013 person of the year.

Get recommendations for exploiting the big data phenomenon using a new architecture in this complimentary report, Deliver On Big Data Potential With A Hub-And-Spoke Architecture.

Comments

Real-time architectures will swing to prominence

Hi Mike,

Nice clear, concise list.

I bet the farm on #3, early report from the frontier is that you are right. : )

Best,
Dave

Real-time needed to serve up predictive models

Hi Dave,
Thanks for comment. Many think of real-time analytics, but it is also about deploying the predictive models. Glad to hear things are going well.
Mike

Hi Mike, Totally agree

Hi Mike,

Totally agree with your post. I would see coming together of Data Warehouse and Big Data, particularly in the area of real-time decision making where traditional DW forms the backbone for building/tweaking the ruleset and big data providing engagement part. Infact vendors have started offering real-time and in-memory processing on HDFS. Look forward to an exiciting journey

Rgds,
Srinivas

Algorithm Wars

I agree with item 2 - without algorithms to run on your compute nodes, Big Data is just distributed processing infrastructure. The scale out was the easy bit - the hard part requires the mathematically inclined to tweak / invent numerical recipes.

Great predictions, finally!

Great predictions. Clicked on the link thinking I'd read the same things over again that I've been reading everywhere else.

#1 - The challenge is figuring what data, subset if you will, of the big data is actually useful. It seems like it'll be difficult given that there might be data that they already have somewhere, but don't realize that it's there. Yes, all data is big data. Not all data is useful data; big data is a hodgepodge of mess. Data collection, cleansing, prep, etc. will become even more important.

#2 - Sounds like a real resolution right there. Turn lazy data into something productive. Time to shape up!

#3 - I'm wondering whether these real-time architectures will mean scrapping the old, legacy architectures already in place? They weren't built to handle big data. Do we need to completely scrap them?

#4 - I love this prediction. Who's to say that big data isn't human? (reference to Rick Smolan's "The Human Face of Big Data")

The subject matter brings

The subject matter brings Jeff Hawkins comments about the need to pull the data from the stream rather than an archive as its value deteriorates rapidly.

www.qlickreport.com

We have developed this tool/solution to help report building and data analysis.

Would like to get it reviewed by you.

Very Nice article and we share the views.

Regards

Ajai Kumar Agrawal
Managing Director

Future Space Race for Big Data

I fully agree...the future of Big Data Analytics will be the race to develop more insightful algorithms to derive more insight into ones big data. The top companies will leave the others in the dust.

True Predictions

I agree with you. The data is already there. Earlier statisticians and enginners might have tried useful algorithms to find out patterns. With hardware becoming cheap and technology like Hadoop is available, there are again opportunities for Algo people to come up with interesting algorithms for Predictive Models.
Request you to elaborate more on Real time Architecture.

Geovisualization of data

Mike,

I particularly like #4 and can already see the cover!

One question/comment: in your call for harnessing predictive and descriptive analytics to go deeper into the data and help identify exceptions and outliers (that presumably uncover latent patterns and reveal opportunities) - I am surprised not to see you mention geovisual contextualization of data, especially given your support of delivering more personalized, contextualized customer experiences.