Just a few years ago, when big data was associated primarily with Hadoop, it was like a precocious child…fun for adults, but nobody took it seriously. I’m attending Strata in San Jose this February, and I can see things have changed. Attendance doubled from last year and many of the attendees are the business casual managers – not the blue jeaned developers and admins of days gone by. Big data is maturing and nobody takes it lightly anymore.
I’m ramping up to attend Strata in San Jose, February 18, 19 and 20th. Here is some info to help everyone who wants to connect and share thoughts. Looking forward to great sessions and a lot of thought leadership.
I’ll be setting aside some time for 1:1 meetings (Booked Full)
[Updated on 2/17] - I have set up some blocks of time to meet with people at Strata. Please follow the link below to schedule with me on a first come basis.
[Update] - I booked out inside 2 hours...didn't expect that! I may open up my calendar for more meetings but need to get a better bead on the sessions I want to attend first. Shoot to catch me at breakfast, will tweet out when I'm there.
I’ll be posting my thoughts and locations on Twitter
The best way to connect with me at Strata is to follow me on Twitter @practicingea.
You can post @ me or DM me. I’ll be posting my location and you can drop by for ad hoc conversations as well.
I’m very interested in your point of view - data driven to insights driven
I am concluding very quickly that “big data” as we have viewed it for the last five years is not enough. I see firms using words like “real-time” or “right-time” or “fast data” to suggest the need is much bigger than big data – its about connecting data to action in a continuous learning loop.
By now you have at least seen the cute little elephant logo or you may have spent serious time with the basic components of Hadoop like HDFS, MapReduce, Hive, Pig and most recently YARN. But do you have a handle on Kafka, Rhino, Sentry, Impala, Oozie, Spark, Storm, Tez… Giraph? Do you need a Zookeeper? Apache has one of those too! For example, the latest version of Hortonworks Data Platform has over 20 Apache packages and reflects the chaos of the open source ecosystem. Cloudera, MapR, Pivotal, Microsoft and IBM all have their own products and open source additions while supporting various combinations of the Apache projects.
After hearing the confusion between Spark and Hadoop one too many times, I was inspired to write a report, The Hadoop Ecosystem Overview, Q4 2104. For those that have day jobs that don’t include constantly tracking Hadoop evolution, I dove in and worked with Hadoop vendors and trusted consultants to create a framework. We divided the complex Hadoop ecosystem into a core set of tools that all work closely with data stored in Hadoop File System and extended group of components that leverage but do not require it.
In the past, enterprise architects could afford to think big picture and that meant treating Hadoop as a single package of tools. Not any more – you need to understand the details to keep up in the age of the customer. Use our framework to help, but please read the report if you can as I include a lot more there.
Technology has given your customers choices and digital predators the edge. Ask marketing, they will tell you about the decreasing effectiveness of traditional campaigns. Look at your business strategy and find plans to address new digital competitors that have become serious threats overnight. Across the board, Forrester finds high expectations that emerging technology will help firms stay competitive amidst these changes. But which ones should you pay attention to? Cloud, mobile, social, Read more
Customers crave contextual and personal experiences on their mobile devices. Companies are looking to the reams of location and behavior data spun off mobile device to deliver them. Meanwhile, executives long for the insights lurking just below the surface of the new data they collect on customers and prospects to improve services and chart the best business strategy.
In most companies, mobile engagement, customer analytics, innovation, and business strategy happen in silos and often half-heartedly. But disrupters like Uber, TripIt, Netflix, Flipboard, and Starbucks deliver great and personalized mobile and digital experiences -- and optimize outcomes -- with insights derived from all the data they can gather.
Big data is undergoing big change, but most companies are missing it or just grasping at the edges. My colleague Fatemeh Khatibloo and I have just completed an exhaustive study of the big data phenomenon. We found a familiar pattern: business confusion in the face of stern warnings about the dangers of big data and vendor-sponsored papers extolling its benefits. Here’s what we found hidden beneath the buzz:
As data explodes, so do old ways of doing business.
Everywhere we look, we find businesses using more diverse, messier, and larger data sets to stay competitive in the age of the customer — like the consumer goods firm that allocated marketing dollars based on flu trend predictions and the oil and gas companies that used weather data to predict iceberg flows and extend their drilling season. Savvy businesses find ways to turn more data into a competitive advantage. If your firm doesn’t get this, it won’t be pretty — starting in the not too distant future.
Technology managers and architects can’t afford to sit back and think that their Hadoop project will deliver everything the business needs. Nor can you afford to think that big data isn’t for you because you don’t have that much data. Why? Because “big data” is really the practices and technologies that close the gap between the available data and the ability to turn that data into business insight — insight that your firm needs to survive and thrive in the age of the customer. Four things to understand:
When I stumbled across Bitcoin (or Bit-O-Coin, as my wife likes to call it) a few years back, my spidey sense started tingling. Since that time, I’ve made a few off hand remarks about the future of crypto-currency and received the expected “it’s another Dutch Tulip thing”. While I’m not an expert on the financial markets, I do have an excellent track record for identifying disruptive technology changes and I’ve concluded that crypto-currency is here to stay.
Mobile, social, and cloud computing have created seismic shifts in the business technology landscape over the last seven years. Instead of simply evolving our 2011 top trends research, this year we’re taking a fresh look at how technology is fueling business change anchored by three broad themes:
Engagement: Engaged customers drive the pace of business change.
Smart: Firms rely on smart systems for competitive advantage.
Nimble: IT embraces change to help them be both nimble and secure.
In our recently completed Q3 2013 Global State Of Enterprise Architecture Online Survey, big data for real-time analytics moved from the No. 3 most revolutionary technology to the No. 2 position, according to the 116 enterprise architects who participated. This reflects the importance firms now place on turning vast amounts of data into immediate insight. And this trend is extremely important to telecommunication industry communication service providers (CSPs), who are sitting on a gold mine of data about what subscribers are doing on their mobile devices.
Let’s break this down a bit more -- according to the United Nations, there are about 2 billion mobile broadband subscriptions globally (that’s about 28% of the world’s 7.1 billion people). That’s a huge number of perpetually connected people, using bunches of apps for both work and personal. This is part of what we call the mobile mind shift, and it’s not about smartphones and tablets; rather, it’s about the changing expectations that pervasive mobile computing and broadband wireless have. According to a recent report, "The Mobile Mind Shift Index," we estimate 21% of the adult online US population now expects that any information is available on any appropriate device, in context, at their moment of need (see Josh Bernoff’s May 2013 blog Introducing The Mobile Mindshift Index). And this number is going to grow significantly over the next few years.
Big data noise has reached the point where most are reaching for the ear plugs. And with any good hype bubble, the naysayers are now grabbing attention with contrarian positions. For example, The New York Times expressed doubt about the economic viability of big data in "Is Big Data an Economic Big Dud?" This post grabbed a lot of attention, but, like many others I read, it fundamentally misses the point of what big data is all about and why it's important. The article compares the productivity boom associated with the first wave of the Internet to the lack of growth experienced since the inception of "big data"; it implies that big data’s expected economic impact may not happen. Furthermore, the article implies that big data is something that firms will do or implement. Thinking about big data this way or differentiating between data sets as big, medium, or small is dangerous. It leads to chasing rabbits down holes.