- Forrester Councils
- Councils Overview
- log in
Posted by Mike Gilpin on May 16, 2011
“Big Data” is coming up more often on the agendas of key vendors as well as some of the more-advanced users of information management technology. Although some of this increased activity reflects PR calendars – companies promote new offerings in the Spring – there’s more than that going on. The range of design patterns that fall under this large umbrella are genuinely on the increase in a wider range of usage scenarios, driving continuing innovation from both technology providers and users. In part because of the frequent use of open source technology such as Apache Hadoop to implement “Big Data,” this is the type of innovation the industry most needs at this early stage of the market. A few key data points:
What does it all mean?
That is the subject of much research from Forrester this year, not only from Brian and Noel but also from Jim Kobielus, Gene Leganza, and others. Here’s my quick take based on what I know today:
Making complex things more accessible to developers by evolving the development model is right in the sweet spot for our team that serves application development and delivery professionals. We’ve already begun to address this issue, at least in a general way, by defining the emerging elastic application platform (EAP), great new work from John Rymer and Mike Gualtieri that shows how “NoSQL” techniques for data management will evolve as part of a broader platform for apps built on private, public, or hybrid cloud architectures.
What is the industry doing to make “Big Data” easier for developers?
Some of the existing approaches for making “Big Data” platforms accessible to more developers work by bringing familiar APIs like SQL to bear. While this may be appropriate for some applications, SQL brings baggage, too – primarily that it can “lock” the data schema for the application, depending on how developers use it. But the most-flexible applications that work with unstructured content need to be able to dynamically evolve the data schema based on the data that’s showing up through the input content or streams.
For example, social web content that marketing mines for customer insights may evolve new kinds of information about new kinds of products or services, dynamically, at any time. Marketing pros can predict neither what these topics will be ahead of time nor what they will want to know about them – the structure evolves naturally from the content. Applications that work with unstructured data can benefit from this kind of dynamic schema evolution, and developers can work using Agile processes in such an environment, but they need a development model that is similarly dynamic to support their efforts.
From the data services/“Big Data” use cases we’ve seen so far, data services appear well suited to meeting this requirement. Developers can introspect (query) services at runtime to ask what information they have about which topics and then access that information for dashboards or other flexible and interactive means of visualization, or to inform other processes with analytical insight.
What do you think? Are data services potentially relevant to your use of “Big Data”?
Lead BT Transformation
Develop customer-obsessed strategies to drive growth »
Forrester's CX Index
Predict how actions to improve CX will affect revenue performance.
Measure the customer experiences that matter most »
Free On-Demand and Live Events
Latest events from Forrester analysts, online and in person. »