I kid you not.

What’s clear is that Hadoop has already proven its initial footprint in the enterprise data warehousing (EDW) arena: as a petabyte-scalable staging cloud for unstructured content and embedded execution of advanced analytics. As noted in a recent blog post, this is in fact the dominant use case for which Hadoop has been deployed in production environments.

Yes, traditional (Hadoop-less) EDWs can in fact address this specific use case reasonably well — from an architectural standpoint. But given that the most cutting-edge cloud analytics is happening in Hadoop clusters, it’s just a matter of time — one to two years, tops — before all EDW vendors bring Hadoop into their heart of their architectures. For those EDW vendors who haven’t yet fully committed to full Hadoop integration, the growing real-world adoption of this open-source approach will force their hands.

Where the next-generation EDW is concerned, the petabyte staging cloud is merely Hadoop’s initial footprint. Enterprises are moving rapidly toward the EDW as the hub for all advanced analytics. Forrester strongly expects vendors to incorporate the core Hadoop technologies — especially MapReduce, Hadoop Distributed File System, Hive, and Pig — into their core architectures. Again, the impressive growth in MapReduce as a lingua franca for predictive modeling, data mining, and content analytics will practically compel EDW vendors to optimize their platforms for MapReduce, alongside high-performance support for SAS, SPSS, R, and other statistical modeling languages and formats. We see clear signs that this is already happening, as with EMC Greenplum’s recent announcement of a Hadoop product family and indications from some of that company’s competitors that they have similar near-term road maps.

Predictive analytics is not just about forecasting what’s coming down the pike. It’s also about keeping the bad alternative futures from happening. If you can see the nasty things that might happen far enough in advance, you have a better chance of neutralizing or squelching them entirely.

In fact, many real-world applications of predictive analytics are “interdictive,” a term often used in military and law enforcement contexts to refer to tactics that delay, disrupt, or shut down an adversary’s forces or supply routes before they can do damage. Anti-fraud is one of the principal interdictive applications of predictive analytics technology. Companies everywhere rely on data mining to determine who’s been engaging, alone or in groups, in stealing money, supplies, finished goods, cellular airtime, and other valuables — and also where they’re likely to strike next. Likewise, anti-terrorism efforts rely on predictive models to sift through massive collections of historical and real-time intelligence in a Jack Bauer-like race against time and imminent disaster. You best believe that social network analysis is a key weapon in your arsenal for predicting and interdicting these sorts of malignant social patterns.

Hadoop is riding the hype wave right now. You’ll find many IT professionals who know just enough about Hadoop to be dangerous in a cocktail party setting, but not enough for their own comfort to respond to grilling from the chief technology officer or the geekier business executives.

If you’re slightly bewildered by all the buzz over this new technology with the funny-sounding moniker, you’re not alone. The official story is that Hadoop was the name of the inventor’s kid’s stuffed elephant. However, for most IT professionals, it could easily be an acronym for "Heck, Another Darn Obscure Open-Source Project." The fact that Hadoop, managed by Apache, includes subprojects with similarly opaque names — such as Pig, Hive, Chukwa, and ZooKeeper — contributes to the queasy feeling that this is an untamed menagerie of squealing beasties.

And if you’ve pegged Hadoop as an advanced analytics initiative to mine petabytes of unstructured information, prepare for further bewilderment. The Apache Hadoop project states that it develops open-source software for “reliable, scalable, distributed computing.” Yes, that’s true, but the better-informed among you may be puzzling over the linkages that people often draw between Hadoop, in-database analytics, and MapReduce.

Guesstimates are often essential for market sizing and trending. To be useful, especially where primary data are lacking, they demand a valid conceptual framework. 

Like you, I’m looking forward to the responses to Boris Evelson’s quick Web-based survey, which you can access from his most recent blogpost.It’s always a challenge to assess how truly pervasive BI is—and pervasive it could potentially become.

To generate a valid first approximation, Boris scoped his blog comments and quick survey to “traditional BI” applications (i.e., historical reporting, query, dashboarding). He scoped his estimate only to large enterprise and midmarket firms (i.e., those with 100 or more employees) and only to BI usage in the US.

In order to keep this task manageable, Boris excluded some use cases that are often included in the “traditional BI” category: spreadsheets and other “homegrown” analytics apps; BI embedded in line-of-business apps; and non-interactive, static, published BI outputs. He leveraged both public and Forrester-gathered primary data to gauge how many actual and potential BI users there might be.

Scoping it as he did, Boris estimated that slightly more than 1.5 million people in the US are using traditional BI applications, which is between 2-3 percent of the employees of BI-implementing firms. He suspects the actual percentage might be as high as 6-8 percent of employees, but he’s not sure. That’s why he’s running the Web-based quick survey.