How Much Intelligence Can You Pack Into a Tweet?

In the analytics wars, one of the quasi-metaphysical topics I try to avoid is debating the distinctions between “information,” “intelligence,” and “insight.”

That’s about as academic a dispute as you can imagine. It’s not as if we’re discussing the physical distinctions between, say, gases, liquids, and solids. There is no magical phase-change where information--extracted from databases and contextualized into reports and graphs-- becomes the precious gem “intelligence” and catalyzes some alchemical reaction to produce “insight.” And let’s not even go into the corollary debate regarding what makes some insights more “actionable” than others.

All of these discussions seem to hinge on some vague “usefulness” spectrum, with gobs of unfathomable data at one extreme and breathtaking revelation at the other. It’s with this general skepticism that I reacted to the recent Kognitio-sponsored survey with the headline: “Steady Growth For Business Intelligence (BI) Seen In 2010, But Twitter Data Won’t Be In The Growth Plans.” Whereas one trade-press article picked up the negative vibe in that headline, and offered its own downer variation, “Social media not useful for BI yet,” I read the study’s findings and came away with a slightly different interpretation.

Do BI practitioners, who were the ones surveyed, really hold that low an opinion on the potential BI utility of Twitter tweets, Facebook updates, and other user-generated social-networking info? Well, keep in mind that fourteen percent of respondents said they plan to incorporate social network info into their analytic applications. On the other end of the fulcrum, twenty-three percent called tweets and other social network info "overrated,” and, given the excessive hype surrounding social media, I might agree with them. But that’s not the same as claiming they see it as useless. Indeed, you might reasonably argue that BI as a whole is overrated, but it has certainly taken hold and proved its worth everywhere.

In the Kognitio survey, a majority of respondents—63 percent—reported that they are still on the fence regarding the BI utility of social network information. They said they were “undecided" about the potential value of aggregated social-networking data. Once again, they didn’t say they saw it as “not useful for BI yet,” or that it won’t be in their BI “growth plans.” Or at least that’s how I’m interpreting the survey results that this vendor shared through their press release.

Nevertheless, I was glad to see that a Kognitio exec softened this position in his discussions with the reporter who wrote that article. John Thompson, CEO of Kognitio's North American operations, said BI professionals will increasingly incorporate aggregated tweets and other user-generated social-network intelligence into marketing applications. They will leverage text mining, sentiment analysis and geolocation to tap into customer conversations in these very public social networks. In other words, he’s essentially saying that all this intelligence is very useful, and recognized as such by the most forward-looking BI professionals, but that they are still not fully equipped to mine it effectively.

And I agree with all that. Clearly, everybody’s feeling their oats right now with Twitter, Facebook, LinkedIn, MySpace, and what have you. And, quite frankly, the signal-to-noise ratio of the information you can glean through these services—in terms of marketplace intelligence that might be useful to enterprises—is often low and laughable. But if you crawl, correlate, categorize, mine, and explore it with the right tools, and store it in the new generation of content warehouses that are geared for unstructured information and complex event feeds, it can yield unexpected insights.

But the intelligence value of any individual tweet in isolation is negligible. Then again, what’s the intrinsic value of a single attribute-value pair in a relational database?

Intelligence emerges from the aggregate.