The Social Media Data Arms Race

Today, Converseon announced an agreement with Twitter to feed the firehose of tweets – around 65 million per day – into Converseon’s listening platform. This deal improves upon Converseon’s data sourcing, a critical step to the listening process. Only a small handful of other vendors dare tread in the rushing rapids of the Twitter firehose – for now, just Jive and Crimson Hexagon – but the race for social media data is on.

Alternatively, during the past year I’ve seen a shift in vendors' pitches, moving away from touting superior data sourcing and instead focusing on better data processing or insight delivery. Data became a commodity. And while the movement toward improved insights continues – because what good is a plethora of data if you can’t find what you’re looking for – the recent focus on pulling in all of Twitter makes for two key assumptions:

  • The importance of real-time data. The very nature of Twitter and the appeal of its data is the nearly instantaneous spread of content. Twitter thrives on fast-moving content, breaking news before mainstream media – or even bloggers – can write up and publish it, and giving its users the ability to share opinions (on products, services, brands, marketing, and more) right as they form their thoughts. The access to real-time data opens up a new level of Social Intelligence functions – allowing customer service to function as quickly as phone channels, marketers to move to more proactive engagement tactics, and Customer Intelligence professionals to collect more immediate data than ever before.
  • The bet on Twitter as the salient data channel. Although Twitter is one of hundreds of social channels, there’s no denying that Twitter’s taken the online world by storm. Foursquare might be the buzz of the season (or is Gowalla the new hip service?), and Facebook may serve more than 10 times the users, but Twitter owns public conversation. The popularity of social sites seems temporal, but watching multiple progressive social technology vendors invest in Twitter’s data proves that there’s value within the emerging channel.

But there’s a bigger question at hand: What will these vendors do with the firehose? Back to my rhetorical question: What good is a plethora of data if you can’t find what you’re looking for? With more data comes increased importance of text analysis. Being able to pull in 65 million tweets a day is great in theory but requires vast amounts of processing and analysis to actually turn it into something of value. When someone tweets about what they had for lunch or I tweet about a soccer game I’m watching, these irrelevant items build into the countless others that make up that firehose. Dealing with that volume of data is no small task. Each of the vendors with access to the firehose also has strong underlying text analysis capabilities – a must for managing that level of data.

Ultimately, this is a big step in the listening platform market and certainly won’t be the last announcement of its type. Members of the large and fragmented vendor landscape will rush to add what data sources they can to differentiate and establish themselves as strong data providers.

What it means for listening platform vendors: It’s time to assess your social data sources. Do you have enough – both quantity and quality – data to deliver the insights your clients demand? The key to Social Intelligence remains the insights found within social data, but if adding sources to your platform increases the quality of insights your clients receive, you’d better find a way to make this happen. Keep in mind that Twitter’s firehose comes at an undisclosed, but speculatively high price tag – one that will exclude many smaller vendors. Bottom line: Focus on the insights your platform delivers.

What it means for Customer Intelligence professionals: accessing real-time information on your customers just because it's a bit more realistic. It's no longer a pipedream: Data savvy marketers - used to spending time retroactively updating databases - may now have up-to-the-minute information on their customers. But are you ready? Just as vendors have a lot to determine before diving into the data arms race, CI professionals must also evaluate their preparedness for a never-ending set of data.

Comments

To the future :)

I can tell I'm going to enjoy reading your blog quite often, Zach.
I was watching a video that other day that was describing the transitions that some people expect to occur as we transition from billions of pages to trillions of pages.
Who will win SEO battles?
Who will be "influential"?
The records that we are making now are going to be incredibly interesting to compare with what we will be able to do in a few years.

Michelle @Synthesio

Data Firehose

Thank you for these thoughts, and those about Social Intelligence. I am a marketer and we have embarked on this journey with listening platform and our own use of Social Media Analytics. It always seems to come back to the need for clear goals.
One of my co-workers coined a phrase..we are all becoming students of the data and we have the ability to adjust our campaigns so quickly to achieve different and better outcomes.