Posted by Zach Hofer-Shall on February 6, 2012
Although most of my Cambridge-based colleagues don't want to bring it up, last night's Super Bowl was exactly the spectacle we've come to expect from the nation's most-watched event. We saw hundreds of new commercials (some good and many bad), a crazy half-time show (with a random tightrope walker), and one other thing . . . what was that? Oh, yeah, a football game.
In the weeks leading up to the game, I noticed a trend around the game itself. Dozens of blog posts and news articles claiming they could predict the Super Bowl winner using social media. Although most of these were fluff pieces to fill a slow news week and capitalize on the nation's renewed interest in the NFL, my research skepticism kicked into overdrive with some of them. Not to call anyone out directly, but with all of the PR teams sending me press releases about "predicting" the outcome, I just can't let this slide. So, can social media predict the outcome of the Super Bowl? No.
Each of these predictions came from collecting and analyzing social data. Some predictions came from simple metrics like the volume of mentions around one team against the other. A few of the predictions used the sentiment of mentions — such as a positive mention for the Patriots versus a negative mention for the Giants. And some predictions even used influence calculations to understand how different market segments discussed their favorite teams. In the end, this means that some of the predictions were right and some were wrong. But hey, it was a 50/50 shot anyway. Even with coin-flip odds, it seems that more than half were wrong . . . but that actually distracts from my argument, because even if they guessed right, they were wrong to do so.
It seems many of these predictions forgot that there's absolutely no causal relationship between fans' opinions and players' actions. Just because more people tweeted about one team does not mean that team will win. Unless the NFL changes the rules at some point in the future to award the team with the most social-media-active fans extra points, then the fact stands that these two things have nothing to do with each other. Unlike voting shows like American Idol, where fans can influence the outcome, the Super Bowl comes down to factors outside of fans' control. Using general opinion to predict a game winner simply doesn't work.
I'll spare you the rest of this rant and jump to the reason that I'm ranting in the first place: This misuse of social data is bad for the state of social intelligence. Posting these predictions, whether they're right or wrong, gets people thinking that we can ignore data standards when using social media. It cheapens the data. More and more businesses are treating social media as a valuable source of customer data, but when they see that it incorrectly picked the Patriots to win, it makes them question its validity when they really should question the research methodology.
Maybe I'm just in a bad mood because the Pats lost, or maybe it's time that we stop isolating weak metrics to make unrelated predictions. But as a proponent of social intelligence, I'd say instead that it's time we start treating social media like real data, institute real research standards around social data, and begin answering real business questions.