- Forrester Councils
- Councils Overview
- log in
Posted by Nate Elliott on October 17, 2011
I've noticed a disturbing trend in one of the markets I study. Thirty percent of marketers say their top social media goal is creating brand impact, but only 10% tell us they measure brand impact — a gap of 20 percentage points. But then while just 4% say sentiment or engagement are their top goals, a whopping 26% measure these numbers —leaving us with an almost identical gap of 22 percentage points, but in the other direction. It’s clear what's happening here: Marketers are using sentiment and engagement numbers as a proxy for brand impact surveys.
Deep down I love the idea of measurement proxies. A properly constructed and proven proxy could be a cheap, quick, and effective stand-in for direct measurement of things that are quite frankly hard to measure — like brand impact.
But there’s a big problem here: I've been looking pretty hard for good measurement proxies for a while now, and I’ve found very few that could be described as "properly constructed and proven." And I'm pretty sure none of the marketers in our survey have proven their proxies — because if they'd tried, they'd have almost certainly failed.
In fact, Converseon, which we rated as a Leader in the listening platform space, tried to compare its standard sentiment data for one of its clients to that client's brand surveys and found what it called a "virtually nonexistent correlation." (That's a correlation coefficient of -.002 for you stats geeks.) So if you're busy using sentiment as stand-in for actual brand metrics, you may be interested to learn that one of the better vendors in the industry says there's probably no connection at all between the two sets of numbers. And that kind of "proxy" is pretty hard to love.
The good news is, there's hope that proxies can get better. For instance, by applying specially-designed weights and filters to its standard sentiment data, Converseon was able to improve the correlation between sentiment and brand surveys for that one client to .629 — still far from perfect, but a big step in the right direction. With any luck, it or another vendor will get that up to a more palatable .8 or better.
But until that happens, and until you cough up a good-sized amount of money to use one of these customized sets of weights and filters, your proxies probably aren’t telling you what you think they are. And that’s a big problem.