The Problem With Measurement Proxies

I've noticed a disturbing trend in one of the markets I study. Thirty percent  of marketers say their top social media goal is creating brand impact, but only 10% tell us they measure brand impact — a gap of 20 percentage points. But then while just 4% say sentiment or engagement are their top goals, a whopping 26% measure these numbers —leaving us with an almost identical gap of 22 percentage points, but in the other direction. It’s clear what's happening here: Marketers are using sentiment and engagement numbers as a proxy for brand impact surveys.

Deep down I love the idea of measurement proxies. A properly constructed and proven proxy could be a cheap, quick, and effective stand-in for direct measurement of things that are quite frankly hard to measure — like brand impact.

But there’s a big problem here: I've been looking pretty hard for good measurement proxies for a while now, and I’ve found very few that could be described as "properly constructed and proven." And I'm pretty sure none of the marketers in our survey have proven their proxies — because if they'd tried, they'd have almost certainly failed.

In fact, Converseon, which we rated as a Leader in the listening platform space, tried to compare its standard sentiment data for one of its clients to that client's brand surveys and found what it called a "virtually nonexistent correlation." (That's a correlation coefficient of -.002 for you stats geeks.) So if you're busy using sentiment as stand-in for actual brand metrics, you may be interested to learn that one of the better vendors in the industry says there's probably no connection at all between the two sets of numbers. And that kind of "proxy" is pretty hard to love.

The good news is, there's hope that proxies can get better.  For instance, by applying specially-designed weights and filters to its standard sentiment data, Converseon was able to improve the correlation between sentiment and brand surveys for that one client to .629 — still far from perfect, but a big step in the right direction. With any luck, it or another vendor will get that up to a more palatable .8 or better.

But until that happens, and until you cough up a good-sized amount of money to use one of these customized sets of weights and filters, your proxies probably aren’t telling you what you think they are. And that’s a big problem.

Comments

Measuring social

Nate,

I think your basic direction is insightful, but would love to suggest that you advise your readers and subscribers to treat their social media marketing programs with the same rigor they treat any of their other more traditional online initiatives when it comes to tracking, measuring and optimizing:

tag everything.

It really is that simple. We've been doing this with increasing science and precision for the past several years and you can too. Whenever your social channels managers drop a comment or a link into the social stream - encode it with the appropriate tags to track it. Simply using a free tool like Bitly will support a brand's actual tracking and measuring the effect of their social initiatives in this way.

Whenever you post a video for sharing, favoriting - encode it with a tracking code.

Whenever you tweet, encode the shortened URL embedded in the tweet.

Whenever you create a piece of social content, ensure it's got +1 embedded.

Whenever you post a pic or a note to your brand's Tumblr encode it with a trackable code.

Pushing a short, sweet post your brand's Facebook Wall? right, encode the shortened link to wherever the click takes the user from their stream.

Whether you're using Google Analytics, Site Catalyst, WebTrends or free-loading happily with Bitly, you simply build these tracking measurements into your existing dashboards and weekly marketing campaign reports.

Then you optimize your social media programs accordingly.

I'm with you --- "sentiment" tracking is soo 2007. Treat your social media like all your other media and you will soon recognize the immense opportunity for shifting traditional "digital" budgets, with all their CPM-driven waste, over to the more scalable, trackable and infinitely more engageable social media of the brand.

Thom Kennon | @tkennon | + Thom Kennon | bigevidence.blogspot.com

Advocacy is the best proxy for sales and share

In fact we spent a year with the statisticians at Northwestern in 2006 exploring all of the social metrics that we measured looking for the best correlations with sales or share.

We found that advocacy (brand recommendations) had the highest correlation. We have been validating this ever since and, to be honest, were surprised to find that it holds true in both high and low interest categories.

Some more information can be found here:http://www.motivequest.com/blog/index.php/social-media-measurement-roi/?utm_source=rss&utm_medium=rss&utm_campaign=social-media-measurement-roi and here: http://humanvoice.wordpress.com/2011/04/11/social-media-roi-advocacy-for...

Cheers

David Rabjohns, CEO, MotiveQuest