What Happens To Our Traditional “Crutches” As We Adopt New Methods?

I have been chipping away at a unique research project for quite a few months now. This particular project has me diving into a data set comprised of respondents who have taken our Technographics® mail surveys for the past four years. My plan is to examine the technology behaviors and attitudes of these individuals over time, exploring things like how they adopt technology and how their attitudes change (or don’t change). While the exercise is challenging, and at times utterly maddening, it got me thinking about the role of trending and other traditional research approaches in the world of “new” market research (MR). Referring to trend lines created in quantitative surveys has been a staple of traditional market research, and benchmarking one year’s numbers over another is a must-have in any report. But do these traditional research activities still fit into the world of new MR? And if so, how? For instance:

  • What happens to trending? New methods such as social media monitoring don’t lend themselves to trend lines and cohort analysis very well yet. Does this mean we need to pivot our thinking away from leaning on trends to “normalize” our data? Or do we need to establish new benchmarks?
  • Who’s going to provide the benchmark figures we need? Will each company have one “traditional” survey it runs every year to gather benchmark numbers and complement its additional research with other methods? Or will this be a service vendors will offer — like a kind of Census Board? Will the lower dependency on traditional survey methodologies influence panel usage, and if so, what does this mean for panels if members are only fielded an occasional survey?
  • How do we remove our dependencies on traditional approaches? One of the great things about trending is that it acts as a safety mechanism. I’ll be the first to admit that I always ask to see trends when applicable so that I can determine whether a fluctuation in the data is real or not. Perhaps you’re a trend addict, too, or maybe you’re addicted to another particular sampling approach. Whatever it is, how do we get ourselves, and our stakeholders, to let go of those research crutches and move forward?

I’m not passing judgment on the trending (or non-trending) issue. Instead, I’m opening the conversation about how we move from traditional research to new methods. At the end of the day this is really about more than just trending data. As we explore and incorporate new methodologies, we need to have contingency plans in place that move forward with us. This probably includes research on research as well as running different research methodologies in parallel to understand the influence of the methodology on the research outcomes.

It may not be an easy or a direct path, but once it’s laid out, it will be easier for everyone involved to be bought in. In fact, I think I’m going to work on a road map for myself right now so that on my next report, I won’t be banging my head against the wall.

Comments

Ah, the Black Swan raises its

Ah, the Black Swan raises its head!

See 'The Black Swan', by Nassim Nicolas Taleb. It will challenge your view of statistics.

Back to your post:

Interesting line of research! What a dichotomy.

The goal of technology innovation is to disrupt the status quo in a massive way, and this conflicts with our emotional need to know the future, even when it is unknowable, and our propensity to 'make one up' to comfort ourselves.

This excerpt from your post seems to summarize the dichotomy:

*How do we remove our dependencies on traditional approaches?
and
*Who's going to provide the benchmark figures we need?

Notice that 'benchmark figures' is an assumed need from traditional approaches, which assume stable, predictable events, which we can no longer assume if we are to remove traditional approaches.

Facebook may have been a Black Swan event to most people; it was rare; it had an extreme impact; and, in retrospect, it was predictable. We say, "I wish I'd thought of that!" It seems so obvious in retrospect.

Yet it was not predictable. Not even to the founders. MySpace did not predict the rise of Facebook. Nor did any other Web2.0 business. Nor did Facebook foreshadow Twitter's rise.

It's a fascinating dichotomy:

What would you like to know? I want to know the future.

What are you working on? I'm changing the world! They'll never see this coming! It's gonna be Huge!

The mind moves in mysterious ways.

Warm regards,
Wayne

Hi Wayne, thanks for your

Hi Wayne, thanks for your comment. It's interesting that you mention that book as my colleague Reineke has been raving about it for quite some time now too. It is on my list of must reads!