Research Panels vs Research Communities -- What's the difference?

Tamara Barber [Posted by Tamara Barber]

I’m currently working on a piece of research we here at Forrester call the Wave, which is our vendor comparison methodology. I’m examining vendors in the Market Research Online Community (MROC) space and am very much looking forward to sharing my findings once the research is published.

In the meantime, I’ve been doing some soul searching on what differentiates a research community from a typical online panel. Vendors and clients alike generally have very specific views on the difference. Here’s a sampling of quotes from a recent survey I fielded on MROCs[i]:

“An online panel suggests that you'll be using it solely for survey research. While an online community is more involved - discussions, community building activities, as well as surveys.” 

“An online community is an ongoing social networking forum which I would consider qualitative. A quantitative online panel allows you to do one off quantitative research projects.”

“Panels are large (often a thousand or more) and are often utilized once or twice per research project (i.e. a survey and a follow-up).  MROC[s] are long-standing communities (often a year or more) with a smaller group (300-500) of participants.”

All valid points, which I think highlight three key differences between communities and panels. Communities are:

  • Qualitative. Results are directional and usually biased toward a sub-population of a researcher’s target market. Community research should not take the place of statistically sound, representative quantitative research, but it should definitely be used for idea generation, ethnographic exercises, or getting a quick pulse on a certain type of consumer.
  • Interactive. Companies lose the community feel if they don’t engage their members in an interactive manner. This means researchers should manage communities so that they ask for and provide feedback to the members, and also allow and encourage members to interact amongst themselves.
  • Iterative. While communities can certainly run for short periods of time, the actual research exercises that happen in a community are based on building ideas off of each other. Rather than a typical radio-button survey that elicits further questions but are never followed up on.

But does size matter? Most vendors in this space have the capability to host tens of thousands of members in one community, but I’ve found that, the larger a ‘community’ is, the more often it’s used for quantitative insights only – just as a panel. And the larger a community gets, the harder it is to manage. My conclusion for the time being is that there is such a thing as a community that is too big for doing proactive market research exercises. I am very interested to hear your views, since the topic is a little controversial at the moment. I welcome (and encourage) your feedback and comments!

[i]These quotes are from Forrester’s Q3 2009 Global MROC Online Survey, which we fielded among 78 market researchers.

Comments

re: Research Panels vs Research Communities -- What's the diffe

Your readers may be interested in Ray Poynter's discussion of Communities for research in a recent Webinar we hosted, "At the Bleeding Edge of Online Research". You can download the Webinar using the link below. An audio file is also available on our Downloads page.Webinar: http://www.peanutlabs.com/peanutlabs/Cookie?link=Webinar_At_the_Bleeding_Edge_of_Online_Research.pdfAudio file:http://static.peanutlabs.com/ConferenceRecordingAttheBleedingEdgeWebinar.mp3

re: Research Panels vs Research Communities -- What's the diffe

Hi TamaraMatt Foley from PluggedIN always has a useful and interesting perspective on this topic. Of particular relevance in this case;http://www.pluggedinco.com/blog/bid/24746/My-best-online-research-community-adviceThe rest of PluggedIN's blog is well worth a look too.Katie

re: Research Panels vs Research Communities -- What's the diffe

Hi TamaraI think you're pretty much spot on. Your piece resonates with what my colleagues have been saying:Community value: http://www.marketingmagazine.co.uk/news/924079/value-online-research/Communities vs panels: http://www.virtualsurveysdiscussion.com/vslblog/?p=125CheersJon Beaumont, Director, Virtual Surveys

re: Research Panels vs Research Communities -- What's the diffe

Tamara, if you are trying to get insights, then the relevant question is less "How big should the community be?" and more "How do you get the most engagement?". Often, people claim to have a large community, but despite the large numbers of members, participation is very low. In an effective research community, you want to maximize numbers of members who are engaged and creating valuable content on a continuous basis. The reason many clients like smaller, more intimate communities are that participation is often MUCH higher when things are intimate -- just like in the offline world. Think: Gee, that lecture was interesting...now let's break up into smaller groups so that we can talk about it!

re: Research Panels vs Research Communities -- What's the diffe

Thank you, all, for your comments and links to other presentations and blogs about this topic. Diane you make a great point about the importance of engagement, and how that should drive decisions about size. Many of the links I got talked about this as well. I think we’d all agree that clients and vendors alike must be able to justify why any given community should have 50 members versus 500 versus 1000. And also have a plan for how to keep those members engaged regardless of size. The issue becomes even trickier when communities are used for both qualitative and quantitative research, where in the latter you want to have a sample size that will yield statistically significant results. This is something that I see more and more clients interested in doing. I look forward to keeping this dialogue open, and to hearing your thoughts on the MROC research I'll be publishing in the coming months.

re: Research Panels vs Research Communities -- What's the diffe

Tamara, I can't resist! You're talking about one of my favorite topics! :)I hear you on statistical significance. However, if that's really your ultimate goal, you can already get that from a traditional panel. The difference between a panel and a community is that in the community, people know each other and build relationships -- and thus the resulting insights have the potential to be infinitely more textured, honest and insightful. Statistical significance, while useful, is very rarely the driver behind the need for research communities because clients already have through other methods. At least for us (I'd be interested in hearing from others), they want MROCs because of what they can't get from other methods, which is a fast and easy way to bring the voice of the customer to life. Having said all of that, our experience is that we rarely get more actual participation when we double the size of our communities. Intimacy is lost, and thus, engagement goes down exponentially.