The Online Panel Quality Debate

A few years ago, Procter & Gamble publicly stated that it had experienced inconsistent research results from successive online research projects. Other organizations shared similar experiences, and questions were raised about “professional respondents.” The trustworthiness of online research was in question, and multiple initiatives arose. In the past two years, we’ve seen a lot of debate around this topic, and associations such as ESOMAR and ARF have come up with protocols that all good panels should follow — and many have. But what does this mean from a client perspective? How have initiatives like ARF's Quality Enhancement Process, MarketTools' TrueSample, or processes like machine fingerprinting changed the industry?

Next month, I'm hosting a panel at Forrester's Marketing Forum 2010 with participants from Microsoft, Procter & Gamble, and the ARS Group to understand what the challenges with online sampling are today — and how they affect adaptability.

Questions I will discuss with the panel include the following:

Read more

Capturing The Unconscious

Last week, I attended Research 2010, the research conference organized by the UK's Research Organization. One session was on innovative research methodologies, and although it's not completely new to the industry, I was surprised to see two of the presentations covering research methodologies that capture people's unconscious behavior through technology.

The first was a presentation about lifelogging, or “glogging” for those in the know. Simply put, lifelogging documents somebody's life through technology worn by the “respondent.”

Bob Cook from Firefish presented how this technology helps researchers better understand the tradeoffs that people constantly make. Lifelogging has a long history, and it was started by Steve Mann. In the early 1980s, he walked around with recording gear that looked more like a suit of armor.

Read more


The Data Digest: Which Devices Have People Connected To Their TV

About 40% of US online adults now have a home theater audio system connected to their TVs, providing a better sound experience than the typical speakers connected to a PC or those built into a boom box. Forrester’s Consumer Technographics® data shows that US consumers who have home theater systems take home entertainment seriously; they have a variety of entertainment devices, including set-top boxes, connected to their TVs.


More information can be found in the report 'Audio In The Home Is A Feature, Not A Device Category'.

Learn To Set Your MROC Up For Success

Recently, I’ve been having conversations with clients who know they want to start up their own market research online communities (MROCs) and know what they want to learn through this kind of research, but they are overwhelmed with where to start. For a lot of market researchers who are dipping their toes into using private communities for research, this is a common pain. Anxiety can easily creep into the MROC ramp-up process when it comes to vendor selection, recruiting, research planning, and reporting – to name a few. And, although many vendors and conferences provide case studies on how MROCs have delivered amazing results, it’s hard to find client case studies around the do’s and don’ts of building one successfully.

I’ve presented and written around this very topic in the past, but for Forrester’s Marketing Forum in LA this April, I wanted our audience to hear from the ”frontlines” of companies who have actually been through this process themselves. If you are trying to navigate the ins and outs of community research, please join me in LA on April 23 for a panel discussion, entitled “Elevating Market Research Through Real-Time Community Insights.” I’ll be leading the conversation with three well-known brands as they talk about their challenges and successes with integrating MROCs into their insights processes. Each of the panelists uses different vendors and they come from distinct company cultures, so I anticipate a very informative 45 minutes.  

Read more

The Data Digest: Online Payment Preferences In International Markets

I've been analyzing consumer technology uptake for years — helping retailers, for example, understand the barriers to and drivers of online buying behavior. Forrester's Technographics® research shows that preferred online payment methods differ greatly between countries, and companies need to understand this complexity of payment options and how that affects consumer behavior.

Unlike in North America, where the top payment methods tend to be similar in the countries surveyed (the US and Canada), the payment preferences of online buyers in Europe differ both between countries and from their North American counterparts. For example, the popularity of prepaid cards is unique to Italy: Roughly a third of Italian online users have taken advantage of prepaid cards. Global organizations need this detailed understanding of consumer payment preferences across markets in order to be successful internationally.

For more information: This week, Forrester published a report called “Understanding Online Payment Preferences In International Markets” that examines consumers' preferred payment methods in North America, Europe, and Asia Pacific to help eBusinesses identify opportunities for payment localization.

Preferred payment options globally

Three Key Considerations On Social Media For Market Research

For my current research on social media and market research, I’m interested in listening platforms (and the text analysis that’s usually packaged with them) for the purposes of mining the social Web – be it on blogs, open community sites, social networks or the like. 

There’s a lively debate around the value of social media listening for market research, and there are many people willing to share their opinion. Last week, I attended a Webinar on this very subject, hosted by Peanut Labs, with multiple guest speakers from the industry. Here are some of the key points that market researchers should consider when assessing the need for – and effort in -- social media research. 

“Process and methods need to be developed to make social media data be another source for Marketing Research” -- From Jean Davis, co-founder of Conversition, and former president of Ipsos Online, North America. This means:  Platforms need to be created with the market researcher in mind.  They must be able to reliably sift through online conversations to sort out low-quality data; apply weighting schemes to that data reflects that true share of volume that different sources have online; and create constructs so that data from social media can be proxied to represent common measures such as five-point scales and top-two boxes. 

Read more

Never believe a single data point when forecasting

We often receive questions such as “researcher X’s forecast is much higher/lower.”  I always take these remarks seriously – there are many elements an analyst has to take into account in forecasting and  I’ve learned over time that the content of these conversations drives the value of the forecast (for both parties).

Recently I had a discussion with a client who was skeptical of our growth projection for digital music subscriptions in our recent Music forecast.  We discussed that a key input to our estimate are discussions with companies providing these services and understanding their growth outlook.  But these discussions are expectedly biased, that’s why we routinely challenge providers to defend their expectations but we also look for corroborating data to indicate direction and scale of change. Similarly, if survey respondents happen to love a product idea, it has potential but there’s no certainty that consumers will eventually spend money on that product.  What we look for is a solid pattern of evidence supporting a growth hypothesis along with a paucity of evidence supporting the alternative (decline).

As part of the forecast process, we as analysts debate various hypotheses and I shared the content of those discussions with this client, which helped him understand that our growth expectation for digital subscription is supported by evidence beyond the confidence of subscription providers.

Read more

The Data Digest: The State Of P2P File Sharing

This week my colleague James McQuivey published a report called 'Casual Video Piracy Kept At Bay For Now'. Forrester's Technographics research shows that online video piracy is a minority behavior. Just 7% of US online adults regularly engage in peer-to-peer (P2P) file sharing, and less than half of them use it for video files. In fact, more people have given up on P2P file sharing than currently still do it.


Peer to Peer filesharing uptake

Another data point in this report revealed that everyone prefers a legal alternative. Our respondents were eager to reassure us that they prefer to use legal sites for watching videos, if these would give them a convenient way to serve their video needs.

Introducing our new blog platform

It's with great pleasure that I introduce our new blogging platform to you! Please let me know your thoughts.


In this first post on the new platform, I'd like to introduce Cliff Condon, the project manager, who likes to share his thoughts on Forrester blogs and the new functionality with you:

  • Everyone’s welcome here.  Forrester analysts use blogs as an input into the research they produce, so having an open, ongoing dialogue with the marketplace is critical. Clients and non-clients can participate – so I encourage you to be part of the conversations on Forrester blogs.

  • We still have team blogs focused on role professionals. Our role blogs, such as the CIO blog and the Interactive Marketing blog, are a rollup of all the posts from the analysts serving that specific role professional. By following a role team blog, you can participate in all the conversational threads affecting a role.

  • And now we’ve added analyst blogs as well.  If you prefer to engage directly with your favorite analyst, you can. Look on the right-hand rail of the team blog and you’ll see a list of the analyst blogs.  Just click on their name to go to their blog.  Or type their name into “Search”.  An analyst blog is a place for the analyst to get reaction to their ideas and connect with others shaping the marketplace.  You’ll find the blogs to be personal in tone and approach. 

Read more


Market Researchers Need To Embrace Knowledge Management

In the past couple of months I've been working on a document called 'Information Management For Market Researchers', released earlier this month to our dedicated Forrester Market Research Leadership Board Members. Although I can't share all lessons learned with you yet, there are a couple of insights I'd like to bring to your attention.

The most important outcome from my interviews with market researchers and knowledge managers is that a culture of sharing creates better products and helps companies be more successful innovators. Simply said: to innovate, knowledge from various departments needs to come together, irrespective of role or rank.

Read more