A question I often get when discussing online qualitative exercises is: what does the output look like? It’s true that qualitative data doesn’t come as easily packaged in a nice graph or chart as quantitative data does. In fact, how you analyze and captivatingly display qualitative results is a process that requires not only an analytical and logical mind but also a creative touch.
In particular, if you lack experience with qualitative data, it’s hard to find the story behind all the quotes and opinions you've received. I’ve put together a simple three-step process that you can use to begin synthesizing this information and creating your output.
Step 1: Bring order to the chaos. It’s important to know what the majority of your participants are saying. Start by creating a list of key themes as you read through the data, then tally each response that falls under that theme. You are not aiming to report numbers or percentages in the results, but this is the only way to begin to make sense of the wealth of information that you have at your fingertips.
Step 2: Stay focused and logical.You can’t create a sequential story if you let off-topic comments derail you. It's easy to focus on the negative comments — even if they don't reflect what the majority of participants are saying. Mark interesting comments or points, for example, by color-coding them, and come back to them after you have categorized the responses and answered the questions that underpin your main objective. Creating “callouts” in your report is a great way to highlight elements that aren't key parts of the story but that are interesting enough to share.
In a new report out today, my colleague Sarah Rotman Epps writes about the emerging but limited market for fitness wearables like the Nike+ FuelBand and the Jawbone UP. The report finds that only 4% of US online adults, or about 8 million consumers, fit a target profile predictive of buying a fitness wearable. Why so few? It turns out that mainstream consumers’ attitudes are very different from the health-conscious tech optimists buying these products today.
A few months back, we set out to understand how mainstream consumers feel about these devices using our Market Research Online Community (MROC) of 1,500 general US online consumers. As I’m using a wearable health-tracking device, I was excited to learn whether these consumers saw the same value that I saw in these innovative products.
Well, they don’t. In fact, “excited” isn’t even in their vocabulary when it comes to wearable devices. “Waste of money” was more how they described them.
Consumers feel that they know what to do to maintain a healthy lifestyle and use the concept of “moderation” to monitor their health, rather than fancy devices. In general, though, they lack self-awareness of their own unhealthy habits, they don’t feel accountable for their own health, and they expect their primary care doctor to monitor their well-being over the long term. Their perception is that wearable devices are for people who are chronically ill, need help with weight loss, or have obsessive personalities.
Some of you may not be aware that Forrester manages a market research online community (MROC) comprising 1,500 US online consumers recruited from our quantitative Consumer Technographics® surveys. While our Technographics surveys tell us what consumers do, the proprietary data we collect from our MROC completes the story by highlighting why consumers behave that way.
This year, Black Friday saw a record 89 million shoppers, up 3 million from 2011, according to a survey conducted by the National Retail Federation. In anticipation of this behemoth shopping day, we recently tasked our online community members with telling us anything and everything about their holiday shopping plans. This past Black Friday and Cyber Monday, we fielded a few short surveys to capture what our members were doing at that moment — what they bought, who they bought for, where they bought, and how much of their holiday shopping they accomplished.
At the end of this year, we will post a comprehensive report on our findings for the full holiday season. But to give you a small preview of what’s to come, here are a few “fresh from the field” insights that intrigued us right off the bat.
On both Black Friday and Cyber Monday, consumers primarily shop for their children and spouses. However, consumers are also shopping for themselves as well.
I’ve always been an advocate of storytelling when it comes to qualitative research, as per my blog post a few months ago. Often, this means multiple slides including in-depth explanations, quotes, and visualizations (e.g., imagery, infographics, etc.) — all must-haves for telling a proper story.
But in the fast-paced world in which we live, is there still time to develop a good story? I’ve had clients who only want to see the relevant information in quick, bulleted lists with a few short quotes perhaps. Are we moving to a model where the executive summary is the report? I hope not.
We don’t say “a picture is worth a thousand words” for nothing, and this is especially true for qualitative research. What would you find more valuable? Seeing a quote from a consumer in a Word doc or seeing a PowerPoint slide of that quote along with the consumer’s picture and demographic stats? Think of how this affects teams internally. All of a sudden, this quote isn’t just one voice telling the marketing or product teams what they’re doing wrong or how they should improve. All of a sudden, they see Bob H., a father of three who’s been buying your company’s product for 20 years because he thinks it’s the best on the market. It’s easy for an organization to ignore the voices but much harder to ignore the faces.
We do almost everything online these days, so why not research? I’m often surprised when I find others hesitant to conduct research online, but now and then I run into the occasional person who has reservations about moving research from offline to online.
Their primary concern centers on the quality of participants. How do we know they are who they say they are? How do we know they are giving good responses?
Fair enough. I might be concerned if people didn’t ask these questions. However, the general feeling is that there will be more quality issues with online respondents than offline respondents — but, of course, no one has ever lied about who they are in person, right?
As my previous comment might indicate, my response is that there really is no difference between the quality of online respondents and the quality of offline respondents. You face the same possible issues with respondent quality — and those who may fib about parts of their lives to qualify for a study or those straightlining respondents who participate solely to earn the incentive/be entered into that drawing. However, if you’re really concerned, sample providers such as Lightspeed Research have several metrics in place to ensure the quality of your respondents — as communicated in a recent blog post.
Talking to someone you have nothing in common with isn’t fun, and this is even more true when you are online. This is something you need to take to heart when you’re thinking of launching a community. Apart from coming up with a content plan, you need to take the time to clarify your objectives and participant pool.
The objectives of the community (i.e., what you want to learn from your community) and your participants (e.g., the people who will help you accomplish your objectives) are what we market research online community (MROC) experts call the “social glue” — they bond your community together to ensure it has a proper foundation to build upon.
The stronger the social glue, the better engagement you will have. It’s easy to start adding to your objectives and demographic requirements for participants, but it’s better to hold back. Sticking to one point of focus will get you the most in-depth and detailed results. A recent blog post by iModerate highlights this point for research objectives: “We tell many of our clients from the outset that they can either snorkel (cover more range, but stay shallow) or scuba dive (cover less, but go much deeper). If they want the richness that comes from thorough exploration, they need to follow our lead and we as researchers need to stay focused and diligent.” This also applies to the participants you select. Essentially, the broader the pool you have, the less in-depth your results will be because you have such a diverse group that you only can cover a wide variety of objectives superficially.
The analogy I always use to talk about qualitative research is that it’s the illustration to the quantitative story. What my own analogy assumes is that qualitative data on its own is an illustration. However, it’s really up to the analyst to bring this data to life.
Creating a visual story to display your qualitative data is an equally important part of the research process as the analysis phase, and something that is often rushed and not executed well. In my last blog post, I highlighted the fact that qualitative research is not just “quotes on a page.” You are doing yourself and your respondents a disservice if you rely solely on quotes and text to tell your story. Here are my tips to create an engaging report:
Kill your data darlings. My colleague Reineke Reitsma posted about this last month, and I couldn’t agree more. Don’t go overboard with numbers. Especially in qualitative research, too many graphics or percentages only distract from the story. Pick a few data points that strongly highlight your qualitative story, and challenge yourself to display them without using graphs and pie charts (i.e., via infographics.)
I have to share something with you — I’m upset. Why? Because many clients have no idea of the value of good, solid qualitative research, nor the investments needed. Recently, I was discussing a prospective qualitative research project; upon revealing the cost of such a project, one of the group members replied, “That is the same price as for a quantitative project; how can you justify that price?”
The conversation reminded me of my favorite quote from the movie You’ve Got Mail: Tom Hanks inquires about a book with hand-tipped illustrations and asks, “That’s why it costs so much?” and Steve Zahn retorts, “No, that’s why it’s worth so much.”
So, why is qualitative research worth so much?
Because there is a lot of skill involved in uncovering insights from qualitative research. Qualitative research is not about putting a couple of quotes on a page. It requires time, thought, and creativity to produce successful insights. What and who you put into your qualitative research process will determine what you get out of it. And it requires special skills. Unfortunately for us qualitative researchers, there aren’t many tools to help us with data analysis. Usually, it’s a manual process combined with a natural ability to read between the lines to pull out those impactful findings — combined with a creative mind to transform these into a compelling story.
This week there was a lot of discussion about panel quality and engagement after a respondent panel at CASRO. Part of the discussion was around incentives. Throughout my tenure in qualitative research, I have had many discussions on the pros and cons of offering incentives when conducting (online) research. In addition to my thoughts around incentives, I also surveyed Forrester’s online community of US consumers to get their opinions on the topic (quotes in italics below).
Generally you should always offer an incentive to participants for online research. However, what you offer and the value depends on a number of factors.
First, consider what you are asking of your participants.Are you asking for their feedback on a product they own or personal experience with a brand? This is where a lower incentive or, in some cases, no incentive could work because consumers who care about the product or brand are usually willing to share their experiences, and they can provide feedback on this type of topic fairly quickly. You see this, for example, a lot in co-creation communities. But when you ask a participant to complete a long study or multiple studies or when you ask for participation in a longer-term engagement such as an online community, it always requires an incentive to sustain their participation and ensure good-quality responses.
“It really depends on how much time I have to invest. If it’s a quick survey that doesn't take much time, I don't expect anything in return. But if it's going to be a lengthy process, I want something for my time.”
As the newest addition to the Market Insights team, allow me to introduce myself. My name is Lindsey Colella, and I recently joined Forrester as a Senior Community Manager.
It is a pleasure to “meet” all of you, and I look forward to many future interactions and discussions regarding market research. My background is in qualitative research and, in particular, cultivating insights through online community management. I take great pleasure in showing clients the value of qualitative and online community research and bringing them to a new level in understanding consumer behavior.
As some of you may know, Forrester runs its own online research community for two purposes — to conduct proprietary research as well as to run custom client research projects, both of which I manage. Our proprietary research is a monthly document called Community Speaks that discusses trends in consumer behavior. This product provides a unique offering because I work closely with expert analysts who provide additional insight around the findings.
As an example, I published a document last month covering how brands should engage consumers via social networking sites. A key finding from this report is that for a brand, earning a “like” is in fact the easy part but keeping that “like” is even harder. The key to maintaining a “like” from consumers is to provide information and promotional offers that relate to their interests. As one of our community members shares:
“I have unliked a lot of brands lately. There are just too many on Facebook to like. I try to limit liking brands that I actually use and interact with often and would benefit from learning more about that brand.”