Posted by Tom Grant on March 23, 2009
I've had to put blogging on the back burner for the last week because one research document, covering how product managers can use social media (blogs, Wikis, forums, etc.) as a new source of product requirements, underwent mutation, and then mitosis. Now, it's three separate documents, each of which demands all the empirical and stylistic discipline that Forrester demands. In short, I've been busy.
However, now that two of the three documents are drafted and in the capable hands of my research director, I need to vent. The target of my ire isn't the tribble-like spawning of new documents, which at the end of the day, is the right call, and has made the author much happier with the final product.
No, what's really churning in my guts is the inevitable outcome of talking to people about product requirements, writing about how to improve them, and re-visiting the topic during the editing process. Let's be honest: usually, they stink on ice.
I like that idiom because it evokes a truly epic scale of odiousness.If, in the depth of winter, when everything liquid or semi-solid freezes, and the great slab of ice on which you're standing leeches all the warmth and life out of you, slowly killing each of your senses in turn you're smelling something foul, it must be really foul.
No single post can hope to capture the complete odiousness of traditional requirements. I'll space this philippic across at least a couple of post, but let's start with the most obvious problem, and the destruction that it wreaks on sensible product decisions, not to mention the working relationship between product management and development.
Wandering in the informational desert
What are the most frequently used sources of requirements in the technology industry? According to our surveys, the results are anything but surprising to us veterans: (1) customer meetings, either face-to-face or over the phone; (2) bug database entries; (3) enhancement requests transmitted through e-mail, often via mailing lists. Customer meetings are also the most trusted source of insight, because they provide far more information than a vague "I have a customer" e-mail from someone in Sales, or a bug report that conveys more frustration than information.
Of course, as "useful" as customer meetings are, they're hardly frequent enough to create a statistically significant sample. People should be skeptical, when the N is small, of how representative the customers are of the larger market, and how reliable the conclusions based on this information can be. Scheduling a few more conference calls, or getting leave to visit another customer on-site, will not provide enough empirical fuel to launch to slip the surly bonds of skepticism.
All of which leads to statements we've all heard before:
holds the opinion of the last customer to whom he spoke."
"This customer is full of spurious requests. How much can we trust their judgment?"
"I've spoken to as many customers as you have. What makes you a greater expert on customers than me?"
"We don't have time to keep going back to these people to see what they really want."
And so on, truly ad nauseum.
Therefore, one of the virtues of social media as a new source of insight into product decisions is the sample size. Would you rather base the next 12 months of development on an hour's conversation with five key customers, or information collected from hundreds of customers posting on discussion forums and blogs?
Certainly, there are lots of potential ways to misuse social media in this fashion. Aggregate information may be miles wide, but inches deep. Collected sloppily, the sample may be just as skewed as the handful of customers with whom you normally deal.
Still, it's hardly a courageous leap to say that social media provide some useful information that can bear on product decisions. At the very least, these other sources of information can supplement the traditional stuff.
When we return, be prepared to hold your nose as we wade through the other problems with traditional requirements.
[Cross-posted at The Heretech.]