Traditional requirements stink on ice

I've had to put blogging on the back burner for the last week because one research document, covering how product managers can use social media (blogs, Wikis, forums, etc.) as a new source of product requirements, underwent mutation, and then mitosis. Now, it's three separate documents, each of which demands all the empirical and stylistic discipline that Forrester demands. In short, I've been busy.

However, now that two of the three documents are drafted and in the capable hands of my research director, I need to vent. The target of my ire isn't the tribble-like spawning of new documents, which at the end of the day, is the right call, and has made the author much happier with the final product.

No, what's really churning in my guts is the inevitable outcome of talking to people about product requirements, writing about how to improve them, and re-visiting the topic during the editing process. Let's be honest: usually, they stink on ice.

I like that idiom because it evokes a truly epic scale of odiousness.If, in the depth of winter, when everything liquid or semi-solid freezes, and the great slab of ice on which you're standing leeches all the warmth and life out of you, slowly killing each of your senses in turn you're smelling something foul, it must be really foul.

No single post can hope to capture the complete odiousness of traditional requirements. I'll space this philippic across at least a couple of post, but let's start with the most obvious problem, and  the destruction that it wreaks on sensible product decisions, not to mention the working relationship between product management and development.

Wandering in the informational desert
What are the most frequently used sources of requirements in the technology industry? According to our surveys, the results are anything but surprising to us veterans: (1) customer meetings, either face-to-face or over the phone; (2) bug database entries; (3) enhancement requests transmitted through e-mail, often via mailing lists. Customer meetings are also the most trusted source of insight, because they provide far more information than a vague "I have a customer" e-mail from someone in Sales, or a bug report that conveys more frustration than information.

Of course, as "useful" as customer meetings are, they're hardly frequent enough to create a statistically significant sample. People should be skeptical, when the N is small, of how representative the customers are of the larger market, and how reliable the conclusions based on this information can be. Scheduling a few more conference calls, or getting leave to visit another customer on-site, will not provide enough empirical fuel to launch to slip the surly bonds of skepticism.

All of which leads to statements we've all heard before:

"He strongly

  • holds the opinion of the last customer to whom he spoke."

  • "This customer is full of spurious requests. How much can we trust their judgment?"

  • "I've spoken to as many customers as you have. What makes you a greater expert on customers than me?"

  • "We don't have time to keep going back to these people to see what they really want."

And so on, truly ad nauseum.

Therefore, one of the virtues of social media as a new source of insight into product decisions is the sample size. Would you rather base the next 12 months of development on an hour's conversation with five key customers, or information collected from hundreds of customers posting on discussion forums and blogs?

Certainly, there are lots of potential ways to misuse social media in this fashion. Aggregate information may be miles wide, but inches deep. Collected sloppily, the sample may be just as skewed as the handful of customers with whom you normally deal.

Still, it's hardly a courageous leap to say that social media provide some useful information that can bear on product decisions. At the very least, these other sources of information can supplement the traditional stuff.

When we return, be prepared to hold your nose as we wade through the other problems with traditional requirements. 

[Cross-posted at The Heretech.]



re: Traditional requirements stink on ice

Tis but a poor musician who blames his instrument.Look, the real problem with requirements is not that we've never had blogs/wikis/etc. with which to collect some form of cognitive thought about what our customers REALLY want next. The real problem is that we simply don't know how to ask the right questions.I mean, who ever had a class / course on how to ask probing questions? Sales folks do it better than product managers, but even they are fumbling their way along.You need to enter into a discussion with a customer with an open mind and no preconceived notions. Then you need to play the role of a 2-year old and keep asking "why" until they become exhausted - it's their last answer to this question that is the REAL requirement.- Dr. Jim Andersonwww.TheAccidentalPM.comThe Accidental PM Blog"Home Of The Billion Dollar Product Manager"

re: Traditional requirements stink on ice

I agree wtih Dr. Jim.The problem with collecting requirements isn't with the method of collection; but, rather, the method used in asking the questions.Too often I have seen companies require that their product management team have the ability to build the solution they are responsible for managing. A more useful approach is in having the product manager being able to have the business discussions to understand the market problems. The problems drive the requirements. Not the developers! This is all too frequent. Those companies that understand the investment in uncovering/discovering the problems at the core, are the ones with effective product management, and success with the product.

re: Traditional requirements stink on ice

The "art" of product management is separating the signal from the noise - knowing which statements by which customers should be actionable, and which statements are just fluff. I've worked at several companies where either (1) everything Sales said was the gospel truth, or (2) everything that Sale said was bunk. Neither of these are true, in reality - everyone that touches your product has some useful input at some point in time.The danger with relying on social media is really twofold to me - (1) you're diluting the pool of sources by opening your door to a broad audience, and (2) by doing so you're increasing the noise ratio. Not that it shouldn't be done, of course, but just like everything else that we do, there has to be a well-thought-out strategy behind it. You can't just open up a Facebook account and hope it brings in quality feedback - that's just ridiculous.

re: Traditional requirements stink on ice

Interesting thoughts. I like to think the value I bring to product management is exactly that ability to tell useful from useless, to determine the business need behind the request, and to then communicate a requirement to the dev side that addresses the business need.I like social media as well as the next person, but it definitely has its limits. If you don't have very many customers (as in, I work at a small startup), they you don't have critical mass for a thriving community, which requires about 400 members. If the members of your community are representative of the web as a whole, you might end up with a result like did: "Recent crowdsourcing effort proved to be not entirely representative of the United States, considering that during time of war and financial crisis, two of the top five vote-getters from the "Citizen’s Briefing Book" were issues related to marijuana legalization." (from the Personal Democracy Forum via Tech President: summary, I put a good, well-trained, product and domain knowledgeable product manager ahead of a particular choice of tool or approach. Even the next new thing, social media.

re: Traditional requirements stink on ice

Dr. Jim said:"Look, the real problem with requirements is not that we've never had blogs/wikis/etc. with which to collect some form of cognitive thought about what our customers REALLY want next. The real problem is that we simply don't know how to ask the right questions."I'd never say that traditional sources are the only problem with requirements. I think you've put your finger on something important here, the inability to ask good questions.The technology industry isn't the only corner of the world that struggles to phrase a good question. Many professional question-askers, like the White House Press corps, suffers from the same ailment.BTW, I spent a fair amount of time in the second social media document talking about this issue. "Define a clear hypothesis to test" is fundamental for any requirements exercise. Social media just bring up a good excuse to re-visit that issue.