Posted by Tom Grant on October 21, 2010
Undoubtedly, when you read the title of this blog post, you thought, "But politicians are users." No, I'm not talking about that kind of user. Instead, I'm thinking of the user we normally discuss around these parts, the kind of person whom product managers and product marketers try to understand, but often don't.
The 10Questions Project posed questions from salt-of-the-earth, ordinary folk to California senatorial candidates Carly Fiorina and Barbara Boxer. It's a noble failure, a change of medium that had no discernible effect on the quality of message. The candidates' answers are exactly the sort of vague, high-level statements that are more about sentiment (what candidates hint they might do) than policy (what they'd actually do).
These frustrating YouTube snippets resemble the kind of bad answers that users often give when PMs ask them questions like, "So, what would you like to see in the next release?" The reasons for the uselessness of the answers are the same, too:
- The questions.
- The channel through which they're answered.
- The lack of follow-up.
We Get The Answers We Deserve
When I was an undergraduate, the advisor for our campus newspaper, Joe Bell, was a veteran reporter at The Los Angeles Times. Whenever he saw an article in our college rag that failed to meet his standards, his criticism invariably boiled down to, "You didn't ask the hard questions." While the ursine Joe was probably growling hard questions at his parents as soon as he could talk, the rest of us needed to acquire the skill of asking hard questions. That was the whole point of having an advisor, to help us pose better questions to university administrators. But who teaches us how to pose hard questions to politicians or customers?
Open-ended questions are obviously bad. Anything as mushy as, "What do you think of the new UI?" or, "What's your position on net neutrality?" is an open invitation for people to give you the answers they want to give, or the answers they think you might want. Both are bad.
The less obvious mistake is posing a question that sounds tough, but really isn't. On the 10Questions site, this question about Afghanistan is a prime example. If you build into a question a preferred outcome that the other person may not share – in this case, bringing US personnel home from Afghanistan as quickly as possible – don't be surprised if that person ignores your question entirely. After disagreeing with the premise of the question, Fiorina delivers the answer she already prepared ("I support drone attacks, David Petraeus is my BFF, blahblahblah").
PMs make this mistake all the time, without realizing it. If you replace the question, "What do you think of the new UI?" with something more specific, such as, "Do you think the new UI is perfect for knowledge workers between the ages of 25 and 35?" the customer may be just as disinclined to answer, perhaps for different reasons. (Neither caring nor knowing anything about the work habits of that demographic probably tops the list.) If the customer does want to make an observation about some other aspect of the Web UI, you may just wind up talking past each other.
Not Every Answer Should Be A Sound Byte
Another problem with the 10Questions site is the brevity of the answers. I'm sure that the candidates happily agreed to the laconic 10Questions format, not just because it fit into their busy schedules, but also because it made it possible to avoid saying anything specific. For example, Boxer ate up 20 seconds out of this 55 second answer just setting up her statement about the job-growing policies she'd support. She then asserted that a particular bill contains eight tax cuts that will help small businesses grow...But what are they? And how exactly are any of them likely to inspire your local auto dealership to hire another mechanic? Where would I even find these answers?
PMs, too, grapple with limited communications. Some may be as self-imposed as the 10Questions format, but many are not. You can only put so much information into a bug report, and you can't control how much information a customer puts into a post on a community site. PMs have a few corrective mechanisms – adding fields to the bug database, socializing customers into providing more information, triangulating among multiple requirements sources – but they often don't take these steps, or even realize they're necessary.
Most Answers Are Incomplete
Probably the most maddening part of the 10Questions site is the lack of follow-up. Their brief YouTube clips allow both candidates to mouth whatever blather they want, without fear of messy follow-up questions. The site has a thumbs-up/thumbs-down mechanism for rating the answer, which is about as useful as rolling down the window while driving on the freeway so that you can yell, to no one in particular, "Vote yes on Proposition 9,134!" I gave a thumbs down to Carly Fiorina's 7 second answer to a serious question about our nation's drug laws, but so what?
What might actually have some value on the 10Questions site is blazingly obvious: a comments feature. Sure, the candidates will ignore the comments, but they give voters an opportunity to talk to one another, instead of remaining a gaggle of atomized, anomic people passively watching YouTube clips. Even if 10Questions failed to push candidates into substantive exchanges with voters, it might have created some valuable conversations among voters.
Conversations with customers can be equally incomplete. Some of the most commonly used sources of requirements, such as infrequent customer advisory board meetings and bug databases, don't lend themselves easily to follow-up questions.
Most of these communications don't enable or encourage conversations among customers, which can elicit information you wouldn't glean from compartmentalized exchanges between a vendor and an individual customer. For example, during my recent research on SaaS, one PM said that, during a heated exchange between two customers on the vendor's community site, they were really looking to the same product to accomplish different ends. Therefore, there was no simple way to balance the interests of Customer X versus Customer Y, because one of them wanted a solution that the vendor didn't support.
So That's Why You're Interested In Those Topics
Better to rely, whenever possible, on a medium that lowers the risk of making these mistakes. Visualization tools, for example, change the medium of requirements in a way that encourages a back-and-forth exchange in terms that both customer and vendor can understand. Without these tools, it's easy for the voice of the customer to shrink to the size of an exclamation ("I want something!"). With them, you're far more likely to enjoy a meaningful exchange about what exactly the customer wants, why it's important, and how the technology should work.
That's one reason why I spent a substantial amount of time earlier this year on the evolving requirements tool market. Ditto for serious games, which replace unstructured conversations with structured activities, and open-ended desires with exclusive choices.
Getting people to make exclusive choices, or answer other meaningful questions, takes genuine skill. In politics, when someone fumbles the opportunity, politicians don't provide useful answers. Should we be surprised when customers do the same?
Search Forrester's Blogs
- Anjali Yakkundi (25)
- Boris Evelson (140)
- Claire Schooley (2)
- Clay Richardson (1)
- Diego Lo Giudice (16)
- Gene Cao (1)
- George Lawrie (17)
- Holger Kisker (38)
- Ian Jacobs (2)
- James Staten (8)
- Jeffrey Hammond (27)
- John R. Rymer (45)
- Jost Hoppermann (33)
- Kate Leggett (120)
- Kurt Bittner (4)
- Kyle McNabb (12)
- Margo Visitacion (9)
- Mark Grannan (9)
- Martha Bennett (12)
- Michael Barnes (21)
- Michael Facemire (14)
- Mike Gualtieri (114)
- Noel Yuhanna (10)
- Paul Hamerman (2)
- Phil Murphy (24)
- Randy Heffner (15)
- Rob Koplowitz (2)
- Stephen Powers (23)
- Ted Schadler (4)