Doing Research In An Agile Fashion: A Retrospective

Our first application of Agile principles to the research process, which occurred during our study of thought leadership in the tech industry, was a very Agile-esque journey into the unknown. We learned a great deal that applies to future applications of Agile Research Development, so here's my retrospective.

You Can Be Agile Up To The Boundaries Of What You Control
Ultimately, you control only part of the value stream. This maxim holds true wherever you apply Agile, and software developers learn this principle right away. You can be the most disciplined developer imaginable, never checking in code that doesn't work, always building in tests, and still be at the mercy of your own QA team. And even if the larger team, including testers, works in blissful harmony, someone needs to deploy the code on a live system. (Which is why dev ops is receiving a lot of attention from within the Agile community these days. See Jez Humble's recent book for a good example.)

If a stream is an appropriate metaphor for value creation and delivery, then this stream always takes many twists and turns, frequently encountering dams and obstructions. The success of Agile, therefore, depends to a great extent on eliminating these kinks, navigating around them, or learning to accept them as part of the value stream's trajectory. 

Research experiences its own kinks in the value stream. For example, if you're doing interviews, as we did for the thought leadership project, your velocity is never going to exceed the pace at which you schedule the interviews. Similarly, survey-based research can't proceed until you get an acceptable number of responses. Once we've written the findings, we depend on our crack editing team to ensure the quality of the final product. They're juggling a pipeline overflowing with documents submitted at the end of the quarter, so I have to be realistic about how quickly the overall Forrester value stream is going to move. 

Next time I apply this methodology to research, I'll do a better job of anticipating the time needed to schedule the interviews. If I can, I won't time experimental research projects to conclude at the end of the quarter. Still, there will be completely unpredictable complications, such as my laptop's repeated hardware failures during a critical phase of this project. (Unfortunately, my laptop was a single point of failure for my own productivity.)

Agile Is A Collaborative Process
In earlier blog posts, I described how our miniature Agile team, consisting of myself and research associate Eric Hsieh, divided up our official roles and responsibilities. Theoretically, I could have done all the work myself, but I knew it would be a grave mistake.

Everything about Agile -- the short iterations, the emphasis on product over documentation, the opportunity to be more adaptable, etc. -- depends on collaboration with other people. For example, I could set short time horizons for myself, but I'm more likely to stick with them if I'm working side-by-side with someone who shares my interest in cleaving to these deadlines and can help remove some of the obstacles. Whether or not you adopt the most collaborative aspects of Agile, such as pair programming, collaboration is always a vital component of team productivity, frequently in less than obvious ways.

Writing, Like Coding, Is A Process With Some Unforeseeable Results
Before writing the document, I published an outline in the Forrester community and invited comments. I put a great deal of thought into the outline – perhaps a little more than average, because it was going to be visible to a larger audience. Therefore, the outline reflected my genuine intent for what the document would contain, just as requirements and specifications are supposed to define the contours of the product you build.

Our brains don't work the same way when outlining a document versus writing it. Both companies, Wipro and salesforce, stressed the importance of the Web site as the vehicle for communicating thought leadership. Compared to what? I wondered, as I was starting to write that section of the document. I paused to take another look at our data on sources of influence behind technology adoption, this from a slightly different perspective. I'm glad I did, because it became immediately obvious that the Web site is, in fact, very important, but only at very specific points in time. Therefore, to achieve one of the goals of the research, explaining how thought leadership depends on communication and perception, I had to go "off spec" slightly to add content that I didn't include in the original outline. 

Do You Want The Voice Of The Customer, Or A Greek Chorus?
This project was the first occasion for including our community in the research process from start to finish. (Heck, we didn't even have a community site until a few months ago.) Therefore, we were not 100% sure how much participation to expect, or which opportunities for input might be most enticing. Whatever form the community's participation took, we knew it wouldn't be the same from start to finish. 

That assumption turned out to be correct. Many people commented on the development document, which detailed our approach for this project. No one commented on the list of questions we'd ask during the primary research. Should we be disappointed?

No, because not everything we do is intrinsically interesting, inspiring a Greek chorus of community members to comment on it. As I learned during some offline discussions, some community members thought the questions looked just fine. One person admitted that the value of reviewing the interview guide wasn't clear, especially compared to the work involved.

But that's OK. Our own research into social media reminds us, practically every day, that participation depends on incentives. As long as the participation we did receive helped us achieve the goals of the project, I'm not going to stress over the participation we didn't get. Transparency is better than mystery, as I discussed in an earlier post. Providing an opportunity to influence our research increases the value of the result, since we can incorporate our community's feedback as we go. Transparency, therefore, is often a precondition of adaptability, which is a prime reason to adopt Agile in the first place.