Tom Grant serves Application Development & Delivery Professionals. See the full Analyst bio. Visit Forrester.com to learn how we make Application Development & Delivery Professionals successful every day.
Fiction writers I've met have said that the hardest section of a novel to write is not the beginning or ending but everything that happens in between. The middle chapters trace the course of the protagonist's struggle in way that must be both engaging and credible. The story of how people adopt Agile successfully also has a beginning, middle, and end. The middle part here, too, poses some of the most difficult challenges. The first chapter is a grabber, with teams energetically and fervently doing daily stand-ups, blazing through sprints, christening a product owner, prioritizing their backlogs, and living through all the other exciting events that happen at the very beginning.
And then the plot takes a different turn. Success at the small team level is fantastic, but how do you fit into a development organization? What if you need to work with an offshore team? How do you maintain velocity when builds take several hours or maybe even a full day? Is it possible to deal with compliance requirements without a significant amount of automation? How do you work better with the ops team so that the speed of deployment matches the speed of development?
Since Agile went mainstream, the number of teams reaching the difficult middle chapters of Agile adoption has increased markedly. Both I and my colleague Dave West answer questions about the middle phases every day. Many of these questions also arise during the yearly conference that the Agile Alliance holds in the US. (This year, it's in Salt Lake City to mark the tenth anniversary of the signing of the Agile Manifesto in nearby Snowbird.)
In my previous post about Agile practices and compliance requirements, I described the first of two big surprises encountered while doing the research. Compliance, as it turns out, is not quite as high a barrier to Agile as we thought. The second surprise has to do with the approach teams have developed to getting over or around that wall.
Leaving Scrum, Sarbanes-Oxley, and related concerns aside for the moment, a hot topic these days in app dev circles is product-oriented development. While teams in IT departments might have different motives than ISVs, systems engineers, or people in other situations, they're all interested in roughly the same thing. What it takes to qualify as a product may not be altogether clear, and there may be no definitive way of measuring whether your team's thinking and behaviors have shifted from project-centric to product-centric. As rough-hewn as the concept of product-oriented development might be, it's still an attractive destination for people coming at it from multiple directions. (Not coincidentally, this is the topic of a soon-to-be-published doc.)
In an unexpected way, many of the app dev teams that have been most successful at dealing with compliance are, as it turns out, acolytes of the product-oriented approach. They may not realize it, as their work output may not be any more productized than it was before. Instead, compliance is what turns into a product.
Now that Agile has moved into the mainstream, it is encountering a whole new raft of challenges, including compliance. The word on the street for at least the past couple of years is that trying to be Agile and satisfy regulatory requirements is a lot like juggling chainsaws and machetes: theoretically possible but certainly not advised.
Fortunately, the word on the street is nearly always wrong. When I started interviewing people who had made Agile succeed in highly regulated environments, I expected to hear a lot of handy best practices that I could synthesize into a research document — essentially, a tactical guide to compliance. If you're a medical device company and you need to document six ways from Sunday how you validated and verified the software embedded in a new device, here's what you might do. If you need to deal with the auditors, here's where an investment in an application life-cycle management (ALM) tool might help.
Although this type of research depends on interviews, it's worth taking a peek at the available survey data to see if it has any additional insights. And boy howdy, am I glad I did. Sifting through the data collected in the survey that Forrester did in conjunction with Dr. Dobb's Journal, I found the first of two big surprises about Agile and compliance:
Agile adoption in the most regulated industries is not significantly different from the adoption rate everywhere else.
It feels as though the word "value" has appeared in more discussions about software development and delivery than in the previous two decades. We see this increased demand for immediate, tangible value across the entire range of technology producers and consumers. The dubious value of legacy applications, which have grown like kudzu, is the impetus for many painfully difficult cutting and pruning jobs within IT departments. Faster realization of value is driving more applications and infrastructure into the cloud. Software vendors are realizing that, while revenue is vital, the long-term relationship with the customer depends on the mutual value that both parties think they're getting from the relationship.
If we measure software by value, instead of cost, revenue, completeness, or other possible measures, we have to measure the software development process in a complementary way. What characteristic of software development is most likely to generate a valuable result? If your answer is "speed," think again. Predictability is a much better measure.
At the IBM Innovate conference last month, Walker Royce made a very plausible case for valuing predictability over velocity. Here's his keynote address, which is definitely worth watching.
Practically everyone who visits the Vatican stops to take a picture of the Swiss Guards. Ditto for the Queen's Guard at Windsor Castle, the Royal Life Guards at Rosenborg Castle in Copenhagen, and the Evzones at Greece's Tomb of the Unknown Soldier. Those multicolored uniforms may not have a place on the modern battlefield, where camouflage is far more important than panache, but they do attract the tourists.
If, by this measure, these ceremonial units have some value (albeit none militarily), why not have more of them? You could post the newly created Sartorial Guard at tourist locations that haven't been attracting enough foot traffic lately. And who knows, they might even attract more recruits into the real military. (Though I'm not sure what the career path is once you've held the rank of Feldweibel in the Swiss Guards.)
Obviously, I'm not being serious. Once you start manufacturing new ceremonial units, you cheapen the brand. You don't need more than one as a "loss leader" in the military, and there's no need to get the people who actually fight up in arms. Figuratively, that is.
Here's why managing a portfolio is critical for managing products. It wouldn't be hard to find some enterprising "champion" for a new Swiss Guards-ish unit who was willing to sew the uniforms and stand around looking fierce. (We call them re-enactors, and we don't put them on the public payroll.) No matter how much attention they attract, they'll still be a failure from a national perspective.
I'm a dedicated podcast listener, and one of my current favorites is the BBC's In Our Time. The host, Melvyn Bragg, selects a bewildering array of topics, such as Daoism, the Battle of Bannockburn, random numbers, the medieval university, and metaphor. A recent episode about the Industrial Revolution unexpectedly and unintentionally turned into a very lively discussion about the sources of invention, a topic that's near and dear to application development and delivery professionals.
Here's the punch line to this discussion: Not everyone's brain is ready to conjure up new ideas, so you need a catalyst. And here's the connection to this blog: serious games can be that catalyst. Our brains regularly need to be shaken up this way, during both those magisterial moments when we're trying to look over the horizon and the more desperate moments when, as I discuss in a new study, we need to dig ourselves out of a hole.
Last year, colleague Mary Gerush and I wrote overviews of the requirements tools market, noting how it was segmenting to address different problems. (Click here and here for the two studies.) Application lifecycle management (ALM) tools may be undergoing the same evolution, forcing us to accept either a much larger definition of ALM, or several specializations within ALM.
In the requirements market, new tools, or new emphases among tools, were a sure sign that some kind of change was afoot. Several years ago, visualization tools were not only absent from the list of requirements tool capabilities, they were almost completely unknown. Now, any list of requirements capabilities that omits visualization would be incomplete. Several years ago, requirements management was the emphasis of these tools. Now, much of the interesting innovation is happening in requirements collection.
During my research for the just-published document "For Developers, Dog Food And Champagne Can't Be The Only Items On The Menu," I had an interesting conversation with the team at Adobe that handles internal pilots, which in their case entails more than just putting the next version of an Adobe product on the network for people to try. Instead of the typical "spaghetti against the wall" approach to "eating your own dog food" (to mix food metaphors), the Adobe team actively looks for use cases that fit the product. If a product like Flex or Photoshop is a tool, then it should be the right tool for some job. (And if you can't find any use for the software, you're definitely in trouble.)
This approach might require additional work above the "spaghetti against the wall" approach, but it definitely has dividends for many different groups. The product team identifies functionality gaps or usability flaws. Marketers and salespeople have a much easier time figuring out what to demo. As a result, Adobe runs a better chance of both building technology that's compelling, and then explaining what's compelling about it to potential customers.
Recently, I spoke with a major airline about their adoption of Agile, which they considered critical for a major customer loyalty project. Based on previous experience, the dev team expected the business users involved in this project to move slowly, so they saw Agile as a strategy for being ready to pounce on any opportunity to make progress. How slowly? The current estimation for the project's completion was....[drumroll]...five years. Now that's a customer loyalty program ensured to be left with just the most loyal customers imaginable.
As hair-raising a situation as this might be, it's hardly unique. App dev teams contributing embedded software elements to hardware products must time their deliverables to arrive at key landmarks in the overall release schedule. Compliance requirements weigh down software development with extra documentation and validation. Flawed requirements force teams to go back to the drawing board. Dev managers live and die by the schedule, and there's always something that could jeopardize the schedule. Development is pretty pointless unless someone delivers the bits and bytes, but dev ops still remains a relatively mysterious and unpredictable process for dev teams, over which they have little control once they throw their code over the wall.
A big part of my research agenda for this year is productization. Many app dev teams see productization as a way to innovate better, achieve more sustainable results at a lower cost, deal with some of the tough challenges downstream, and in general lead a happier and more productive life. The allure of productization varies across different types of organization, as do the approaches. Therefore, to do the product justice, we're going to look at five different settings in which app dev teams are striving to productize their work:
Companies that have customer-facing applications on their websites. The classic example is online banking, but there are plenty of others. Some of these applications are tools for existing customers, while others are mechanisms for interactive marketing.
IT departments. In this case, productization is a way to improve the long-term return on technology investments, by treating them as products and assigning them a product owner.
Services companies. Productization may reduce inefficiencies in developing and delivering offerings, as well as marketing and selling them.
Embedded software. The ubiquity of software as a component of a larger product (car, medical device, etc.) creates new challenges in defining what the product is, and where software development fits into it. That's one reason why PTC, a product lifecycle management (PLM) vendor, was interested in acquiring MKS. (Other than their shared interest in TLAs.)
Fiction writers I've met have said that the hardest section of a novel to write is not the beginning or ending but everything that happens in between. The middle chapters trace the course of the protagonist's struggle in way that must be both engaging and credible. The story of how people adopt Agile successfully also has a beginning, middle, and end. The middle part here, too, poses some of the most difficult challenges. The first chapter is a grabber, with teams energetically and fervently doing daily stand-ups, blazing through sprints, christening a product owner, prioritizing their backlogs, and living through all the other exciting events that happen at the very beginning.
In my previous post about Agile practices and compliance requirements
Practically everyone who visits the Vatican stops to take a picture of the Swiss Guards. Ditto for the Queen's Guard at Windsor Castle, the Royal Life Guards at Rosenborg Castle in Copenhagen, and the Evzones at Greece's Tomb of the Unknown Soldier. Those multicolored uniforms may not have a place on the modern battlefield, where camouflage is far more important than panache, but they do attract the tourists.
I'm a dedicated podcast listener, and one of my current favorites is the BBC's In Our Time. The host, Melvyn Bragg, selects a bewildering array of topics, such as Daoism, the Battle of Bannockburn, random numbers, the medieval university, and metaphor. 
Last year, colleague Mary Gerush and I wrote overviews of the requirements tools market, noting how it was segmenting to address different problems. (Click 
During my research for the just-published document 
[For the first part in this series, 
A big part of my research agenda for this year is productization. Many app dev teams see productization as a way to innovate better, achieve more sustainable results at a lower cost, deal with some of the tough challenges downstream, and in general lead a happier and more productive life. The allure of productization varies across different types of organization, as do the approaches. Therefore, to do the product justice, we're going to look at five different settings in which app dev teams are striving to productize their work: