Agile software development practices have been transforming AD organizations for more than a decade. With more rapid development cycles has come a bottleneck at the deployment boundary - at the frontier between Development and Operations. The DevOps movement is working to remove this bottleneck, and in the process is transforming both Dev and Ops for the better. In many respects it is a logical evolution of the agile movement, but practices like continuous deployment are deeply transformative of the way that organizations think about customer engagement, business engagement, testing, development and requirements - in fact, nearly every aspect of agile development is subtly but powerfully affected. The implication of a check-in resulting in code being deployed to production gives a whole new emphasis to the word "commit"!
I didn't get the chance to jot down my thoughts after a couple of days at IBM Pulse last week but I didn't want to not share my observations and thoughts. So here we go as I fly off to itSMF Norway's annual conference ... It's somewhat random but what did you expect from me? A Katy Perry inspired title?
My view of the IBM Pulse keynotes …
The IBM keynotes covered many of the things you would expect (see my pics below) such as: big data, cloud, mobile, smart-things, and big data. And did I mention big data? It's a key challenge/opportunity for IBM and its customers.
What really resonated with me during the keynotes, however, was not big data but the use of a certain lexicon – with words like "value," "customer-centricity," business outcomes," and even "Outside-In." It was my first proper IBM Pulse so I was unsure whether this was the norm or whether IBM has started "thinking outside the data center" – a criticism I have previously used with other vendors.
Given IBM's traditional focus on enterprise-spanning deals and business, rather than IT challenges/opportunities, it's probably the former but IMO a key part of helping enterprise IT organizations support their customers is IT service management (ITSM). And IBM despite having a fit for purpose ITSM offering and probably thousands of ITSM "experts" throughout its organization has just not been in people's minds and ITSM conversations the last two years.
IBM markets at the enterprise level and this means many potential customers don't think “IBM” and then think “ITSM” (or the reverse) as they would with other ITSM tool vendors. It might seem a harsh thing to say but I believe it to be the reality. I think this might be about to change though – I'll come back to this after a quick detour.
DevOps is a movement for developers and operations professionals that encourages more collaboration and release automation. Why? To keep up with the faster application delivery pace of Agile. In fact, with Agile, as development teams deliver faster and in shorter cycles, IT operations finds itself unprepared to keep up with the new pace. For operations teams, managing a continuous stream of software delivery with traditional manual-based processes is Mission Impossible. Vendors have responded to DevOps requirements with more automation in their release management, delivery, and deployment tools. However, there is a key process that sits between development and operations that seems to have been given little attention: testing.
In fact, some key testing activities, like integration testing and end-to-end performance testing, are caught right in the middle of the handover process between development and operations. In the Agile and Lean playbook, I’ve dedicated my latest research precisely to Agile testing, because I’ve seen testing as the black beast in many transformations to Agile because it was initially ignored.
We've seen a number of misconceptions about Agile come and go. For example, the urban myth that Agile is all about velocity gets far less circulation. More people have seen Agile in practice, witnessing first hand the other potential benefits (more chances for mid-course corrections, greater predictability of outcomes, better business/IT alignment, etc.) than just writing code faster. More people are starting projects with these potential benefits in mind, so Agile has clearly moved past the perception that it was some perverse cult of speed. Speed has no intrinsic business value, aside from keeping a twitchy software developer who has consumed way too much Mountain Dew from chewing his own foot off in frustration about delays and obstacles.
Agile Gets To Specifics Quickly
Business value is the goal for Agile transformation; benefits like quality, predictability, and business/IT alignment are either measures of that value or steps needed to achieve that value. Both topics require a lot of clarity or specificity. Otherwise, Agile can look a lot like a road trip gone horribly wrong: we're not sure where we're going, how close we are, or whether we're going the right way at all.
Agile succeeded brilliantly because it started with some very specific practices and values. Don't end a sprint without working code. Keep your backlog prioritized. Build the expectation of new or changing requirements into planning. Don't build your plans on information you don't have. That's a lot more-specific guidance than something like"Achieve this maturity level and you'll be fine" or "Follow this KPI to the ends of the earth."
OK ITIL, the IT service management (ITSM) best practice framework, is neither a cult nor a religion, but hopefully I grabbed your attention.
The point of this short, but hopefully interesting, blog is that when we usually think about ITIL we normally focus on the IT service management and IT operations organizational domains as its playground. And, while we appreciate that other IT roles might have an interest in ITIL (especially enterprise architects or those looking at DevOps), I imagine most will be surprised at the following percentage readership demographic for my recent “Adopting ITIL” report.
Percentage split of Adopting ITIL readers across Forrester “role types”
As many of you know, Forrester conducted a joint research study earlier this year, in conjunction with the US chapter of the IT Service Management Forum (itSMF-USA). The report is finally now available to the deserving. Forrester clients can download it using the normal access methods. Members of itSMF-USA will receive their copy from itSMF-USA. If you contributed, but do not fall into either category, Forrester will be sending you your copy.
You can read a few of the finding in my original post announcing the completion of the study. An example of the findings is the level of satisfaction with service desk solutions. While satisfaction in general is higher than one would think, a SaaS model has proven especially satisfactory:
Please let me know if you are having difficulty obtaining your report. Thank you again for all the participation that led us to these findings! We look forward to next year’s study!
If you've been reading the research I've been writing over the past year, you know that I'm a fan of implementing an application life-cycle management strategy that focuses on increasing development flow and supports high-performance teams. You don't need to religiously implement all 22 CMMI processes or deliver dozens of intermediate development artifacts like some leading processes advocate. Rather, there are certain important processes that you should spend your time on. We wrote about change-aware continuous integration and just-in-time demand management in last year's Agile Development Management Tools Forrester Wave™. They are two of my favorite areas of focus, and they are great areas to invest in, but once you have them working well, there are other areas that will require your focus. In my opinion, the next process where you should focus on flow is everything that happens post build and preproduction. Most folks think about this process as release management or configuration management, but I think there's a better term that focuses on how quickly software changes move through both processes. It's called continuous delivery. When you focus on establishing a process of continuous delivery, you'll find that your capacity to release changes will increase, your null release cycle will shrink, and a larger proportion of the productivity gains you've seen from your Agile development efforts will flow through into production.
Complexity is the nemesis of application developers everywhere. While software products' complexity may be, to a great extent, unavoidable, we don't have to complicate software development to match it.
That's the conclusion driving many of the new approaches that app dev teams are embracing. For example, Agile breaks down humongous, complex projects into incremental deliverables that are far easier to scope, build, test, and deliver. Lean practices simplify processes, making it easier for teams to achieve a state of flow. DevOps advocates are trying to eliminate the needless complexities that result from throwing software over the wall from one team to the next.
But there's more to Agile, Lean, and DevOps than just a common enemy. It's no accident that practitioners often speak of this trio in practically the same breath. The Kanban board is the symbol of this convergence of practices. Here's an increasingly common use case: an Agile teams uses a Kanban board to manage work with other groups, including the DevOps team.
The evidence for this convergence is more than just anecdotal. When we survey app dev professionals, they report that they're mixing methodologies all the time, including Agile and Waterfall. For a variety of reasons, Agile teams have made the adjustments necessary to accommodate Waterfall. Not every planning activity at the beginning of a project can be handled in a strictly Agile fashion, and not everyone involved at the end of the project (release management, operations, and business users) is necessarily sold on the idea of rapid, continuous delivery. If you work in a regulated environment, you're required to take extra steps at the beginning and end of a project.
My colleague, Glenn O’Donnell, and I (do I sound like the Queen?) have delivered a Forrester report called “Improving The Ops In DevOps” inspired by the long-bemoaned tension between “change-the-business” (dev) and “run-the-business” (ops) IT teams and their activities, and the need for change.
This tension inflicts a detrimental impact on the business. In fact, most organizations suffer this curse, and stereotypes that reflect this animosity abound. Does this sound familiar? Ops people see dev people as sitting in their ivory towers cranking out code all day and wanting to release applications oblivious to real-world constraints; dev sees ops as cog-turners ensuring that the IT infrastructure doesn’t break under the strain of poorly written code. Chances are that your organization is not this bad. But this exaggeration is indicative of the tension between these two IT “tribes” and their opinions of each other. These stereotypes exist because organizational behaviors do exaggerate genuine conflicts, and both parties must act quickly to change.
Getting DevOps right will address many of the issues enterprises consistently have with IT, such as applications failing to meet both functional and nonfunctional requirements, delivery delays, increased costs, and an inflexibility to change. But is DevOps enough to save I&O from extinction?
It feels as though the word "value" has appeared in more discussions about software development and delivery than in the previous two decades. We see this increased demand for immediate, tangible value across the entire range of technology producers and consumers. The dubious value of legacy applications, which have grown like kudzu, is the impetus for many painfully difficult cutting and pruning jobs within IT departments. Faster realization of value is driving more applications and infrastructure into the cloud. Software vendors are realizing that, while revenue is vital, the long-term relationship with the customer depends on the mutual value that both parties think they're getting from the relationship.
If we measure software by value, instead of cost, revenue, completeness, or other possible measures, we have to measure the software development process in a complementary way. What characteristic of software development is most likely to generate a valuable result? If your answer is "speed," think again. Predictability is a much better measure.
At the IBM Innovate conference last month, Walker Royce made a very plausible case for valuing predictability over velocity. Here's his keynote address, which is definitely worth watching.