Our bi-yearly Forrester Agile survey suggests that Agile development (or simply "Agile") continues to see consistent, strong adoption. However, the same survey data shows that only a small percentage of firms are outsourcing Agile application development due to a lack of experience with the development sourcing approaches and governance models needed to make it work. Successfully outsourcing Agile development, either fully or partially, involves redefining roles and responsibilities, change management processes, metrics and SLAs, service descriptions, and other contractual elements. Merely using traditional outsourcing language and practices risks jeopardizing the benefits of Agile. There is no single way of doing this right.
When computers were invented 60 years ago, nobody would have thought that gazillions of 0 and 1s would soon rule the world. After all, that’s all there is in any computer memory, be it a laptop, a mobile phone, or a supercomputer like Watson; if you could open memory up and visualize the smallest elementary unit, you would “see” only an infinite sequence of 0s and 1s, something that would look like this:
Interestingly, that has not changed. Computers are still processing 1s and 0s. What has changed is that we live in an age of digital disruption, an age where software applications run and rule our business more and more. To be successful, those applications need to be engaging and entertaining so that consumers enjoy and are delighted by them; they also have to be mobile and accessible anywhere and at anytime, and they have to leverage tons of information, no matter if it comes from a database, a tweet, or Facebook.
Within the modern applications era, regardless of whether new software applications are being developed and delivered for mobile, tablets, or the Web, the truly successful app-dev leaders will be those who focus on delivering constant value and incremental improvement to their business. That is a totally different perspective from “I need to keep control of my team’s productivity to make sure that we stick to our estimated costs, scope, and project dates.” Of course, the interest in cost is never going away, but app-dev leaders today have a great chance to enhance their conversation with executives and business stakeholders and add value to the conversation.
However, as the recent research I just published, Agile Metrics That Matter, proves, while some of the most advanced Agile teams do use new progress, quality, efficiency, and value/benefits metrics (these to a lesser degree), some software development industry luminaries have worked and are working on new methods to measure value in software development. But it’s still early days!
I’d like to summarize here some good old practices on establishing metrics that count together with some of the new findings of the research:
What a strange summer this has been! From Boston to London to Paris to Turin, the weather has offered weekly and even daily reversals, with continuous change from sun to rain, from hot and damp to cool and crisp. I missed a nice spring season. Even today, from 35º-38º Celsius (95º-100º Fahrenheit), we just went to 22º Celsius (71º Fahrenheit) with a perfect storm! A continuous climate and sudden change is quite unusual in some of these countries. Certainly it is where the Azores Anticyclone usually dominates from mid-late June to mid-late August, offering a stable summer. How many times have you had to change plans because you discover weather is about to change!?
You might be thinking, "What does this have to do with this AD&D blog?" It’s about change! I am wondering if, in our daily lives, getting used to unexpected conditions and having to handle continuous change favors a mindset where change is just something we have to deal with and not fight. A new mindset very much needed given the change we see ahead in how we develop, test, and deploy software!
My focus in this blog is testing, although the first change we need to get used to is that we can’t talk any longer about testing in an isolated fashion! Testing is getting more and more interconnected in a continuous feedback loop with development and deployment. (See my colleague Kurt Bittner's report on continuous delivery; I could not agree more with what Kurt says there!)
I just finished my new report on the Agile testing tools landscape. I’ll point Forrester readers to it as soon as it publishes. But there are few things that have struck me since I took over the software quality and testing research coverage at Forrester and which I would like to share with you in this preview of my findings of the testing tools landscape doc.
My research focus area was initially on software development life cycles (SDLCs) with a main focus on Agile and Lean. In fact, my main contribution in the past 12 months has been to the Forrester Agile and Lean playbook, where all my testing research has also focused. Among other reasons, I took the testing research area because testing was becoming more and more a discipline for software developers. So it all made sense for me to extend my software development research focus with testing. But I was not sure how deep testing was really going to integrate with development. My concern was that I’d have to spend too much time on the traditional testing standards, processes, and practices and little on new and more advanced development and testing practices. After 12 months, I am happy to say that it was the right bet! My published recent research shows the shift testing is making, and so does the testing tool landscape document, and here is why:
In theory, the agile Product Owner is a simple yet compelling solution to a tough problem: the development team needs, and often does not get, clear direction from the business. The ability to eliminate the confusion caused by the cacophony of voices of multiple stakeholders, and the ability to have continuous engagement with the business, certainly make the Product Owner attractive to the development team. And the business benefits, too: they, in theory, get continuous visibility into project health and status. Buried in that last sentence is the phrase that often sinks good ideas: "in theory", and therein lies a problem.
When we are developing software and find components that have responsibilities too broad for one component to encompass, we "refactor" it, breaking it down into a set of components with more manageable and cohesive sets of responsibilities. We have an analogous problem with the Product Owner, whose responsibilities are so broad as to be nearly impossible for one person to fulfill. In brief, the Product Owner is expected to:
Understand the needs and desired outcomes of the business
Negotiate consensus among stakeholders
Represent the interests of all stakeholders to the development team
Define the characteristics of solutions that meet the desired outcomes
Be a change agent in the organization to support the solution
Communicate and promote the vision to all interested parties
Define and prioritize items on the Product Backlog
Early this year, on January 15, I published our first research on testing for the Agile and Lean playbook. Connected to that research, my colleague Margo Visitacion and I also published a self-assessment testing toolkit. The toolkit helps app-dev and testing leaders understand how mature their current testing practices and organization are for Agile and Lean development.
The Agile Testing Self-Assessment Toolkit
So what are the necessary elements to assess Agile testing maturity? Looking to compromise between simplicity and comprehensiveness, we focused on the following:
Testing team behavior. Some of the questions we ask here look at collaboration around testing among all roles in the Scrum teams. We also ask about unit testing: Is it a mandatory task for developers? Are all of the repeititive tests that can be run over and over at each regression testing automated?
Organization. In our earlier Agile testing research, we noticed a change in the way testing gets organized when Agile is being adopted. So here we look at the role test managers are playing: Are they focusing more on being coaches and change agents to accelerate adoption of the new Agile testing practices? Or are managers still operating in a command-and-control regime? Is the number of manual testers decreasing? Are testing centers of excellence (TCOEs) shifting to become testing practice centers of excellence (TPCOEs)?
DevOps is a movement for developers and operations professionals that encourages more collaboration and release automation. Why? To keep up with the faster application delivery pace of Agile. In fact, with Agile, as development teams deliver faster and in shorter cycles, IT operations finds itself unprepared to keep up with the new pace. For operations teams, managing a continuous stream of software delivery with traditional manual-based processes is Mission Impossible. Vendors have responded to DevOps requirements with more automation in their release management, delivery, and deployment tools. However, there is a key process that sits between development and operations that seems to have been given little attention: testing.
In fact, some key testing activities, like integration testing and end-to-end performance testing, are caught right in the middle of the handover process between development and operations. In the Agile and Lean playbook, I’ve dedicated my latest research precisely to Agile testing, because I’ve seen testing as the black beast in many transformations to Agile because it was initially ignored.
There is no doubt that Agile growth in the market is significant, and the growing daily number of inquiries I’ve been getting on Agile from end user organizations in 2012 gives me the impression that many are moving from tactical to strategic adoption. Why’s that? Many reasons, and you can read about them in our focused research on Agile transformation on the Forrester website. But I’d like to summarize the top five reasons from my recent research “Determine The Business And IT Impact Of Agile Development” :
Quality was the top — quite astonishing, but both the survey we ran across 205 Agile “professional adopters” and the interviews across some 21 organizations confirmed this. My read is that this is about functional quality.
Change was second to quality. We live in an era where innovation strives and organizations are continuously developing new apps and projects. But your business does not necessarily know what it needs or wants upfront. The business really appreciates the due-course changes that Agile development allows, as they enable the business to experiment and try out various options so it can become more confident about what is really right for the organization. Cutting edge cutting edge systems-of-engagement (Mobile, Web-facing, Social-media, etc) require lots of Change in due course.
Think of a medieval fortress: It was originally used for a small army, it has walls nine meters thick, and it’s surrounded by buildings hundreds of years old. Upon entering, you are confronted with the concept of eternity.
This fortress is located in the smallest state on earth — though it is also perhaps the best-known state in the world. The business housed within the fortress is what many might classify as a SME but with with complexity of a large enterprise, holy but busy, centralized but truly global — its work spans hundreds of countries with hundreds of currencies and hundreds of languages — and it serves very special and demanding clients.
Have a clue yet of where we are?
Zoom on Italy, then zoom on Rome, then zoom on Vatican City, and you can’t miss the round tower (Torrione Sisto V) where the Vatican Bank, or Istituto per le Opere di Religione (IOR ), is located. You won’t be allowed in if you are not a client, an employee, or part of a religious congregation. Change comes hard to institutions this steeped in tradition. To give you a clue, IOR’s previous managing director spent his entire career at IOR — 60 years — and retired at the age of 80. We all know it’s the soft and cultural aspects of transformation that are the hardest part for any organization.
Nevertheless, IOR has been going through a major change since 2008, working to replace its legacy IT system with a modern BT one. The new BT system brings more flexibility for the business, richer business functionality, and greater integration and development capabilities. Enabling fast change is the key driver for IOR’s IT transformation program from IT into BT.