Early this year, on January 15, I published our first research on testing for the Agile and Lean playbook. Connected to that research, my colleague Margo Visitacion and I also published a self-assessment testing toolkit. The toolkit helps app-dev and testing leaders understand how mature their current testing practices and organization are for Agile and Lean development.
The Agile Testing Self-Assessment Toolkit
So what are the necessary elements to assess Agile testing maturity? Looking to compromise between simplicity and comprehensiveness, we focused on the following:
Testing team behavior. Some of the questions we ask here look at collaboration around testing among all roles in the Scrum teams. We also ask about unit testing: Is it a mandatory task for developers? Are all of the repeititive tests that can be run over and over at each regression testing automated?
Organization. In our earlier Agile testing research, we noticed a change in the way testing gets organized when Agile is being adopted. So here we look at the role test managers are playing: Are they focusing more on being coaches and change agents to accelerate adoption of the new Agile testing practices? Or are managers still operating in a command-and-control regime? Is the number of manual testers decreasing? Are testing centers of excellence (TCOEs) shifting to become testing practice centers of excellence (TPCOEs)?
DevOps is a movement for developers and operations professionals that encourages more collaboration and release automation. Why? To keep up with the faster application delivery pace of Agile. In fact, with Agile, as development teams deliver faster and in shorter cycles, IT operations finds itself unprepared to keep up with the new pace. For operations teams, managing a continuous stream of software delivery with traditional manual-based processes is Mission Impossible. Vendors have responded to DevOps requirements with more automation in their release management, delivery, and deployment tools. However, there is a key process that sits between development and operations that seems to have been given little attention: testing.
In fact, some key testing activities, like integration testing and end-to-end performance testing, are caught right in the middle of the handover process between development and operations. In the Agile and Lean playbook, I’ve dedicated my latest research precisely to Agile testing, because I’ve seen testing as the black beast in many transformations to Agile because it was initially ignored.
One-Size Software Development Methodologies Do Not Fit All
Dozens of software development methodologies exist, from waterfall to Agile to pure anarchy (Agile has always rubbed me wrong). Mark Kennaley speaks the truth when he says that “there is no ‘best’; there’s only contextual fitness for purpose.” Mark is the founder of Software Development Experts, a software development methodology historian, a consultant, and the creator of an expert system that helps organizations determine the best software methodology to use based on 10 factors: development team size, domain complexity, technical complexity, the geographical dispersion of the development team, time-to-market pressure, enterprise specialization, contract relationships, compliance, criticality, and culture. This makes perfect sense, and so does Mark. Unfortunately, entrenched dogma and high ceremony can obscure what really matters.
Composite, Dynamic Software Development Methodologies Are Best
TechnoPolitics speaks with Mark about how firms can choose the best methodology based on the 10 factors that matter. One size does not fit all. Listen to find out why and how to move forward.
Podcast: One-Size Software Development Methodologies Do Not Fit All
Agile and Lean transformations depend to a great extent on cultivating a good sourcing ecosystem. The decisions you make around the partners and providers supporting your transformation and projects will be at the core of a successful strategy. But sourcing strategy needs to go beyond just resource or services providers (read outsourcing) and address a larger ecosystem made of Agile SW development and delivery choices, collaboration and communication capabilities for distributed teams, and teams' physical work spaces, standard equipment, and office layout.
In September, I published a report on how to source your Agile strategy, that describes what the ecosystem looks like and how to navigate it effectively, the document is part of our larger research container on Agile - The Agile and Lean Playbook. The report gives an overview on how large vendors, SIs, and medium to small consulting organizations can (not) help you with your Agile journey but also what you need to do to be successful. Here are some of the takeaways from the research:
What you think about Agile and Lean might not be what your SI thinks. You need to take control of your own destiny with Agile and Lean. Change your application development and delivery sourcing strategy to embed the best talents around the world to help you make it happen. But be careful with the traditional SIs, because Agile is as disruptive to them as it is you, and if they have not been seriously transforming themselves, it will be hard for them to deliver Agile services to you. Some good alternative new fully Agile players exist, including highly specialized external consulting firms. You might want to start testing the ground with these options.
I just finished analyzing our Q3 2012 Global State Of Enterprise Architecture Online Survey, where we asked a number of questions at the end of the survey on how firms identify and introduce new technology – new technology that your firm is counting on for innovation and competitive advantage. The results underscore a conviction that is growing in me: IT’s “one-size-fits-all” approach to standardizing everything and general aversion to risk isn't cutting the mustard. Simply put, opportunities for competitive advantage through technology-fueled disruption get missed, and this means digital extinction. Some data from our survey of 207 enterprise architects:
58% reported that sales and marketing is among the top five most likely organizations to deliver technology innovations, and they are chasing windows of opportunity that close in months. IT typically takes at least a year to do anything.
52% say there is at least some business dissatisfaction with the level of new technology introduction. The top reason, given by 78% of respondents, is that IT is too slow.
70% of respondents admit their firms have trouble reacting to disruptions caused by emerging technology, and 60% admit to difficulty reacting to change in general.
Last December, I published three things I'd tell your CIO. Since then, I've spent time with dozens and dozens of sourcing and vendor management professionals, CIOs, and leaders of application development and delivery, including last week's Paris Forrester Forums. Most days, I share our ongoing research on what impact today's software-fueled, consumer-led digital disruption has on your ability to meet and exceed the expectations of your customers and the employees serving them. For some folks, software and software development remains a commodity. But for many, the need to deliver great software has taken hold of 2013 planning discussions. With July just around the corner, and as you start 2013 planning, focus on what you need to start delivering great software (remember, software is your business), and keep these three things I'd tell you and your CIO in mind:
Just over 3 months ago, I made note of three things I'd tell your CIO, all of which focused on your need to build a software development competency to help your firm thrive in this age of software-fueled, consumer-led disruption. Since then, we've heard from a number of clients stating that they're having a tough time convincing their executives, from COOs and CFOs through to CIOs, that they need to stop looking at software and app development as a commodity.
Vendors you work with aren't helping. System integrators and consultancies continue to tell your CFO and CEO to outsource your software development work to them, that they can deliver more quickly, and more cheaply, than you can. Software application vendors build their marketing around needing no customization, even "no software." This helps fuel the perception and myths many executives hold that software development, especially app dev, is a commodity.
Recent research published by Phil Murphy and survey data we recently collected in our Forrsights Software Survey, Q4 2011 can help you bust those perceptions and myths and help you show your executives the importance of software development.
Think of a medieval fortress: It was originally used for a small army, it has walls nine meters thick, and it’s surrounded by buildings hundreds of years old. Upon entering, you are confronted with the concept of eternity.
This fortress is located in the smallest state on earth — though it is also perhaps the best-known state in the world. The business housed within the fortress is what many might classify as a SME but with with complexity of a large enterprise, holy but busy, centralized but truly global — its work spans hundreds of countries with hundreds of currencies and hundreds of languages — and it serves very special and demanding clients.
Have a clue yet of where we are?
Zoom on Italy, then zoom on Rome, then zoom on Vatican City, and you can’t miss the round tower (Torrione Sisto V) where the Vatican Bank, or Istituto per le Opere di Religione (IOR ), is located. You won’t be allowed in if you are not a client, an employee, or part of a religious congregation. Change comes hard to institutions this steeped in tradition. To give you a clue, IOR’s previous managing director spent his entire career at IOR — 60 years — and retired at the age of 80. We all know it’s the soft and cultural aspects of transformation that are the hardest part for any organization.
Nevertheless, IOR has been going through a major change since 2008, working to replace its legacy IT system with a modern BT one. The new BT system brings more flexibility for the business, richer business functionality, and greater integration and development capabilities. Enabling fast change is the key driver for IOR’s IT transformation program from IT into BT.
Ultimately, customers don't judge you based on how well you gather business requirements, choose development technologies, manage projects, or march through the development process — they judge you based on how they feel before, during, and after they use your software. This is the digital experience. If you get the customer experience wrong, then nothing else matters. And expectation inflation is sky-high thanks to the Apple-led smartphone revolution. To succeed in the new age of digital experience, application development professionals must collaborate with their business partners and customers to create experiences that customers love. You need a new approach represented by these five axioms:
Software is not code; it creates experience.
Development teams are not coders; they are experience creators.
Technical talent is table stakes; great developers must be design and domain experts.
Process is bankrupt without design; you get what you design, so you had better get the design right.
Software is a creative endeavor, not an industrial process like building automobiles. Structure your methodology to empower your creative talent.
My colleague and friend Mike Gualtieri wrote a really interesting blog the other day titled "Agile Software Is A Cop-Out; Here's What's Next." While I am not going to discuss the great conclusions and "next practices" of software (SW) development Mike suggests in that blog, I do want to focus on the assumption he makes about using working SW as a measurement of Agile.
I am currently researching that area and investigating how organizations actually measure the value of Agile SW development (business and IT value). And I am finding that, while organizations aim to deliver working SW, they also define value metrics to measure progress and much more:
Cycle time (e.g., from concept to production);
Business value (from number of times a feature is used by clients to impact on sales revenue, etc.);
Productivity metrics (such as burndown velocity, number of features deployed versus estimated); and last but not least
Quality metrics (such as defects per sprint/release, etc.).