Early this year, on January 15, I published our first research on testing for the Agile and Lean playbook. Connected to that research, my colleague Margo Visitacion and I also published a self-assessment testing toolkit. The toolkit helps app-dev and testing leaders understand how mature their current testing practices and organization are for Agile and Lean development.
The Agile Testing Self-Assessment Toolkit
So what are the necessary elements to assess Agile testing maturity? Looking to compromise between simplicity and comprehensiveness, we focused on the following:
Testing team behavior. Some of the questions we ask here look at collaboration around testing among all roles in the Scrum teams. We also ask about unit testing: Is it a mandatory task for developers? Are all of the repeititive tests that can be run over and over at each regression testing automated?
Organization. In our earlier Agile testing research, we noticed a change in the way testing gets organized when Agile is being adopted. So here we look at the role test managers are playing: Are they focusing more on being coaches and change agents to accelerate adoption of the new Agile testing practices? Or are managers still operating in a command-and-control regime? Is the number of manual testers decreasing? Are testing centers of excellence (TCOEs) shifting to become testing practice centers of excellence (TPCOEs)?
DevOps is a movement for developers and operations professionals that encourages more collaboration and release automation. Why? To keep up with the faster application delivery pace of Agile. In fact, with Agile, as development teams deliver faster and in shorter cycles, IT operations finds itself unprepared to keep up with the new pace. For operations teams, managing a continuous stream of software delivery with traditional manual-based processes is Mission Impossible. Vendors have responded to DevOps requirements with more automation in their release management, delivery, and deployment tools. However, there is a key process that sits between development and operations that seems to have been given little attention: testing.
In fact, some key testing activities, like integration testing and end-to-end performance testing, are caught right in the middle of the handover process between development and operations. In the Agile and Lean playbook, I’ve dedicated my latest research precisely to Agile testing, because I’ve seen testing as the black beast in many transformations to Agile because it was initially ignored.
I just finished analyzing our Q3 2012 Global State Of Enterprise Architecture Online Survey, where we asked a number of questions at the end of the survey on how firms identify and introduce new technology – new technology that your firm is counting on for innovation and competitive advantage. The results underscore a conviction that is growing in me: IT’s “one-size-fits-all” approach to standardizing everything and general aversion to risk isn't cutting the mustard. Simply put, opportunities for competitive advantage through technology-fueled disruption get missed, and this means digital extinction. Some data from our survey of 207 enterprise architects:
58% reported that sales and marketing is among the top five most likely organizations to deliver technology innovations, and they are chasing windows of opportunity that close in months. IT typically takes at least a year to do anything.
52% say there is at least some business dissatisfaction with the level of new technology introduction. The top reason, given by 78% of respondents, is that IT is too slow.
70% of respondents admit their firms have trouble reacting to disruptions caused by emerging technology, and 60% admit to difficulty reacting to change in general.
There is no doubt that Agile growth in the market is significant, and the growing daily number of inquiries I’ve been getting on Agile from end user organizations in 2012 gives me the impression that many are moving from tactical to strategic adoption. Why’s that? Many reasons, and you can read about them in our focused research on Agile transformation on the Forrester website. But I’d like to summarize the top five reasons from my recent research “Determine The Business And IT Impact Of Agile Development” :
Quality was the top — quite astonishing, but both the survey we ran across 205 Agile “professional adopters” and the interviews across some 21 organizations confirmed this. My read is that this is about functional quality.
Change was second to quality. We live in an era where innovation strives and organizations are continuously developing new apps and projects. But your business does not necessarily know what it needs or wants upfront. The business really appreciates the due-course changes that Agile development allows, as they enable the business to experiment and try out various options so it can become more confident about what is really right for the organization. Cutting edge cutting edge systems-of-engagement (Mobile, Web-facing, Social-media, etc) require lots of Change in due course.
Think of a medieval fortress: It was originally used for a small army, it has walls nine meters thick, and it’s surrounded by buildings hundreds of years old. Upon entering, you are confronted with the concept of eternity.
This fortress is located in the smallest state on earth — though it is also perhaps the best-known state in the world. The business housed within the fortress is what many might classify as a SME but with with complexity of a large enterprise, holy but busy, centralized but truly global — its work spans hundreds of countries with hundreds of currencies and hundreds of languages — and it serves very special and demanding clients.
Have a clue yet of where we are?
Zoom on Italy, then zoom on Rome, then zoom on Vatican City, and you can’t miss the round tower (Torrione Sisto V) where the Vatican Bank, or Istituto per le Opere di Religione (IOR ), is located. You won’t be allowed in if you are not a client, an employee, or part of a religious congregation. Change comes hard to institutions this steeped in tradition. To give you a clue, IOR’s previous managing director spent his entire career at IOR — 60 years — and retired at the age of 80. We all know it’s the soft and cultural aspects of transformation that are the hardest part for any organization.
Nevertheless, IOR has been going through a major change since 2008, working to replace its legacy IT system with a modern BT one. The new BT system brings more flexibility for the business, richer business functionality, and greater integration and development capabilities. Enabling fast change is the key driver for IOR’s IT transformation program from IT into BT.
Two weeks have passed since our successful AD&D and BP Forums in Boston. I’m still struck by conversations we held there and continue to hold now with many of you on how your teams can help deliver to your firm’s ever-important customer experience outcomes. Following one tip can help you either get ahead of this issue or catch up to the expectations of your stakeholders…act more like an interactive agency!
Note I didn’t say “transform” into an interactive agency. No, at the end of the day you have responsibilities to your organization the agencies your business peers use often don’t – you have to manage, operate, and maintain what’s been delivered. What I did say was “act” like one, and in doing so you’ll need to:
Revisit your talent. For those of you that haven’t outsourced big portions of development, make sure you have great, creative developers, build a high-performance development team, and up-skill your business analysts by putting personas and customer journey maps into their tool kit. Why? The agencies your peers use have and cultivate these skills. At minimum, you'll be in a better position to manage and maintain what they’ve put in place if you have complementary skills of your own. If you have outsourced development, we can help you make the case to bring back the right pieces.
Development leaders! Project leaders and business analysts! Application and solution architects! Want to move forward on your business technology (BT) journey and be viewed by your business stakeholders as a valuable team member? Take a tip from last week's Forums held in Boston. Embrace Business Process Management (BPM) And Customer Experience. Don't ignore them, embrace them. Why? They're essential to helping you achieve your business outcomes.
I know, I know. You read the above and now think "Gee Kyle, what's next? Going to enlighten me on some new BPM or customer experience management technology that's going to transform my very existence, my company's future?"
Nope. Let me explain....
Last week we hosted more than 250 of your application development and delivery and business process peers in Boston and focused on how to succeed in the new world of customer engagement. The most impactful discussions I heard were the side conversations we held with attendees, sometimes occurring over dinner and cocktails. We didn't discuss technology. We discussed the skills your peers were developing in two fundamental areas:
BPM - no, not the technology but the Lean and Six Sigma based methods, techniques, and tools organizations use to focus on business processes and not functions; to strive for continuous improvement; and to focus on customer value.
Customer experience - defined more eloquently by my peer Harley Manning, but I'll summarize as the methods, techniques, and tools used to understand how customers perceive their interactions with your company.
Over the last few weeks, I have had a variety of conversations with clients that have centered around the scope for the term BPM. I think we all agree that BPM is not purely a technology – but how far does it go.
BPM – The Discipline
Forrester sees BPM as a broad framework of methods, approaches, techniques and technologies that support organizational change, value optimization and ongoing performance improvement. While some see BPM as a narrow technical approach, Forrester regards BPM as including a wide range of improvement methods such as Lean and Six Sigma, along with customer-centric (outside-in) engagement approaches and organizational change management – each one of these levers ties back to a flexible and adaptable enterprise architecture that implements an evolving business strategy. Such an all-encompassing approach can help focus on strategic priorities, as well as opportunities to both differentiate the value proposition, and sharpen the competitive edge.
While some would argue that Lean and Six Sigma are separate – that they are “in the business” – our research data suggests that the most successful BPM initiatives are run by the business, for the business and are of the business (to paraphrase Lincoln). Something like just 20% of BPM process improvement initiatives are run out of IT. Indeed, I would go a little further than that – BPM initiatives run out of IT are just not sustainable in the long term. If you are charged with maintaining a BPM program from within IT (perhaps running a BPM CoE), then one of your primary tasks is to a) identify and b) work with any Lean/Six Sigma programs that are out there.