As firms face growing competition for customers, they naturally seek to compare themselves with their peers and competitors, but there is a trap: Leaders don’t compare themselves with competitors anymore. Instead, they compare their current performance with where they need to be as a leader, and that’s what the business expects.
In the past, it was common to benchmark organizational performance against “industry averages,” and being “above average” was considered good. Today, “above average” is no longer good enough; fickle customers demand exceptional experiences. Delivering those experiences requires exceptional performance; anything less means that another company may steal your customers.
Benchmarking is for followers, not leaders. Organizations want to be “unicorns,” like the Etsys, Netflixes, Googles, and Salesforces of the world. They don’t want to be losing “horses.”
Most benchmarking approaches target the IT of the past, not BT. Benchmark methodologies and data were created and heavily used when software delivery capability was considered a cost, not a differentiator. In business technology, software is a key differentiator, and BT leaders want to be the best and continuously improve.
Modern application delivery leaders realize that their primary goal is to deliver value to the business and its customers faster. Most of the modern successful change frameworks, like Agile (in its various instantiations), Lean, and Lean Startup, which inspire developers and development shops, put metrics and measurement at the center of improvement and feedback loops. The objective of controlling and governing projects to meet vaguely estimated efforts but precisely defined budgets as well as unrealistic deadlines is no longer on the agenda of leading BT organizations.
The new objective of BT organizations is to connect more linearly the work that app dev teams do and the results they produce to deliver business outcomes. In this context, application development and delivery (AD&D) leaders need a new set of metrics that help them monitor and improve the value they deliver, based on feedback from business partners and customers.
Preproduction metrics. Leading organizations capture preproduction data on activities and milestones through productivity metrics, but they place a growing emphasis on the predictability of the continuous delivery pipeline, quality, and value.
Within the modern applications era, regardless of whether new software applications are being developed and delivered for mobile, tablets, or the Web, the truly successful app-dev leaders will be those who focus on delivering constant value and incremental improvement to their business. That is a totally different perspective from “I need to keep control of my team’s productivity to make sure that we stick to our estimated costs, scope, and project dates.” Of course, the interest in cost is never going away, but app-dev leaders today have a great chance to enhance their conversation with executives and business stakeholders and add value to the conversation.
However, as the recent research I just published, Agile Metrics That Matter, proves, while some of the most advanced Agile teams do use new progress, quality, efficiency, and value/benefits metrics (these to a lesser degree), some software development industry luminaries have worked and are working on new methods to measure value in software development. But it’s still early days!
I’d like to summarize here some good old practices on establishing metrics that count together with some of the new findings of the research:
There is no doubt that Agile growth in the market is significant, and the growing daily number of inquiries I’ve been getting on Agile from end user organizations in 2012 gives me the impression that many are moving from tactical to strategic adoption. Why’s that? Many reasons, and you can read about them in our focused research on Agile transformation on the Forrester website. But I’d like to summarize the top five reasons from my recent research “Determine The Business And IT Impact Of Agile Development” :
Quality was the top — quite astonishing, but both the survey we ran across 205 Agile “professional adopters” and the interviews across some 21 organizations confirmed this. My read is that this is about functional quality.
Change was second to quality. We live in an era where innovation strives and organizations are continuously developing new apps and projects. But your business does not necessarily know what it needs or wants upfront. The business really appreciates the due-course changes that Agile development allows, as they enable the business to experiment and try out various options so it can become more confident about what is really right for the organization. Cutting edge cutting edge systems-of-engagement (Mobile, Web-facing, Social-media, etc) require lots of Change in due course.
My colleague and friend Mike Gualtieri wrote a really interesting blog the other day titled "Agile Software Is A Cop-Out; Here's What's Next." While I am not going to discuss the great conclusions and "next practices" of software (SW) development Mike suggests in that blog, I do want to focus on the assumption he makes about using working SW as a measurement of Agile.
I am currently researching that area and investigating how organizations actually measure the value of Agile SW development (business and IT value). And I am finding that, while organizations aim to deliver working SW, they also define value metrics to measure progress and much more:
Cycle time (e.g., from concept to production);
Business value (from number of times a feature is used by clients to impact on sales revenue, etc.);
Productivity metrics (such as burndown velocity, number of features deployed versus estimated); and last but not least
Quality metrics (such as defects per sprint/release, etc.).
Recently, I discussed complexity with a banker working on measuring and managing complexity in a North American bank. His approach is very interesting: He found a way to operationalize complexity measurement and thus to provide concrete data to manage it. While I’m not in a position to disclose any more details, we also talked about the nature of complexity. In absence of any other definition of complexity, I offered a draft definition which I have assembled over time based on a number of “official” definitions. Complexity is the condition of: