"Using Working SW As The Measure Of Progress Is Narcissistic...." How Do You Measure The Value Of Agile Instead?

Hi all,

My colleague and friend Mike Gualtieri wrote a really interesting blog the other day titled "Agile Software Is A Cop-Out; Here's What's Next." While I am not going to discuss the great conclusions and "next practices" of software (SW) development Mike suggests in that blog, I do want to focus on the assumption he makes about using working SW as a measurement of Agile.

I am currently researching that area and investigating how organizations actually measure the value of Agile SW development (business and IT value). And I am finding that, while organizations aim to deliver working SW, they also define value metrics to measure progress and much more:

  • Cycle time (e.g., from concept to production);
  • Business value (from number of times a feature is used by clients to impact on sales revenue, etc.);
  • Productivity metrics (such as burndown velocity, number of features deployed versus estimated); and last but not least
  • Quality metrics (such as defects per sprint/release, etc.).

I think that today the Agile Manifesto focus on working SW must be (and probably is in those passionate Agile teams) working SW that does have all the qualities Mike would like to see, including UX design and a hell of a lot of "experience creativity." In fact, as you can see above, working SW is not the only measure app delivery teams use!

So while good, attractive, engaging, highly qualitative, etc, working SW is the objective here, I'd like to hear how you are measuring the value of Agile development in your organization, what metrics you are most successful with, and for what goals.

I am also interested in interviewing more organizations, so if you have experience to share and about 45 minutes of time and talk privately about it with me, please fire an email at dlogiudice@forrester.com.




Narcissism? Nope, just shorthand for useful...

"Working software" (to comment on Mike Gualtieri's original post) really means, working in the eyes of the user/customer. So it's not working unless it meets the need the user has.

Now one could argue that "working" is not enough, even in that broader understanding that it means more than just "code complete, compiles." "Working" means only one of the three dimensions of experience - useful. Doesn't necessarily mean usable or desirable.

Some of the metrics Diego mentions would relate to those other dimensions. For example, adoption would correlate to usability and desirability. That is, if you make an app more usable and desirable, more people will use it. Depending on your business goals, that will be less or more important, although adoption is always important to at least some degree. But the financial benefits of adoption are obviously even greater in customer-facing apps than internal apps, even if it still matters, internally.

Agile Development versus Business Technology

In reality we need an AGILE business and not just AGILE development. I propose that agile development does not create an agile business. It just makes the development more user focused but the process in itself holds the business back.

The development metrics will not turn SW developers into UX designers and domain experts. The problem has to be solved by NOT HAVING TO WRITE APPLICATIONS in Java.

Look at the impact of mobile client server. Trying to have a dev team on the go to make the same app work on an unlimited set of frontend devices is impossible and hope for this to be agile in anyway is futile.

The future must be application systems (like our Papyrus Platform) that provide a business studio to non-technical developers. What many vendors call a Business Studio today is however a developer tool. Developers need to write application SYSTEMS in which non-developers can model and define the data, content, rules, processes and UX that the business needs. I am still fighting the BPM flowcharting crowd because that is already too rigid for the needed agility. What the business needs is ultimately defined in a Business Architecture. So the platform has to expose the Business Architecture and link goal-oriented, application processes right into it. ONLY that is Business Technology. Coding for user frontend applications is a dead-end, agile or not.

More here: http://isismjpucher.wordpress.com/2011/10/17/goal-orientation-in-process...

...here comes modeling !

Hi Max, first of all thanks for reading the blog and commenting.

You are awakening the "business modeling" or "model driven dev" soul in me which i read in between your lines....:-) !!

I'd like to see your platform since for years I have argued that model driven development, and I am using the word "Model" in its broadest interpretation here, is too hard to completely automate because there is still too little provided by technology platforms. Yes BPM tools have made this better but still i don't see the full picture (maybe i just haven't seen enough) !

In this "higher level of abstraction" world I would split by the business experts developing their solutions and someone behind the curtains making it all happen (that is, the platform and technical developers fine tuning, configuring and supporting; check slide 6 of this PPT research http://bit.ly/pdhTHX for an idea of what I mean).

I guess Agile Development can be more agile if you use higher level of abstraction languages so that business people can self service their own development.. But independently of who does the "program" and what the "coding language" is, I think we could still benefit from some of the Agile Dev principles.

So while I agree Agile Development is not the ultimate answer for BT, but does get us closer to the business and therefore realize more BT than other approaches.... Of course Agile alone won't make it happen, and higher level of abstraction languages are welcome over Java!...perhaps the difference is that now you only focus on the “business value” of Agile (Business) development !..

Business Modeling and Agile

Hi Diego, we sponsor most of the Forrester events! We can meet there.

Thanks for the presentation file. Very interesting.

As soon as you generate CODE from the model you have a conversion and model-to-code synchronization problem to deal with. The ISIS Papyrus Platform executes the model and preserves it rather than convert it. Standards are in this not a solution but an additional problem as they are late, incomplete and driven by large vendor interests and not business needs. The complete OMG standards set is IT and not business driven. It will not make Agile happen ... IMHO. BPMN and BPEL are also not usable for business. Even with all the standards there is little conversion from one vendor to the other. And it is irrelevant. Look at Apple who sold a completely unique OS and toolkit on the world.

My point is that we need to get BEYOND development and coding languages. We need to move into BUSINESS LANGUAGE. As soon as code or scripts have to be written then you need to run a project. Makes it less agile. Agile is for me about bringing IT closer to business. A modeling approach that is driven by business (but may still use some technical people resources) is therefore more agile than an IT project that communicates well with users. In Papyrus models are used to describe the business architecture language and not to create declarations for developers to encode.

Business thinks in
1) people,
2) work items (in cases),
3) resource states (data and content),
4) rules and events, and
5) user experience.

Rules and state dependencies link work items together so one needs dynamic relationships and not the rigid ones of a DB. And then each user has a very specific way to look at it, meaning a specific GUI that needs to also now go to Mobile. The business data need to obviously be handled in many cases through a SOA (or similar) backend connection into service tasks. We do not intend to handle background transactions such as accounting, bank transactions or similar.

You need to add to your definitions 'model-preserving' or 'model-executing' as another approach to application modeling. It doesn't have the problems that model-assisted and model-driven approaches have.

Look forward to demo Papyrus to you ASAP. Regards, Max

Business Modeling and Agile

Agile methods emphasize face-to-face communication over written documents when the team is all in the same location. Most agile teams work in a single open office (called a bullpen), which facilitates such communication. Team size is typically small (5-9 people) to simplify team communication and team collaboration. Larger development efforts may be delivered by multiple teams working toward a common goal or on different parts of an effort. This may require a co-ordination of priorities across teams. When a team works in different locations, they maintain daily contact through videoconferencing, voice, e-mail, etc.

No matter what development disciplines are required, each agile team will contain a customer representative. This person is appointed by stakeholders to act on their behalf[10] and makes a personal commitment to being available for developers to answer mid-iteration problem-domain questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment (ROI) and ensuring alignment with customer needs and company goals.

Re: Business Modeling and Agile

Thanks George for your refresh on Agile !

So it seems you measure Agile Value in terms of : ROI, Allignement to Company goals and customer needs...not just working software !

Working VALUABLE software!

Great question! I think there are dimensions to push to get something better than "working software" as a the best measure:

1) Measure further out in the SDLC: measure how the software is used, not just whether it is in production. Define "done" not just as "deployed to production" but as "deployed to production in A/B testing scenarios with an ability to capture metrics on how consumers actually use it." (a la Jez Humble and Continuous Delivery).

2) Measure further up in the SDLC: know what your hypothesis is about what will delight the customer, and structure your software development process as a series of tests to that hypothesis (a la Eric Reis and the Lean Startup).

Blogged about this more or less coherently here: http://pagilista.blogspot.com/2011/10/full-monty-continuous-delivery-bus...


Doing the "right things" vs doing "things right" !

Thanks Elena for your insightful reply and reference to your great blog post.

I am currently researching this area and started by titling it exactly along the lines of the subject above (although won't probably be the final title of my research).

I totally agree that App Dev teams should step up the plate and also measure and be accountable for the value of "doing the right things "... All in all, what's the value of "doing everything right, fast, continuously, bug free...but ultimately not being the right thing the business needed in the first place ?"

I'd love to hear more about Metrics in your second case "measure further up in the SDLC", wonder if anyone has sample metrics to share !

Thanks again,