At a recent SAP customer event on Business Transformation, Alexander Budzier from the Said Business School at the University of Oxford presented findings on IT project outcomes and their correlations with various project factors. When determining project success rates, the researchers considered business benefits, adherence to budget, and on-time delivery.
Interesting findings from this research include:
Project success does not correlate (or very minimally correlates) with size or length of project, or with public versus private sector.
Focusing on one goal too much can have a negative effect on other metrics. Consider the extreme example of the Olympics, which had a 100% on-time result (over 10 Games analyzed) but the highest cost overruns, at an average of 207%!
Agile deployments (versus big bang) had greater success in some metrics, particularly schedule adherence, but not all.
The single biggest factor in determining project adherence to budget and timelines was benefits management. (In this research fewer than half of the projects they studied had actually tracked benefits.) Those who focused on measuring benefits significantly reduced BOTH project cost and schedule risk. Project cost overruns averages decreased from 36% to 6% when focusing on business benefits; schedule overruns decreased from 119% to 51%.
So, what can we take away from this? Project leaders should:
Focus on benefits – throughout the project lifecycle. Benchmarking can help leaders to identify what benefits / metrics to track.
Recognize warning signs / risks early -- and address them before they result in disaster. These risks include unknowns in design, organizational resistance, and shifting project requirements.
There has been a lot of negative press and commentary regarding the recent Queensland Health Implementation of Continuity Project (SAP HR and Payroll), which recently experienced a very public failure as many employees were not paid due to multiple points of failure in the project. The recent Auditor-General's Report on the process is damning, spreading the blame across multiple agencies and the systems integration partner, IBM. I make no claims to be familiar with the intricate details of the process, but I have read the report and feel I have a clear understanding of the (many!) points of failure.
While this project did seem to be a monumental failure, I would suggest that we consider two important facts: