IT Service Management Metrics: Advice And 10 Top Tips

Earlier this week, I attended the Hornbill User Group (or "HUG" as it is affectionately known) to listen to Malcolm Fry, IT service management (ITSM) legend and author of "ITIL Lite," talk about ITSM metrics in the context of ITIL 2011.

There is no doubt that metrics have long been a topic of interest, concern, and debate for ITSM practitioners (I wrote a piece a few years ago that is still the most popular item on my old blog site by a huge margin), and IMO I&O organizations struggle with the area due to a number of reasons:

  • I&O is not entirely sure what it is doing (in terms of metrics) and why.
  • We often measure what is easy to measure rather than what we should measure.
  • I&O can easily fall into the trap of focusing on IT metrics rather than business-focused metrics.
  • I&O organizations often have too many metrics as opposed to a select few (often led by the abundance of reports and metrics provided by the ITSM tool or tools of choice).
  • There is no structure or context between metrics (these can be stuck in silos rather than being “end-to-end”).
  • Metrics are commonly viewed as an output in their own right rather than as an input into business conversations about services or improvement activity.

However, that’s enough of my waffle …

So what did Malcolm say?

An early and interesting point was that we aim for, and then reward, failure. We set targets such as 99.9% availability rather than saying "we will aim for 100% availability, and we will never go below 99.9%." So in some way, we do admit defeat before even starting.

Malcolm also pulled up a slide showing Maslow’s Hierarchy of Needs (hadn’t seen that for a while) to segue into a Hierarchy of Metrics … we should be thinking about where our metrics sit within this.

Malcolm also offered up the concept of looking at ITSM performance as simply as "good day" or "bad day," rolling up lower-level measures into a single view as per the incident management example below.

With follow-up as appropriate:

  • Good Day = quick review and special actions
  • Bad Day = what went wrong and how we can stop it happening again

Finally, Malcolm offered up his top 10 tips for IT service management metrics:

  1. Understand at the metric hierarchy.
  2. Collect everything but only report that what needs reporting.
  3. Make sure each level understand their roles.
  4. Metrics should measure success not endorse failure.
  5. If in doubt collect the data.
  6. Metrics will evolve as trends and futures appear.
  7. Make sure that new technologies capture correct data.
  8. Make sure everyone knows their roles.
  9. Don’t set targets that wrongly drive behaviour.
  10. Ensure that you understand the nature of the recipient.

Care to offer up any of your own IT service management metric advice?

Note: Malcolm Fry kindly consented to Forrester's use of his IP for this blog.

 

UPDATE: Two popular ITIL-related blogs:

Please check out my latest blog ... http://blogs.forrester.com/stephen_mann

Comments

Great tips, here's a few more....

Stephen, great list of tips on metrics. I have a few more that I often discuss with clients: make sure the metrics are ACTIONABLE. If there is no direct action or decision that relies directly on that metrics, why are you measuring it?

I also often find that clients are putting too much time and effort into trying to collect certain metrics... if more than two people or systems need to be queried to get the data for a metric, I would ask two questions: 1) Is the knowledge I'm gaining from this metric equal to the time/money/effort spent to measure it? 2) Is there another metric that could drive the same actionable outcome but may be easier to collect?

SHAMELESS PLUG: I'm running a workshop on metrics for infrastructure and operations professionals (with a strong emphasis on ITSM) called "Building The IT Infrastructure And Operations Balanced Scorecard" in Miami in a few weeks. You can find more info on it here: http://www.forrester.com/events/eventdetail/0,9179,2571,00.html

Metrics

...In your tweet you asked 'Is it just me or have metrics landed center stage?'. In my experience no they haven't. The number 1 ABC card chosen in the customer exercise is 'IT has too little understanding of business impact and priority', whenever we do an Apollo simulation and I ask as Customer 'What have you promised to deliver in terms of KPI's relating to Value, Outcomes, Costs and Risks'I get blank stares. They when I say 'ITIL (or an ITSM framework) is there so that you can demonstrate achievement of these.....why are you doing ITIL in your organization? what are the 3 most important KPI's you will use to demonstrate value to your business?.........I usually get MORE blank stares.

If I use the following maturity model, which I just invented....being a consyultant I can't do anything without a model :-)

Level 5. Duh! Yes we use KPIs and they are aligned with the business? that is what is has always been about, what a stupid question.

Level 4. We have KPIs but they are pretty much internally focused and don't cover end-to-end or business value

Level 3. Of course we have KPI's - 99% availability of the systems ....groan, headache

Level 2. Oh!.....so that what it is all about, using a framework to achieve KPI's

Level 1. Blank stares....

We see that we generally score pretty low. I have asked these questions to thousands of IT people including management teams.

So the answer to your question 'have metrics landed center stage...' , I think they are still off in the side lines.....IT gets a penalty for being too internally focused.

Paul

But...what metrics?

It's all very well giving the rationale about how to select metrics but, bejesus, does Malcolm share examples/practices? Are all I&O orgs so different? Surely in the same vertical, say retail banking, the I&O metrics will be similar? I'd like to see examples, then the rationale so it makes it easier to understand. Rationale only is like a story with no end.

There are 100s of "standard metrics" for ...

I&O or ITSM which is part of the issue. They are also misused - "look at us doing a great job" rather than look at how we have helped the business achieve X or Y. I could email you a very long list of metrics but that is what too many I&O (IT infrastructure and operations) organizations do, i.e. the ones that miss the point about I&O or IT service delivery performance not being about how shiny the data center is or how clever our developers are (the internally focused thinking).