“We Do A Great Job In IT, Our Metrics Dashboard Is A Sea Of Green.” Really?

A Forrester-client inquiry call last night and the creation of some slides for a webinar with Axios really got me thinking about how we measure our success in IT. It just seemed so easy to take the IT version of success (and the associated measures) and create a snide customer retort. It’s a little tongue-in-cheek but please take a read of one of my Axios slides:

I'm sure there are many more to play with.

If you read my blogs on a regular basis you will have seen:

But I need to bring the topic up again.

Common ITSM metrics – help or hindrance?

Don’t get me wrong. Organizations such as HDI do a great job in providing benchmarks across common metrics. I use them regularly and Forrester clients want to receive them. Benchmarks are like gold dust in IT.

What worries me though is that:

  • Organizations might just follow best practice and blindly adopt the common ITSM or IT metrics.
  • We continue to see metrics as a score to beat rather than a pulse to take and the right medicine taken as needed.
  • Benchmarks are often not a case of comparing apples with apples. Are we kidding ourselves that we are better than the average bear? Or are we chastising ourselves over a level of capability we can never reach?

My challenge to you …

Did you hear that? It was my gauntlet hitting the floor.

Please take a look at your metrics or KPIs and do two things:

  • Ask yourself what each metric really means and what your customers would make of it (as per the above).
  • Look for where you have used that metric as a launch-pad for improvement, i.e. that you are not viewing metrics as the final output but rather as an input into something else, such as business conversations about services or improvement activity.

I’d be interested in what you see and think.

Finally, if you want to tune into the Axios webinar on “Using ITSM to increase business user satisfaction and the perception of IT” the sign up is here. And don’t forget the wise words of Ivor MacFarlane of IBM: “If we use the wrong metrics, do we not get better at the wrong things?”

Latest blog on metrics: http://blogs.forrester.com/stephen_mann/13-04-17-it_service_management_benchmarks_for_you_by_you

Also see: http://blogs.forrester.com/stephen_mann/13-01-31-is_your_it_service_desk_customer_experience_up_to_scratch

Comments

What Gets Measured Gets Done

Anytime I get a question from a client on whether or not a client's metrics are best in class so they can show them to the business, I tell them that it's like asking me whether or not you are a good husband or not. I don't know your wife, and I can't make assumptions about the things she values, so it's easier if you just ask HER if you're a good spouse!

To your point on asking your customers what they care about - it's getting to the core of demonstrating the things they value, not the things IT values. If customer sat. isn't at the forefront of your metrics, then you are doing it wrong.

The trepidation

Stephen,
Thanks for this excellent look at some of the things that IT (and other departments as well) tend to get wrong. I think that Ian Clayton is right when he calls this "inside-out" thinking. But there's more.

Often I get questions that begin, "What is the industry standard for X?" where X = first contact resolution, abandon rate, response time or another common metric. I explain why there isn't an "industry standard" and why their organization needs to figure out what is good, what is acceptable, and what is unacceptable, and that they need to talk to the customers to find those things out.

I believe that the reason these questions are phrased that way ("industry standard") is that managers are afraid that their numbers don't match those of other organizations, and I think that feeling comes from higher up--"Such-and-such company gets 80% FCR. We need to get that too." Managers need to get out of their zone of fear and trepidation and open the discussion with customers--and with their own managers--about what is important.

This is an important discussion. IT in general, and support in particular, needs to get the blinders off. Thanks for posting.

Brilliant article for provoking thought

The "Success" graphic is a great breakdown of mindsets in IT.

I frequently ask my customers to "bear with me" while I ask a series of questions. The first in typically innocuous, like "what KPI do you care about?" Then come the "whys":
- Why do you care about it - answer.
- Why do you care about that - answer.
- Why does that person/group want that - answer
- Why does that person/group matter- answer.

Each answer peels the layers back to help them understand that deeper objective beyond "keeping the lights on".

It seems that customers in the Enterprise space have a better grasp about this than some sectors.

Thanks for the article!

Metrics and cost

I love this slide, very inspiring. Perhaps its obvious but I still want to add the dimension expectations vs costs. As often as IT don't deliver the vital 0.003% availability on crucial systems, a lot of money is spent on systems that actually have much lower needs. Two sides of the same coin.

Metrics are IT "quotas"

I think of One of Demings points of management: Eliminate work standards (quotas) on the factory floor. Substitute with leadership.
I agree with Roy's comment that we should understand what is important to the customer. Metrics are simply a measure that should indicate where you are on a journey. Instead, metrics become a destination that once achieved consistently can fake out an IT organization. We have realized that once certain metrics are set, employees lose their motivation to continuously improve IT services for customers.
As in the above example, companies celebrate 70% FCR instead of improving the defects in training, systems, software, and processes that caused so many calls in the first place. What we need is courageous leaders willing to fight to improve quality and continuous improvement all while helping IT understand what the customers really want. Eliminate metrics, substitute with leadership.

There are no absolutes

I've been in more than one meeting with the business on one side and IT on the other, both talking past each other because the latest IT dashboard report does not reflect the business experience or what IT does.

The business needs to say what it wants. Typically it can't. When making a choice between additional cost or reduced service, it won't take responsibility for its own decision. IT is typically poor at understanding the business needs, explaining what it can offer, or helping business to make the trade-off. So what's new?

Poorly designed or *understood* metrics are part of the problem. Take your 99.97% availability example. If IT did actually deliver that number (i.e the *service* truly available) then for 13 minutes a month the system was unavailable. Did IT ever say to the business, "Look, expect *YOUR* services to be unavailable for 13 minutes a month"? If they did, why would business be surprised? Did the business say, "How can we get higher availability. Here's mega-dollars to make it happen"? No - I thought not. But a sensible conversation along those lines is part of the path to resolution.

And BTW, measure's are always an approximation, in this business. And the absolute value doesn't normally mean anything. It's the trend that matters. Is availability increasing? Are the number of Incidents this month lower than last?

But the relationship and understanding is what matters most. If you have a broken relationship between IT and the business, it doesn't matter what your measures say. An all-green dashboard report will look like a red rag.

Having metrics in place is

Having metrics in place is better than none at all. Having said that, a re-orientation is in order for IT departments to view metrics from an end to end perspective in respect to business needs.
The sea of green most times simply means job security for IT, not customer satisfaction.