Babies, Bath Water, And Enterprise Architecture Maturity Models

Recently I took a look at an EA-maturity-model-cum-roadmap from Leo de Sousa, Manager of Business Application Services and Enterprise Architecture at the British Columbia Institute of Technology (click to read Leo’s blog on EA CMM). To my surprise, I liked it. Why was I surprised?

I have never liked EA maturity models. Yes, tracking progress is important. And yes, there should be a consensus about what characterizes a mature EA practice. But I don’t like how they would ostensibly be used to compare one enterprise with another, a la benchmarks. Perhaps I was soured on them by seeing them used as a governance technique in US federal agencies.

The Office of Management and Budget (OMB) required agencies to assess themselves against a standardized maturity model, with progressive hurdles in successive years. Federal agencies, accustomed as they are to all sorts of oversight and compliance mandates, know how to pass compliance audits. Did you ever wonder how (then) Popkin System Architect got so popular  in the federal government? An EA tool was required to demonstrate a certain level of EA maturity and System Architect was the lowest-cost offering at the time (I’m sure there are other reasons as well). Behavior was around letter-of-the-law compliance, and it seldom catalyzed getting with the spirit. Even when Dick Burk at OMB introduced a clever workaround in a second version of the model — you could leapfrog to a level 4 if you showed actual business benefits, regardless of what other checklist items you missed — agencies simply marched through the maturity level checklists getting the requisite items done. The scores were good, but in my opinion they overstated the degree to which EA was ingrained in the culture of the agencies.

And then there’s the notion of a single number representing an enterprise. In most reasonably large organizations, there is some degree of federation of the EA effort. Some business units might have very advanced practices, while others have virtually none, and the enterprise-wide practice may have an entirely different agenda. What does one do, rate all units and average the number? That would be a pretty meaningless metric.


Source: Leo de Sousa, Manager, Business Application Services and Enterprise Architecture, British Columbia Institute of Technology

Source: Leo de Sousa, Manager, Business Application Services and Enterprise Architecture, British Columbia Institute of Technology



So, I’m afraid because  of all this I threw the proverbial baby out with the bath water regarding EA maturity models. But I like Leo’s. He took some fairly standard criteria and modified them to fit his organizational goals and processes. They still resemble other CMMI-like level criteria, so his 3 would be similar to someone else’s 3, but that’s not the main point. The point is to create a roadmap for where he wants his EA practice to go, and then rate and report actual against plan. This approach is about creating a tailored roadmap to consciously mature your EA practice; it is not about scoring maturity of implemented architectures.

No big deal, you say? I tweeted about this and a couple of wise EA folks who usually respond to my tweets essentially said “of course, you have to make it your own, what else would you do?” making me wonder if I’ve been missing the boat all along.  But I don’t think this is a prevalent attitude about EA maturity models. I think the general idea is to rate one’s org against where it ought to be, by some industry-standard scale. I don’t find that worth much. I think an organization should understand where they are and where they want to be, figure out how to get there, and track their progress. If someone else’s model of how EA can improve your org helps you describe your goal or your path, great. But generic doesn’t work — for any generic goal statement, you must edit it into terms specific to your enterprise. And don’t set any goals without including the statement “as measured by ___.” You can use the Goal-Question-Metric technique for filling in the blank, but that’s a topic for another day.

To really manage your EA agenda, I’d recommend taking the maturity model as a roadmap a step further and also populate it with goals that don’t fit the standard CMMI scale language.  For example, if you have an incipient information architecture or business architecture practice, place goals and milestones related to expanding and formalizing those efforts on the roadmap. I’ve been interviewing people about their information architecture initiatives and working with Forrester analyst Jeff Scott on approaches to business architecture that can be broken down into a cookbook-like  sequence, so watch this space for more on phased approaches to these aspects of architecture initiatives.

How do you all measure EA progress in your shops? Do you use a maturity model? And/or have you created concrete custom goals with associated metrics and measure against them? 


Leo's Update on EA CMM as a Roadmap

Gene, I thought I would update you. I am now a MSc in Information Management student at Syracuse University (part time) and am taking a course on Motivational Theories and Information. I am writing a paper to build out on our CMM roadmap approach. Interestingly, I am finding that some of the success factors of this approach relate to how people (particularly our Strategic Practitioners) can gain a level of competence, autonomy and motivation by using our approach. More to come ... I plan on blogging on this but school is a huge workload on top of my day job.

Hope this finds you well, Leo

Architectural Model Building

I am looking specifically at enterprise architecture. i.e. what is the initial stage for (implemented) EA what defines the managed stage for (implemented) EA, etc. What are the steps required to get out of initial.
My own view is that the evolutionary path appears to be mandatory. Management will not accept a strategy that takes you from initial to define without sitting at repeatable for awhile. Consultants also sell it this way - i.e. it will take at least 2 years to get to xyz.
In my view the maturity model should also define the skills required per level, the tools, the management support, and the influence of the (EA) head/team.
Architectural Model Building

re: Babies, Bath Water, And Enterprise Architecture Maturity Mo

We currently don't have a formal EA function and, therefore, we don't have a maturity model, however, I have recently been educating myself on the subject (I just finished reading "EA as Strategy" by Ross, Weill and Robertson and now am reading blogs on the subject :-) ) and I think the point about not measuring yourself against some generic EA maturity roadmap makes sense. I think a concept I picked up from "EA as Strategy" fits well here, namely that a firms EA maturity has to be in sync with the company's operational model. Certain operational models will only permit a certain level of maturity and this level should be your goal. Trying to live up to a maturity level that is not right for your company will most likely fail.