Closing The Innovation Gap

Greetings — thanks for taking the time to read my inaugural blog! Let me introduce myself by way of continuing a discussion that I started at Practicing EA and on innovation and technology that I think strikes at the heart of our challenges as enterprise architects. It also provides a good context for my future research, which I discuss at the end.

Closing The Innovation Gap

In part 1 of this post, I claim that a gap opened while we were fighting the overly complex, expensive current state and trying to help our business partners innovate with new technology.

The gap – We cannot deliver new technology and innovation quickly or cheaply enough.

Shadow IT Is The Symptom, Not The Cause

  • The Symptom – We often blame Shadow IT and manual workarounds for increases in complexity, reduction in quality of service, and obscuring true technology costs. These are symptoms of the problem, not the problem itself.
  • The Cause – Business users know more about what they need and when they need it and are the most motivated to solve their problems now, not once the budget cycle gets around to funding a project. Central IT, where most EAs practice, is a knowledge store for designing enterprise-scale systems but is constrained in its ability to deliver.

So what is to be done? Do you think this is correct? Before you form an opinion, consider the nature of innovation.

Innovation And Lead Users

In his thought-provoking 2005 book, Democratizing Innovation, Eric Von Hippel makes an interesting point: The most transformative, disruptive, and useful innovation comes from product users, not providers. He points out differences in motivation that explain why this is so.

Product users are simply motivated — they want exact, affordable solutions to their immediate problems, and they want them fast. Product producers, on the other hand, are motivated by many forces; certainly customer satisfaction is important, but economic feasibility is of greater concern, as are other factors such as scalability.

Quite often, these complex motivations cause product producers to innovate for themselves, for example, to reduce costs, enable operations at scale, or create reusable value for other customers — resulting in products that are less acceptable to the users for whom the product was originally conceived. Apply this idea to today’s centralized IT shops, and you can see the parallels. Shadow IT and technology-savvy end users are using Access databases and spreadsheets and even employing more-sophisticated technologies to create tactical or workaround solutions in a fraction of the time.

Von Hippel goes a step further, defining a class of users called Lead Users who are at the forefront of business needs and product innovation. These key people deliver most user-driven innovations (and often the most headaches for product producers.) He provides a lot of data about Lead Users that underscores the importance of this class of people. We all know them; we may even be them. They exist in the business and in IT.

Empowering these Lead Users is the way to close the gap! By that I don’t mean letting technology users do whatever they want but rather: 1) providing a safe environment for innovation and then ensuring that enterprise-class solutions are progressively hardened and moved to an appropriately managed environment; 2) shifting our attitude on standards away from inviolate rules and toward a policy- and principal-driven approach — something my team refers to as “guardrails”; and 3) becoming technology consultants and trusted advisors, promoting the appropriate business accountability for technology decisions as a complement to empowerment.

My Research Coverage

So what is my job in all this? What will I write about? Simply put, I will cover trends and emerging technology to help EAs help their customers become better aligned and more empowered. My research will help EAs: 1) develop an intuitive understanding of a trend or specific technology; 2) understand the market response, issues faced, and impacts; and 3) help the business and IT decide what to do about it, referencing research by others in the EA team and technology details covered by other roles.

What are the emerging technologies that you are most concerned about or interested in? Cloud, mobile, social, and data continue to be top priorities, but what are the specific issues you are grappling with? I look forward to your responses.

Thanks in advance for collaborating with me here, in the Forrester EA Community, and via Twitter.


I indeed agree that the

I indeed agree that the Shadow IT of today is a reflection of a high technical competency within the business. Couple that competency with the lethargic IT budget cycle and Shadow IT proliferates. One topic I didn't see mentioned is the overall IT organisational structure and its impact on enterprise architecture. I've posted on this topic myself here:

I'm looking forward for more perspectives you can share on EA and it's present day challenges.

IT Org Structure

IT Organization does have a big impact on EA and possibly a big impact on closing the innovation gap, but I need to think more about that. Thanks for pointing that out.

I agree, to a point...

Shadow IT or what I like to call "feral systems" is inevitable. They are, as Brian suggests, a symptom of our inability to be responsive to the time dependent, cost constrained and sometimes "temporary" needs of business community that simply wants to get their work done and turn a profit.

In the movie Jurassic Park, the character Dr. Ian Malcolm said in response to a question "No, I'm simply saying that life, uh... finds a way." Business users are a force of nature, too and they will find a way.

You can ignore it, but it will still be there.

You can try to regulate it, but most IT groups probably don't have the "juice" to make that work and might in the end destroy the behavior it was trying to harness. Long term ownership and support for these applications is always a difficult problem to address.

I'd rather see us focus on the real problem, not providing the business with type and level of service they need.

As an industry, we must look at tools, services and methods and structure ourselves so that we can respond with "good enough" solutions for the business, so they don't need to do it for themselves. Every solution is not mission critical and doesn't necessarily have a long term role in our application portfolios. Many are tactical, not strategic. Note: These "rough sketch" applications may eventually inspire a future enterprise application where we apply the lessons learned from these "lite" systems.

In order to address this, we need patterns and tools that enable us to meet their needs and expectations. Realistically, these will differ from company to company and environment to environment.

However, most solutions built by users will occur using some version of Microsoft Office. As such, the solution might be as simple as putting their tool under control and making it more scalable through Citrix or converting over using something like Oracle APEX (Application Express) which can turn these into web applications.

More complex requirements (i.e. solutions that need workflow, content management, etc.) might be better served with a Platform as a Service (PasS) and/or Software as a Service (SasS) provider like SalesForce/ where IT can interactively build custom applications using declarative programming with the business. For applications with small numbers of users, the costs are reasonable and linearly predictable. Obviously, they wouldn't necessarily be right for a larger scale enterprise solution, but that's a different problem that requires a different solution/approach.

All the pieces are there, the question is whether we can/will evolve to meet the challenge.

The Business Will Find a Way (I like it)

Like your reference to Jurassic Park....indeed the business will find a way. Regarding your statement about the real problem - is your point that we simply must get better at serving the business? I agree with that, but the question is how we best do that.
Thanks for taking the time to share your insights.

Hi Byron

Didn't see it was you! Thanks for providing your perspective. The idea I'm asserting is that we can take things like Idea Factory, create an innovation zone from it, empower the business AND progressively harden solutions that really should be enterprise. Thoughts?

An idea, but one that can only go so far to fix the gap...

A key piece of the innovation gap you described has less to do with responsiveness (“…not delivering innovation and new technology fast and cheaply enough…” from Part 1 of your post) and more to do with understanding and perhaps perspective. Ultimately, a huge driver behind Shadow IT is that even when a product is delivered to the business to meet a need, the fit is so poor that it amounts to not having a tool for the job at all. So the business must go out find their own—leading to the results you highlight.

What I’ve seen after countless acquisition cycles (federal, commercial) on both sides of the fence (buyer, supplier) is the inability of IT-controlled purchases to meet the needs of the business end-user. Invariably, IT purchasing in most enterprises involves non-process-area technology-group buyers evaluating products using either a binary evaluation (yes / no) of particular features / functionality or an arbitrary “weighting” scorecard that’s supposed to be able to evaluate “how well” a feature / functionality works. The result is a conclusion that looks great and means nothing—after all, there’s nothing quite as effective as hiding guesswork behind a camouflage of rigor and burying those “fuzzy” concepts like “usability.”

To be fair, this buying-by-spreadsheet is not the fault of IT alone. Business users are loathe to pass meaningful requirements for IT acquisition folks to use. For those few concrete needs that manage to be collected, an even smaller number actually reflect an objective understanding of the processes the tool is intended to support. Unable to articulate their processes and harried by IT to give them information, business produces a haphazard mashup of criteria that reflect the concerns of those who could be bothered to respond with whatever thoughts occurred to them that day, hour, minute, or second.

I like the idea of a Lead User. There is potential for the role to more forcefully drive relevant, innovative change. Plus, it’s a nice shift from having IT supply their own ‘business analysts’ to meet the need. However, it will be no more effective than prior models until business and IT have a common platform for understanding one another. While it can help address the responsiveness gap, the problems with how company’s make enterprise application buying decisions will continue to drive the Shadow IT phenomenon.

(This is where TOGAF’s ADM gets an appreciative nod. Although not perfect, it still reflects, in an ideal sense, a great way to consider the business-to-IT handoff. Not a shameless plug on my part, since I’m not an Open Group member—just an appreciative reference.)

Continue Discussion on Forrester EA Community

Please join me on the Forrester EA Community site ( to continue discussing this topic. Some interesting points brought out here that I'd like to talk about further. You don't have to be a Forrester client to join, so please register and reply to the post I have created there on this topic.