Sorry, Kids: APIs Have Not And Will Not Kill SOA

As I move about the industry talking about APIs (application programming interfaces) and the API economy — which hold important and transformative business opportunities — I’m frequently confronted with disparaging remarks about SOA (service-oriented architecture), as if it’s passé, gone, finito. It’s often in the way of (uninformed) assumptions about SOA. I hear things like, “SOA failed because it was too difficult” or “People do REST APIs now, they don’t do SOA” or other such bunk.

I’ll be the first to extol the importance and benefits of APIs, but the tales of SOA’s failure and demise are simply wrong (I really do like APIs; see this report). I had a powerful reminder of all this while attending IBM’s IMPACT conference this week. First off, I arrived late to one customer’s plain-vanilla “this is our enterprise SOA journey” session only to be refused entry because the room was over capacity. Glancing over the conference program, there were at least eight such sessions representing four continents and at least five vertical industries. I attended five of them and also had lunch with another SOA leader. The stories could all be summarized by the following plot line:

  • We saw the value in SOA. Whether the need was multichannel customer engagement, faster time-to-market, retiring legacy, getting past complex and costly point-to-point integration, dealing with duplicate applications, or some other business-technology problem, a core team recognized that SOA could make things better.
  • We slowly built SOA steam by delivering value from one project to the next. Whether by finding a modicum of support among executives or by working one-on-one with delivery projects, the core SOA team built small successes, then bigger successes, then gained a more-or-less enterprise commitment to SOA. On the whole, funding was on the backs of projects.
  • We realized early on that SOA needs a business approach. A technical approach to service design leads nowhere, generally speaking. The most valuable design point for services starts with queries and transactions that a businessperson can understand (see Business-Focused Approach to SOA)
  • We’ve learned a lot and built-up some important SOA best practices. These SOA success stories sounded only vaguely like the SOA hype of the 2000s. That’s because, like flossing your teeth and going to the gym, organizational maturity and discipline don’t make good hype (see SOA Governance Improves SOA Benefit Realization).
  • We still have struggles and there are ways we need to improve. Every organization recognized that, even though they had achieved great value from SOA, they still had problems. Paying close attention to service interface design and building interteam collaboration are deeply embedded in the Five Most Valuable SOA Practices.
  • It’s onward and ahead with SOA to greater maturity and more business value. Each of these organizations saw more, better, and deeper value through continuation and deepening of their investments in SOA (see the high SOA satisfaction from back before Forrester deemed SOA to be de rigueur and so stopped surveying about it)

For none of these major organizations at IBM IMPACT was SOA in any way passé, gone, or finito. Actually, I continue to get client inquiries about starting on SOA, or else on rebooting an SOA program that was previously built on theoretical, industry half-truths about SOA. . .for very good reason.

By picking on SOA struggles while ignoring the high levels of SOA satisfaction, the industry’s reverse-hype on SOA has been quite misleading. You should ignore it in the same way that it was right to ignore the over-hype that led some into shallow, theoretical approaches to SOA. From my many SOA conversations over the years, I’m convinced it’s the shallow approach to SOA that leads to the consistent 2% or 3% that say they tried SOA and didn’t like it. Let me say that a different way: Over years of surveys, only 2% to 3% of those trying SOA have decided to give it up.

For the strong majority, the value delivered by SOA has been quite large enough to overshadow the costs and pains and missteps of SOA struggles. They’ll carry on using SOA to deliver strong business agility, and then they’ll use APIs to extend their business agility into many new business contexts and scenarios (see my bit on Business Agility And API & SOA Maturity).



Thanks Randy. Couldn't agree more.

Randy, I see SOA mostly

Randy, I see SOA mostly valuable where the silos have either well-thought-out or matured SOA interfaces. Projects where all pieces have to be converted to SOA interfaces typically fail. The main issue is that it is not the technicality of SOA that is the problem but a interface design that matches the needs and the underlying functionality on the other end. If you have an old system and then want to add SOA on top of it that can be a nightmare.

I have been involved at one of the lergest European banks who made the rash statement that within two years they will be all SOA and all XML data formats. A large team of consultants was hired too. The project ended after about 18 month with not a single line coded. That was by the way ten years ago. The bank has never restarted the project.

Yes, I know about successful ones too and as I said, they built on existing interfaces.

Indeed, big bang SOA is a mistake

Max - I also see failures when SOA becomes the goal, as with the Euro bank you mention. SOA is not the goal. Delivering business agility and value is the goal.

However, contrary to your experience, I've seen *many* organizations where SOA on top of a old legacy system was exactly the right answer. For one airline, putting SOA on top of mainframe legacy cost 1/4 as much time and money -- and delivered a more flexible end result -- compared to replacing the system. It was very successful, and this is but one of the many I've seen.

So, you don't have to have an existing interface to gain greatly from SOA. But, one must dig behind the top-line industry half-truths about SOA in order to do it well.

Great advice!

Randy is right on target. At digitalML we work with many firms that are successfully evolving their SOA practices to embrace the API Economy. For the many firms that have succeeded with SOA, building a thin API Facade layer that exposes their portfolio of existing SOAP Services in a RESTful form, one matched to the semantics and workflow of the applications that consume those APIs (not just repackaging the SOAP operations as-is), is a great way to get a quick win.

For example one large retailer quickly delivered 200 APIs using this approach. This success then helps build support for more adoption and future success.

OTOH one does occasionally see firms that have struggled with SOA. I agree with Randy that one common reason for this is taking a technology-centric approach, not business-led, that leads to laying down lots of pavement (runtime SOA infrastructure) but doing little or nothing to shape the way services are conceived and designed (practices that shape vehicle design, to extend the metaphor).

But one other common failure mode is a lack of sufficient organizational maturity. That's not to say that such a firm cannot do SOA -- but they will face bigger obstacles that they must work to overcome, before succeeding.

For example a large telco that wants OmniChannel realized it needed a top-down mandate from a new CIO (backed by a strong CEO) to drive the kind of organizational change needed to make OmniChannel a reality, based around APIs.

Change is hard. It's organizational and process change that brings success with SOA (and APIs).

One clarification: "business [design] led" SOA (and APIs)

Yep, Mike. Good stuff. One thing I would clarify, because I often see it misread. I agree with your "business-led" comment, but this doesn't mean that business *people* are doing the leading (i.e., it doesn't need a businessperson saying, "do this or that for SOA"). So, I prefer to cast is as "business design led" -- meaning that those leading SOA efforts take a business-first approach in two primary ways:

1. They design the most important services (business services) to mirror real business transactions and queries the way a businessperson would think about them (and thus, one can review a service design with a businessperson and they can understand it).
2. They don't make SOA the goal, but rather use SOA as an approach to delivering solutions and value as the business needs it (rather than a big bang SOA buildout).

These business-design-led concepts apply every bit as much to APIs (as well as other business oriented practices), although APIs open up the design space a bit compared to straight-on SOA (see the API design report series which starts with the one referenced in the post [] and continues with this one [] and then three others).

Completely agree with the

Completely agree with the post Randy.. Well said.