Old ROI Methods Are Holding Back The Adoption Of New Technology

My colleagues at Forrester and I have been puzzling over the discrepancy between the wealth of attractive new mobile, cloud, and smart computing technologies in the market, and the relatively weak record of actual growth in tech spending that our tech market forecasting numbers show.  Certainly, the recessions in Europe and weak economies in the US, Japan, China, India, Brazil and other emerging markets explain part of the weakness in tech buying.  In addition, cloud computing’s impact on the timing of tech spending (reducing initial upfront capital purchases of owned hardware and software while increasing future subscription payments for use of these resources) means that  spending that in the past would have occurred in current years has now been pushed into the future.  Lastly, as a recent Economist article pointed out, business investment in general has been low compared to GDP and to cash distributed to shareholders this decade, as CEOs with stock option compensation have focused on meeting quarterly earnings-per-share targets instead of investing for the longer term (see Buttonwood, “The Profits Prophet,” The Economist, October 5, 2013). Still, even taking these factors into account, tech investment has been growing more slowly relative to economic activity than in past cycles of tech innovation and growth.

We are beginning to think that an important cause of the slower-than-expected adoption of new mobile, cloud, and smart computing technologies lies in the financial models for calculating return-on-investment (ROI) for these tech purchases, which are stuck using old paradigms of measuring benefits in terms of headcount and similar operating cost savings as a result of automation.  These models for projecting the potential business value of a new technology investment have worked reasonably well for the past seventy years, when in fact the technologies coming to market were primarily focused on process automation and the benefits were mostly in allowing more work to be done faster and with fewer people.  But for many of these new technologies – especially the new mobile technologies of smartphones, tablets, and mobile apps, and the new smart computing technologies of enhanced awareness, analysis, and collaboration – the primary business benefit is not doing more work with fewer people.  Instead, the main benefit of these new technologies is reducing the probability of bad business outcomes, and increasing the probability of good business outcomes.  If a firm tries to measure benefits in terms of headcount or operational cost savings, it won’t find them – and as a result won’t make the investment.  For CEOs who are worried about uncertain economic growth and/or focused on meeting quarterly earnings-per-share expectations, lack of traditional “hard” economic benefits provides a good excuse to delay or minimize investments in these new technologies.

In our interactions with both clients and vendors, we are finding this is a real problem, not a theoretical one:

  • A major transportation company made a significant investment in sensors, but management is now challenging the project because the promised productivity benefits have not materialized.  Instead, the benefits are showing up in increased timeliness of deliveries, reduced loss and damage of goods during shipment, and improved customer satisfaction and repeat business – very positive developments, but harder to quantify.
  • In our evaluation of smart process application vendors, we found that many of the vendors were trying to sell these apps on the basis of making teams of people working on cases, projects, or operations more productive.  They acknowledge that the real benefit was to help make these people make smart decisions that would increase customer loyalty, solve problems more rapidly, increase the rate of project success, or similar business outcomes, but they told us that few clients were willing to base business case on these benefits.   The result was that many of these vendors were not closing as many deals as they could if they had more compelling success metrics.
  • In our discussions with sellers and implementers of big data and other analytics solutions, we have heard that many firms struggle with building an effective business case for these investments.  The three most common justifications are: 1) experience-based justifications -- "we have used similar data analysis in the past and it has delivered value, so we think this new analysis will have similar results ;" 2) objective-based justifications -- "we need this data analysis to determine the best way to justify or achieve our business objective"; and 3) faith-based justifications -- "well, it looks like other firms have used this kind of data analysis with positive results, so we will give it a try." But few big data investments are justified on a probability basis, that is, “this big data analysis will increase the percentage of customers who will buy two or more products from us by x%.”

I think three factors stand behind the failure to incorporate probabilistic benefits into traditional ROI analysis:

1) Management resistance to “soft” benefits.   Most CEOs and CFOs prefer to deal with what they call “hard” savings, and reducing headcount always seems the hardest of these hard savings.  But this kind of hard-headedness can often turn into block-headedness, when headcount costs are reduced without factoring in the impact on business operations.  The failure of the electronics chain Circuit City after its CEO replaced higher compensated but experienced sales staff with cheap but ignorant young sales workers is a classic example of this blockheadedness.  Most seasoned executives understand this.  Still, many executives – especially those trying to make an impression on senior management or shareholders -- take the attitude that cost savings that don’t take the form of layoffs and headcount reductions are not real savings. 

2) Lack of comfort with statistical analysis and probabilities.  Most CEOS and many CFOs never studied more than a semester or two of statistics in college and are uncomfortable with probabilistic calculations.  For them, a 5% risk is more or less the same as a 10% risk.  As a result, they won’t see the value of a solution that can reduce risks of a bad outcome from 10% to 5%.   Still, business executives do grasp good and bad frequency metrics, such as client attrition or retention rates, product defect rates, loan loss ratios, or systems up-time performance.  If a technology can be shown to move these metrics in a positive direction, then even the most statistically challenged executive will understand why it is a good investment.

3) Difficulty in calculating probabilistic benefits.  It takes a lot of analysis to determine the current level of bad outcomes (e.g., clients that stop being clients because of a bad experience, loans that go bad because of poor underwriting, service outages that disrupt business relationships, projects that fail to meet objectives) and the adverse business impacts of these bad outcomes.  It takes more analysis to identify the causes of bad outcomes, and still more analysis to define how and how much they can be reduced.  Yet analysis of this kind is at the heart of effective business improvement.  So, the analysis needed to understand why bad business outcomes occur instead of good business outcomes can also be used to justify investments in the technologies that can reduce the former and increase the latter.

There is enough adoption of new mobile, smart, and cloud technologies taking place to show that old school, hard-dollar ROI analysis has been supplemented by probabilistic benefit analysis in at least some companies (though there is no doubt that many investments have been based on hope or fear rather than solid analysis).  But it is undoubtedly the case that there are many investment in these new technologies that have not been made because the old-school analysis did not show a positive return.  Until that changes, new technology investment will underperform its full market potential.


1-3 VG + 4-6


This was refreshing. We've been in pilot phase for a couple of years with breakthrough technology that focuses on the highest ROI possible, offered at frankly subsidized level for the wealthiest orgs in the world for good matches. Even with some of the most credible people in the world involved--including CIO's own senior-most advisers, third party experts like Forrester & Gartner, & boutiques, very few seem to have the ability to move at all, or are moving so slow that they are inviting very serious trouble.

While I certainly appreciate & agree with your fine analysis, experience with tech adoption over 30 years prior to our pilot phase & testing, speaking directly to many boards & through our prof services network has provided a slightly different perspective, so thought I would chime in on a snowy Friday afternoon in Santa Fe.

I suspect we could identify dozens of influencing factors, but I'll be disciplined with only three.

4) Decision makers are not keeping up with groundbreaking innovation. We spoke to multiple Forrester clients -- example is a highly respected CIO in a top five co in that industry. In this case he's up to date & sharp enough, thought our system was a perfect match, but had little confidence that his board was sufficiently educated or culturally mature enough to adopt -- not just our system, but any advanced tech. They are followers, not leaders, although others look to them for leadership. A book could be written about this one alone--it's an opportunity some consulting firms are attempting to capture. Similar experiences combined with very heavy lifting for EDU led us to tweak our model to more consultative approach--I resisted due to so many out there, but we created several 7 figure contracts without capturing a penny--and I fear customers were not served well in some cases... due to #5.

5) The industry has too many conflicts, including career relationships between CIOs, vendors, media, conferences, some analysts-- no one knows who to trust--both the service model & software models & cultures have serious problems. For adoption of critical tech, esp requiring significant investment, transformation, etc. credibility is critical. The models have become increasingly murky --there is far less credibility out there than most think. Frankly I think there is far more corruption in this industry worldwide than most realize--just based on some of the offers I get & so much smoke. Some of the biggest brands have senior people who want to work for competitors while maintaining their own positions--two in particular appear to have serious systemic issues.

6) Macro risk aversion, including internal protectionist cultures.

Example--we presented to one of the world's largest buyers of IT systems with a decades-old relationship with the org & I think a very high level of mutual trust & respect. The decision maker loved our system--as well they should have as the cases were solid--highest ROI I've ever been involved with. They immediately called in a trusted firm for dd -- we know them very well, respect the people, but the firm has a direct conflict of interest, representing an extreme risk I wasn't willing to take--their future may well depend on stealing our IP & IC. This firm & the client are paranoid about jobs & in this case together represent serious internal competition. We are an IC/IP shop--I walked away.

Of course the continued very high percentage of budget dedicated to maintaining high cost commoditized IT that everyone in the world has--including competitors at much lower cost levels, continues to plague orgs like a slow fatal disease, whether they realize it or not.

Fortunately, we are finding a few leaders who still understand that their future depends on maintaining a competitive advantage, who have done the work to remain sharp enough to recognize it and act on it, but yes it's tough sledding--far more difficult than it should be based on merit.

It's times like these--fortunately rare, that often see new ecosystems, industry leadership, and customers that trounce competitors for being irresponsible.

I signed up for your webinar--look forward to it. -- MM

Old is not the issue, appropriate ROI determination is

What we call REAL ROI™ counters more than 20 pitfalls that commonly render most ROI determinations inaccurate and unreliable, including quantifying supposed intangibles.