Why Do We Consider Only One Option When Building Technology?

Outside the technology industry, engineers sometimes build multiple prototypes before selecting one particular design. Rather than finding the defects of one proposed design, discarding it, and moving on to the next idea, engineering teams that apply this approach, set-based concurrent design, save themselves a lot of time and headaches by running through as many options as possible simultaneously. As you might have guessed, this technique isn't cheap. You need to staff multiple design teams, and prototyping in many industries, such as automotive manufacturing, is always expensive. Nonetheless, by delaying the design decision for as long as possible, until the team has found the best among multiple ideas, development can take less time, with a greater probability of building a good product, than with sequential design.

So why don't we build software this way? For Microsoft, SAP, and other technology companies, prototyping is orders of magnitude cheaper than it is for Toyota and General Motors. Executives at tech companies would love to reduce the unpredictability of development schedules, often thrown off-track by unexpected design issues. So why hasn't set-based design caught fire in the technology industry?

That question has been plaguing me since hearing an excellent presentation by Jean Tabaka and Bill Wake on this topic at Agile 2009. I'm not sure of the answer ("More research required," says the analyst), but I'd be amazed if it didn't include two factors: (1) the nature of the product being developed; and (2) the unspoken assumptions of the tech industry.

Cars don't have too many integration points
If you make too many comparisons to the same industry, you quickly run into the limits of how comparable it is to other kinds of businesses. Proponents of set-based design, an offshoot of the Lean movement, often cite the car industry as an exemplar. Toyota is where Lean started, and Japanese auto manufacturers push themselves to generate and test multiple design ideas before settling on one. But is it fair to compare Toyota to Oracle?

In one respect, integration, the answer is clearly no. When Toyota designs a car, it doesn't contemplate making big changes to the interfaces with other systems. The nozzle at the gasoline pump is going to be the same in 2011 as it was in 2010. The road will continue to be hard, flat, and friction-friendly. The laws governing the minimum and maximum brightness for headlights aren't going to change.

Migration is also easier. We'll never use the phrase "rip and replace" to describe the process of buying a new car, unless we really hate the upholstery. In contrast, "rip and replace" is a constant challenge for software companies, who have to simultaneously respect the customer's desire to avoid changing other systems, while simultaneously cajoling them into fixing one particular system.

Here's where differences in technology might make it harder for development teams to embrace a set-based design approach. As any web developer can tell you, interface points change all the time. The sheer number of interface points are numerous enough to make any single design a fairly complicated venture. Faced with these sanity-challenging levels of complexity, it's not hard to understand why a VP of Development might not be ready to start implementing set-based design the day after hearing an intriguing talk about it.

Geniuses don't need options
But that's hardly the whole story. After all, if car companies, faced with high prototyping costs, figured out how to do set-based design, surely someone in the technology industry can. (In some cases, they have. See, for example, the way that Singularity coaches Agile teams to explore competing ideas.) Other obstacles must exist, and at least one of them is cultural.

The unspoken assumption of set-based design at companies like Toyota is, Any team can devise the winning idea. Regularly favoring one person's ideas would de-motivate the other teams, decreasing the likelihood of generating lots of competitive ideas, undermining the whole reason to have these teams in the first place.

Many, if not most, technology companies don't operate that way. The founding myth of the technology industry is the genius-as-hero, the Steve Jobs-like person whose commanding intellect overshadows everyone else's. While in truth, tech companies don't operate that way, the reality isn't exactly a level playing field of ideas.

The founders of new tech companies, or of new projects within existing companies, are usually a very small cadre, often as tiny as one or two people. Over time, if their Big Idea proves themselves, these founders move into positions of responsibility. New hires are a necessity, but they are also a source of disruption. The worst among them may not "get" the Big Idea, or fumble their part of its creation. The best among them might have great alternative ideas, but after a certain point, the team has to execute, execute, execute on the original idea.

We can measure how that pressure to execute affects the design process. Over the course of a release, the amount of requirements work decreases steadily. If teams were keeping the door to alternative ideas open for as long as possible, the curve would be flatter in the early stages, dropping off more quickly in the later execution phases. Testing ideas would require continued activity to test ideas and validate assumptions, which means continued activity in the requirements process. . . . But that's not how, on average, teams work today.

Some founders are smart and mature enough to recognize how easily they might transform into the guardian of their own orthodoxy. Even in the best of cases, however, the founder-turned-CTO, or the founder-turned-SVP, faces a tough challenge, remaining open to changes without introducing chaos into a complex development process.

Reform begins at home
Neither of these obstacles are insurmountable. For example, in the last decade, the culture of software engineering has shed a lot of its genius-as-hero baggage.

My one suggestion for the proponents of set-based design is, Stop talking about car companies. Development teams in tech companies need to hear success stories from within their own industry, instead of pinning their hopes on finding more similarities than differences between themselves and auto engineers.