January 28th was the anniversary of the Space Shuttle Challenger disaster. The Rogers Commission detailed the official account of the disaster, laying bare all of the failures that lead to the loss of a shuttle and its crew. Officially known as The Report of the Presidential Commission on the Space Shuttle Challenger Accident - The Tragedy of Mission 51, the report is five volumes long and covers every possible angle starting with how NASA chose its vendor, to the psychological traps that plagued the decision making that lead to that fateful morning. There are many lessons to be learned in those five volumes and now, I am going to share the ones that made a great impact on my approach to risk management. The first is the lesson of overconfidence.
In the late 1970’s, NASA was assessing the likelihood and risk associated with the catastrophic loss of their new, reusable, orbiter. NASA commissioned a study where research showed that based on NASA’s prior launches there was the chance for a catastrophic failure approximately once every 24 launches. NASA, who was planning on using several shuttles with payloads to help pay for the program, decided that the number was too conservative. They then asked the United States Air Force (USAF) to re-perform the study. The USAF concluded that the likelihood was once every 52 launches.
In the end, NASA believed that because of the lessons they learned since the moon missions and the advances in technology, the true likelihood of an event was 1 in 100,000 launches. Think about that; it would be over 4100 years before there would be a catastrophic event. In the end, Challenger flew 10 missions before it’s catastrophic event and Colombia flew 28 missions before its catastrophic event, during reentry, after the loss of heat tiles during take off. During the life of a program that lasted 30 years, they lost two of five shuttles.
Emergency management professionals say, “The plan is useless, but the planning is priceless.” There is a lesson in there for risk managers and it’s about the value of scenario modeling.
The Federal Emergency Management Administration (FEMA) conducted a study to determine the likelihood and impact of a hurricane hitting New Orleans. FEMA assembled the paramedics, fire department, emergency room doctors, parish officials, and other responders in a hotel in New Orleans for "Hurricane Pam". Their goal was to plan for the worst-case scenario. The group was given the following scenario:
A slow moving, category-3 hurricane would directly hit New Orleans.
The storm surge would cause the levees to top, but not break.
The National Weather Service showed how the storm would form, what track it would take and what parishes would be effected.
Before joining Forrester, I ran my own consulting firm. No matter how ridiculous the problem or how complicated the solution, when a client would ask if I could help, I would say yes. Some people might say I was helpful, but I was in an overconfidence trap. There was always this voice in the back of my mind that would say, “How hard could it be?” Think of the havoc that kind of trap can have on a risk management program. If any part of the risk program is qualitative, and you are an overconfident person, your risk assessments will be skewed. If you are in an overconfidence trap, force yourself to estimate the extremes and imagine the scenarios where those extremes can happen. This will help you understand when you are being overconfident and allow you to find the happy medium.
Have you ever padded the budget of a project “just to be safe”? I hate to tell you this, but you are in the prudence trap. By padding the project budget, you are anticipating an unknown. Many other managers in your company may be using the same “strategy.” But the next time you do a project like this, you will pad the budget again, because the inherent uncertainty is still there. The easiest way to keep your risk management program out of the prudence trap is to never adjust your risk assessments to be “on the safe side,” There is nothing safe about using a psychological trap to predict risk.