Lies, Damn Lies, Security Metrics, And Baseball

The legendary British Prime Minister Benjamin Disraeli is said to have noted that “There are lies, damn lies, and statistics.” Much of the technology world is focused on statistics and metrics. You’ve often heard it said, “If I can’t measure it, it doesn’t exist.” Known as the McNamara fallacy — named after the business tycoon turned Vietnam-era Secretary of Defense — this famous idea failed miserably as a strategy. While it sounds good to the CEO’s ears, there is a corollary bubbling up below him that implicitly states that “If my boss wants to measure something that doesn’t exist, then I’ll invent it!”

This is especially true whenever leadership is disconnected with the field. As big data gets big buzz, the promises will become self-fulfilling. David Hackworth, reportedly one of the most decorated soldiers in the Vietnam war, explores this premise in Sam Adams’ book “War of Numbers: An Intelligence Memoir,” with an introduction by Hackworth. I met Hackworth in the late 1990s on the set of “Joe Bob Briggs Drive-In Theater,” where I was the show’s engineer. Hackworth didn’t look like a war hero. While his stature was Audie Murphy-ish, his legend was huge. I remember him as always smiling and joking — cheerful despite the way he had been treated by his superiors in the military. His mantra was how the Vietnam War was mismanaged by disengaged leaders who were more focused on counting dead bodies than they were on the strategic aspects of fighting a war. As he clearly demonstrates in his books, this was a disastrous approach, as field officers like captains and lieutenants were given incentives to invent dead enemies to increase the body count. I fear that this may happen in the infosec world if we become more focused on metrics than strategy and on providing glowing reports to our superiors than truth-telling.

Warning: CISOs should beware of metrics.

Too much focus on them will force your teams to manage to those metrics instead of telling you the truth related to your organization’s security posture. Hackworth was a truth-teller and it cost him his career. He never rose above colonel and was eventually drummed out of his beloved military, but history has vindicated him. The eternal reputations of his superiors are tarnished. Generals such as Westmoreland and Abrahams, whose aggressiveness in World War II earned them accolades and rank, are now relegated to the waste bin of history.

Ultimately, metrics must be balanced with your “gut.” This lesson was brought home to me on a recent flight. Evidently the gentleman sitting next to me was famous, because everyone else coming down the aisle were pointing, staring, and whispering to each other. Eventually I said, “Sir, clearly you are famous and I apologize for not recognizing you.” He waved that off with a smile and a flick of his fingers. It turns out this gentleman manages a major league baseball team. Not being much of a baseball fan, he proceeded to educate me about the finer points of the game. During our conversation, I mentioned the movie “Moneyball,” which was in theaters at the time. He scoffed at the concept. “This game,” he said, “is all about the gut. You have to have an instinct for the game.” He said that while the whole Sabermetrics/Moneyball thing worked for a short period of time, it didn’t take the rest of the league long to figure it out. It was a short-term solution that glossed over the lack of gut. These metrics were unsustainable once the opponents understood them and he dismissed the Oakland A’s as an also-ran once again.

I had a moderate interest in baseball on that day, as I live in Dallas and the Texas Rangers were in the fifth game of the American League Championship Series that night. So I asked this baseball legend to tell me what would happen in that evening’s game. To demonstrate the importance of “gut,” he predicted the Rangers’ destiny. “The Rangers will lose tonight,” he said, “because Justin Verlander is pitching. They’ll win the sixth game and go on to the World Series where they will lose in seven.” That’s exactly what happened. He didn’t input any numbers into a spreadsheet to find this out — in fact, he didn’t even have a computer with him. He just checked his gut. It knew the answer.

Somewhere in your organization is someone with gut — someone who can look at a problem and intuitively understand it at its deepest level and probably solve it. Don’t get so caught up in measuring things that you don’t do a “gut-check” once in a while. Find the David Hackworths on your team and listen to them. They’ll tell you the truth. They won’t pull things out of the ether so they can be “measured.” Yes, sycophants are more fun, but they won’t keep your organizations from being hacked.

I say all of this because I will be in Austin this coming weekend for South by Southwest Interactive. I’m on a panel with Andrew Hay and Mark Seward called “Big Data Smackdown on Cybersecurity.” Don’t get me wrong. I’m all about big data. It will contain lots of valuable information that IT and security pros can use. But I worry that our love affair with big data will blind us to the obvious, much the same way the body count emphasis blinded our military leaders and kept them from actually winning the Vietnam war.

At the end of the day, my security executive friend, listen to your gut. It won’t let you down.

If you’re in Austin for SXSW this weekend, be sure to come to our session.