The September Issue of Harvard Business Review is focused on Managing Complexity. A theme throughout the issue is that "rare events can be more significant than average ones -- and may occur more often than we think".
To really understand the rarity and impacts of events, the featured articles speak to switching predictive models, mixing data horizons (lagging, current, leading) and deeper study of outlier events.
An article I found especially interesting was on IT Failure: Why Your IT Project May Be Riskier Than You Think. The authors highlight a plethora of high profile, high-dollar IT nightmares, including Levi Strauss, EADS, Kmart, and Airbus.
As someone active in large IT projects, reading about hundreds of millions, or even billions, of dollars of IT investment wasted, and plummeting a company, or government, into oblivion is stomach-churning. In fact, I had to skim some of the gory details.
However, I took great interest in the authors' findings on the True IT Pitfall: [emphasis is mine]
When we broke down the projects’ cost overruns, what we found surprised us. The average overrun was 27%—but that figure masks a far more alarming one.
Graphing the projects’ budget overruns reveals a “fat tail”—a large number of gigantic overages. Fully one in six of the projects we studied was a black swan, with a cost overrun of 200%, on average, and a schedule overrun of almost 70%.
This highlights the true pitfall of IT change initiatives: It’s not that they’re particularly prone to high cost overruns on average, as management consultants and academic studies have previously suggested. It’s that an unusually large proportion of them incur massive overages—that is, there are a disproportionate number of black swans.
How are you assessing IT success and failure? Are you scoring on a curve -- studying average performance and overruns? How is that working out? Do you find, as the authors did, that the performance (or lack thereof) of the outliers is killing your metrics, or worse, your company?
Well, using their new research model, the authors developed new thinking on IT investment guidelines:
"Leaders should ask themselves two key questions as part of IT black swan management: First, is the company strong enough to absorb the hit if its biggest technology project goes over budget by 400% or more and if only 25% to 50% of the projected benefits are realized?
Second, can the company take the hit if 15% of its medium-sized tech projects (not the ones that get all the executive attention but the secondary ones that are often overlooked) exceed cost estimates by 200%?
Do the guidelines seem alarmist? Perhaps, but as the authors close:
"These numbers may seem comfortably improbable, but, as our research shows, they apply with uncomfortable frequency."
Study your outliers, rather than becoming one.
