Black boxes and embracing failure are united in Tim Harford’s Ted presentation ‘Trial, Error and the God Complex’ (thanks Isley).

He describes the ‘God Complex’ which has been traditionally applied to doctors making life and death decisions with situations intractably complex not just in the biological complexities, but the ethical ones as well. But he applies it to any area of dogmatic certainty over an area of befuddling complexity…

“The God Complex. No matter how complicated the problem, you have an absolutely, overwhelming belief that you are infallibly right in your solution.”

He explores a range of contexts including economics, business, public policy, natural science – “The complexity here is unbelievable…This is perhaps why we find the ‘God Complex’ so tempting. We tend to sort of retreat and say we can draw the picture, we can show the graphs, we get it. We understand how this works. And we don’t. We never do.”

As an illustration of the true impenetrability of many areas, he showed a recent study on the products we consume with the opaquely contorted illustration the study provided.  The graph he showed (see below) is now one of my favourite icons for black box complexity.

But Harford is not just having a go at intellectual arrogance. He is actually prescribing an alternative approach. An antithetical approach to the ‘I am absolutely right’ one. Embracing failure.

“I’m not trying to say we can’t solve complex problems in a complicated world. We clearly can. But the way we solve them is with humility. To abandon the ‘God Complex’ and to actually use a problem solving technique that works. Now, you show me a successful complex system, and I will show you a system that has evolved through trial and error.”

He notes that one of the reasons that the USA grew to become the largest economy in the world is that “10% of businesses fail every year.” He recounted the story of Unilever who were stymied in their engineering efforts to ‘design’ the best nozzle possible until they resorted to ‘trial and error’. After making an array of random changes to the design and testing for the best, they evolved a truly superior nozzle of which they said in the end, “We have no idea why it works.”

Embracing failure is not a novel topic and the domain of ‘complexity’ (black boxes) actually has a scientific discipline about it. So is Harford (and me) just rehashing the blindingly obvious? Well, he has a fine response to the need to continue examining and preaching the embrace of failure…

“I will admit that it’s ‘obvious’ when schools start teaching children that there are some problems that don’t have a correct answer…When a politician stands up [and says] ‘I have no idea how to do it.’ “

That’s sounds like a good mission statement for this blog here and when I will stop writing about embracing failure.

His final words said it all about complexity and failure and the need to keep studying them.  It’s not just hard to embrace failure, it’s hard to do it well.  Harford says, ‘It is very difficult to make good mistakes.”

Cesar Hidalgo graph