23 March 2012

You never know with these things when you’re trying something new what can happen. This is all experimental.

Richard Branson

When you’re analyzing a hypothesis using statistics, there are two types of errors you can make, called a type I or type II error. The first is when you accept a hypothesis that is false, and the second is when you reject a hypothesis that is true.

As an example, say we were testing whether smoking cigarettes makes you healthier. If we came to the conclusion that indeed, smoking does make a person healthier, we would be making a type I error, accepting a hypothesis that is false. If we were studying whether wearing a seat belt kept a person safe when driving, and found it could not be proven that wearing a seatbelt was safer, we would be making a type II error, rejecting something that is true.

Our society is obsessed with not making type I errors, and doesn’t care much about type II errors.

A big part of this is because type I errors can be measured much more clearly than type II errors. For example, if the FDA approves a drug that goes on to accidently kill a bunch of people, we know exactly how many people died because of the FDA’s mistake. Nobody thinks for a second about the type II errors. These are all the people who have died because the FDA makes it so hard to develop new drugs. For all we know, the FDA is killing fifty times more people than it is saving. However, we can’t count the type II errors, the people who would be alive if the FDA made it easier to develop new drugs. We can only count the people who died from drugs that shouldn’t have been released. So we continue with the current system.

This focus on type one errors permeates all of our lives. From business to politics, people are terrified of doing something wrong. As a result, they avoid making as many type I errors, but make far more type II errors.

Imagine a society where instead of needing to prove that something wouldn’t fail, you only needed to prove that it could work. People would take a lot more risks, people would succeed a lot more, and the failure rate wouldn’t even be that different. The only difference would be that the failures would skew more towards the visible type I errors instead of the invisible type II errors that people are currently making.

The crazy thing about this is that, as we get older, we regret our type II errors much more than the type I errors. We regret all the opportunities lost, the risks we never took, and the chances we never pursued. We care much less about the times we tried and failed. At least we tried.

So, armed with the knowledge that, in the future, you will regret your type II errors more than your type I errors, go out and do the things that you think will work, even if they have a chance of failing.