“Imagine that you are a hominid walking along the savanna of an African valley three million years ago. You hear a rustle in the grass. Is it just the wind or is it a dangerous predator? …
If you assume that the rustle in the grass is a dangerous predator but it turns out that it is just the wind, you have made what is called a Type I error in cognition, also known as a false positive….
If you assume that the rustle in the grass is just the wind but it turns out that it is a dangerous predator, you have made what is called a Type II error in cognition, also known as a false negative…”
Michael Shermer, The Believing Brain
If you make a Type 1 error, you pay by spending more energy. If you make a Type 2 error, ‘you are lunch.’ When the cost of a Type 2 error > Type 1 error, people are more likely to theorize.
Setting aside the fact that the typology of errors is only clear conditional on the consequences—-“Is it … the wind or is it a dangerous predator” can be seen as two postulates and each can have their own Type 1 error—there is something to the idea of patternicity, the human proclivity of seeing patterns in noise and making causal linkages.
Our theories don’t come from nothing. There is method to our inference. Here are some common ways
we infer causation and attendant problems with our methods of inference:
7 Habits of Successful People: We have all seen business books with such titles. The underlying message of these books is: adopt these habits, and you will be successful too!… The upshot is that when you select on the dependent variable, i.e., only look at cases where the variable takes certain values, e.g., only look at the habits of financially successful people, even correlation is not guaranteed. This means that you don’t even get to mock the claim with the jibe that “correlation is not causation.”
When Experts Are Wrong
I was at a panel for the National Institutes of Health evaluating grants. … Then someone at the table – and I couldn’t believe this – said, ‘My uncle smoked a hookah pipe all his life, and he lived until he was 90 years old.’ And I had a sudden flash of insight, which was this. Suppose you have something that actually kills half the people. Even if you’re a heavy smoker, your chance of dying of lung cancer is not 50%, so therefore, even with something as extreme as smoking and lung cancer, you still have lots of cases where people don’t die of the disease.
Here’s another way to come to the answer. The average years lost to smoking is about 7. (Compare this to Covid-19 — a conservative estimate (upper bound) is 9 years. See here