How do intuition and analysis compare or differ? - Chapter 5

What does claim 2a entail?

Claim 2a: Successful decision makers rely on logic and statistics, instead of intuition.

In 1954 Paul Meehl published a book describing 18 studies that showed the limitations of human judgment. These studies compared the judgments of professionals against statistical rule-based predictions about parole violations, success in pilot training and academic success. Meehl’s work has been repeated in a number of different areas with the same outcome. The statistical methods are more accurate than the expert judgments. In 2000, Grove et al. published a review of studies comparing clinical judgments using experience with mechanical judgments using statistics.

Phil Tetlock has demonstrated that experts in history and political science are not particularly accurate about predicting world events. Tetlock and others have shown that we can’t trust the so-called experts. And we certainly can’t trust our own judgments and intuitions.

We know that people are getting seriously worried about decision biases and the weakness of experts, because those subjects have hit the sports pages. The message of many people is clear: For any important judgment we should call in the decision analysts and operations researchers, not the experts.

What about the disclaimer?

We need both intuition and analysis. Either one alone can get us into trouble. Experts certainly aren’t perfect, but analyses can also fail.

Intuition isn’t a magical property. Intuition are ways we use our experience without consciously thinking things out. Intuition includes tacit knowledge that we can’t describe. It includes our ability to recognize patterns stored in memory.

In this section will be told some of the limits of relying too heavily on logic and statistics:

Meehl e.g. took advantage of the fact that experts don’t always combine the evidence the best way, and sometimes are inconsistent. In contrast, the analyst's formulas are always consistent and should always come up with a better answer. Sometimes experts give too much emphasis to a factor, whereas the objective data can provide a more accurate weighting.

Therefore, the decision analysts could outperform the experts only after they identified the basic factors the experts were using.

To make predictions we have to do two things:

1. Collect the data

2. Combine the data

Meehl and his followers haven’t shown that expert intuitions are useless. They have just shown that statistical formulas for combining the data can be more accurate than the estimates made by experts. Collecting the data often requires skill, tacit knowledge, intuition and expertise. Next comes the activity of combining all the data to make a prediction.

Although experts aren’t infallible, they are able to amaze us with their performance. For example, most of what differentiates skilled from unskilled chess players is their tacit knowledge, not their ability to calculate move quality. Systematic analysis is important in chess, but intuitive skills and tacit knowledge seem to be more critical.

The claim that we should trust logic and statistics over intuition is wrong because we need both logic and intuition. Either one, by itself, can get us in trouble. The idea of emphasizing logic and suppressing intuition has led to its share of blunders. It gets repeated so often that we might as well examine its shortcomings. Analytical rigor doesn’t guarantee success.

One of the most famous blunders that resulted from a reliance on statistics and analysis is the collapse of the investment company Long-term Capital Management. In this case, rigorous use of systematic analysis still resulted in a collapse. We saw a repeat of the LTCM episode during the Wall Street collapse in 2008. Once again sophisticated investors relied on statistical analyses to assure themselves and others that they were in good shape, and once again they almost succeeded in destroying the US economy.

These examples illustrate why a reliance on analysis and statistics won’t prevent blunders.

What causes the failures of systematic analysis? Overthinking!

Logic and statistics are useful, but they aren’t sufficient for making good decisions, and they sometimes result in worse decisions. Conscious thought doesn’t necessarily help us to make better choices. Any systematic strategy exposes us to the risks of overthinking.

What causes overthinking? One possibility is that we can think of only a few things at a time, so our cognitive bandwidth prevents us from considering all the variables when making choices. Unconscious thought lets us integrate lots of information. Another possibility is that when we consciously deliberate we may over-emphasize some of the features. We may ignore or downplay important features that don’t have convenient verbal tags.

Conscious deliberation presses us to view the world through a keyhole – the limitation of attention. What gets missed by this keyhole view? Often we fail to notice the context. Without context, we can’t reliably make sense of data, particularly in complex settings. The notion of favoring logic and statistics over intuition leads to overthinking. It pretends that we can make sense of facts without taking context into account.

Research on people with brain damage shows the effect of overthinking. Damasio and colleagues studied patients who suffered damage to brain areas that connect emotions with decisions. Such patients make decisions without any emotional/intuitive inputs. The results aren’t pretty.

Why consider the replacement for the claim?

The replacement for the claim that we should rely on logic and statistics over intuition is straightforward: we need to blend systematic analysis and intuition. Neither gives us a direct path to the truth. Each has its own limitations.

Analytical and statistical methods can pick up subtle trends in the data that our intuitions would miss. They can show us when the regularities we think we see are really just chance connections. They can help us appraise the size of different effects so we can take the effects into account more accurately. On the other hand, analytical methods often miss the context of a situation, and they can result in misleading recommendations.

Intuitive judgments reflect the experiences we’ve had and help us respond quickly to situations. They are sensitive to context and nuance, letting us read situations and also read other people.

We can make successful decisions without analytical methods, but we can’t make good decisions without drawing only on our intuitions.

Research showing that a statistical formula can outperform an expert doesn’t mean that the expert is flawed. It just means that the statistics do better in certain types of circumstances. Analytical methods can sometimes outperform experts, particularly if these methods improve on the judgments made by experts.

The examples in this section showed that we can’t rely on the claim that successful decision makers must rely on logic and statistics instead of intuition. It always depends on the situation which method/analysis/statistics or intuition you have to use.

The claim that successful decision makers rely on logic and statistics, instead of intuition matters because systematic analysis may work for well-ordered tasks. But it runs into difficulty in complex settings, and it leads to overthinking. We can make very poor decisions when we rely solely on systematic analysis.

VaR stands for Value and Risk, it calculates how much to hold in reserve. The problem with VaR was that it was insensitive to the pending collapse of the housing market, and the loss of liquidity that followed.

Claim 2a: Successful decision makers rely on logic and statistics, instead of intuition.

In 1954 Paul Meehl published a book describing 18 studies that showed the limitations of human judgment. These studies compared the judgments of professionals against statistical rule-based predictions about parole violations, success in pilot training and academic success. Meehl’s work has been repeated in a number of different areas with the same outcome. The statistical methods are more accurate than the expert judgments. In 2000, Grove et al. published a review of studies comparing clinical judgments using experience with mechanical judgments using statistics.