SPIEGEL Interview with Daniel Kahneman Debunking the Myth of Intuition
Part 2: The Pitfalls of Intuition
SPIEGEL: By studying human intuition, or System 1, you seem to have learned to distrust this intuition
Kahneman: I wouldn't put it that way. Our intuition works very well for the most part. But it's interesting to examine where it fails.
SPIEGEL: Experts, for example, have gathered a lot of experience in their respective fields and, for this reason, are convinced that they have very good intuition about their particular field. Shouldn't we be able to rely on that?
Kahneman: It depends on the field. In the stock market, for example, the predictions of experts are practically worthless. Anyone who wants to invest money is better off choosing index funds, which simply follow a certain stock index without any intervention of gifted stock pickers. Year after year, they perform better than 80 percent of the investment funds managed by highly paid specialists. Nevertheless, intuitively, we want to invest our money with somebody who appears to understand, even though the statistical evidence is plain that they are very unlikely to do so. Of course, there are fields in which expertise exists. This depends on two things: whether the domain is inherently predictable, and whether the expert has had sufficient experience to learn the regularities. The world of stock is inherently unpredictable.
SPIEGEL: So, all the experts' complex analyses and calculations are worthless and no better than simply betting on the index?
Kahneman: The experts are even worse because they're expensive.
SPIEGEL: So it's all about selling snake oil?
Kahneman: It's more complicated because the person who sells snake oil knows that there is no magic, whereas many people on Wall Street seem to believe that they understand. That's the illusion of validity
SPIEGEL: which earns them millions in bonuses.
Kahneman: There is no need to be cynical. You may be cynical about the whole banking system, but not about the individuals. Many believe they are building real value.
SPIEGEL: How did Wall Street respond to your book?
Kahneman: Oh, some people were really mad; others were quite interested and positive. It was on Wall Street, I heard, that somebody gave a thousand copies of my book to investors. But, of course, many professionals still don't believe me. Or, to be more precise, they believe me in general, but they don't apply that to themselves. They feel that they can trust their own judgment, and they feel comfortable with that.
SPIEGEL: Do we generally put too much faith in experts?
Kahneman: I'm not claiming that the predictions of experts are fundamentally worthless. Take doctors. They're often excellent when it comes to short-term predictions. But they're often quite poor in predicting how a patient will be doing in five or 10 years. And they don't know the difference. That's the key.
SPIEGEL: How can you tell whether a prediction is any good?
Kahneman: In the first place, be suspicious if a prediction is presented with great confidence. That says nothing about its accuracy. You should ask whether the environment is sufficiently regular and predictable, and whether the individual has had enough experience to learn this environment.
SPIEGEL: According to your most recent book "Thinking, Fast and Slow," when in doubt, it's better to trust a computer algorithm.
Kahneman: When it comes to predictions, algorithms often just happen to be better.
SPIEGEL: Why should that be the case?
Kahneman: Well, the results are unequivocal. Hundreds of studies have shown that wherever we have sufficient information to build a model, it will perform better than most people.
SPIEGEL: How can a simple procedure be superior to human reasoning?
Kahneman: Well, even models are sometimes useless. A computer will be just as unreliable at predicting stock prices as a human being. And the political situation in 20 years is probably completely unpredictable; the world is simply too complex. However, computer models are good where things are relatively regular. Human judgment is easily influenced by circumstances and moods: Give a radiologist the same X-ray twice, and he'll often interpret it differently the second time. But with an algorithm, if you give it the same information twice, it will turn out the same answer twice.
SPIEGEL: IBM has developed a supercomputer called "Watson" that is supposed to quickly supply medical diagnoses by analyzing the description of symptoms and the patient's history. Is this the medicine of the future?
Kahneman: I think so. There's no magic involved.
SPIEGEL: Some say the next blockbuster movie could be predicted by an algorithm, as well.
Kahneman: Why not? The alternative is simply not very convincing. The entertainment industry wastes a lot of money on films that don't work. It shouldn't be that difficult to develop a program that at least doesn't do any worse than the intuitive judgments that govern these decisions now.
SPIEGEL: But most people tend to be hostile to formulas and cold calculations, and many patients prefer a doctor who treats them holistically.
Kahneman: It's a question of what you're used to. So-called "evidence-based medicine" is making progress, and it's based on clear, replicable algorithms. Or take the oil industry. There are strict procedures on deciding whether or not to drill in a specific location. They have a set of questions that they ask, and then they measure. Relying on intuition would be far too error-prone. After all, the risks are high, and there is a lot of money at stake.