I recently read Radical Uncertainty by the economists John Kay and Mervyn King. A few notes, then a bunch of block quotes that stood out to me…
Notes
- I strongly disagree in practice with their argument against probabilistic reasoning. Only economists who’ve spent time in finance and business schools could possibly think that probability and expected value-based thinking were overvalued; in practice they seem far undervalued. Kay and King tell the story of Obama’s advisors telling him numerically what they think the chances are that Osama bin Laden is in the house — a scene Phil Tetlock describes in his book as a model case of probabilistic reasoning. Kay and King think this is useless and actively damaging: The analysts are using numbers to hide that they just don’t know. I think Tetlock has it right here, and that summarizes how I felt about most of the book.
- That said, Kay and King’s basic point that sometimes it’s pointless to put a probability on something and we should just admit “I have no idea” — that seems right. What will US GDP be in the year 5,000? I’m not sure it’s helpful to try and put numbers and confidence intervals to that sort of question.
- They also stick up for the art of reasoning to the best explanation (abductive reasoning), and they frequently come back to the question, borrowed from a business professor: “What is going on here?” Again, overall I’m mostly skeptical. The evidence seems to suggest this is an overvalued starting point — we’re more likely to zoom too far in than too far out, which is why it’s often wise to step back, look for data and take the “outside view.” But it’s also possible to go too far in that direction and pay too little attention to what’s unique about a single case (I’ve done it plenty). And they’re right that explaining individual cases requires judgment. Sometimes broader data is nonexistent; sometimes conditions are such that broader comparison sets aren’t useful; sometimes diving into the details is what’s required to truly understand a topic. “What is going on here?” is a good animating question.
- Their dismissal of behavioral economics was unpersuasive to me, but the discussion of narratives in decision making was intriguing. They argue that people craft “reference narratives” about how they hope or expect their lives to go, and then they make decisions so as to bring reality as closely into line with the narrative as possible. I was left wanting more on this subject.
Quotes
Plato sought and found truth in logic; for him there ws a sharp distinction between truth, which was axiomatic, and probability, which was merely the opinion of man. In premodernt hought there was no such thing as randomness, since the course of events reflected the will of the gods, which was determinate if not fully known. The means of resolving uncertainty was not to be found in mathematics, but in a better appreciation of the will of the gods.
p. 54
At the end of the nineteenth century, Charles Sanders Peirce, a founder of the American school of pragmatist philosophy, distinguished three broad styles of reasoning. Deductive reasoning reaches logical conclusions from stated premises… Inductive reasoning … seeks to generalise from observations, and may be supported or refuted by subsequent experience… Abductive reasoning seeks to provide the best explanation of a unique event… Deductive, inductive, and abductive reasoning each have a role to play in understanding the world, and as we move to larger worlds the role of the inductive and abductive increases relative to the deductive. And when events are essentially one-of-a-kind, which is often the case in the world of radical uncertainty, abductive reasoning is indispensable.
p. 137-138
Kahneman offers an explanation of why earlier and inadequate theories of choice persisted for so long — a ‘theory-induced blindness: once you have accepted a theory and used it as a tool in your thinking, it is extraordinarily difficult to notice its flaws’. We might say the same about behavioural economics. We believe that it is time to move beyond judgmental taxonomies of ‘biases’ derived from a benchmark which is a normative model of human behaviour deduced from implausible a priori principles. And ask instead how humans do behave in large worlds of which they can only ever have imperfect knowledge.
p. 147-148
In many colleges, students of law are taught to follow a structure described as IRAC: issue, rule, analysis, conclusion. The impressive skill of a top lawyer is to identify the issue; to give structure to an array of amorphous facts, freqnetly presented in a tendentious manner — that is to establish ‘what is going on here.’ … IRAC is a useful acronym for anyone engaged in the search for practical knowledge. In the legal context it leads naturally to thetow next stages of effective practical reasoning — communication of narrative and challenge to the prevailing narrative.
p. 194-195
The legal style of reasoning, essentially abductive, involves a search for the ‘best explanation’ — a persuasive narrative account of events relevant to the case. The great jurist and US Supreme Court Justice Oliver Wendell Holmes Jr. began his exposition of legal philosphy with the observation that ‘The life of the law has not been logic; it has been experience… The law embodies the story of a nation’s development through many centuries, and it cannot be dealt with as if it contained only the axioms and corollaries of a book of mathematics.’
p. 211
A ‘good’ explanation meets the twin criteria of credibility and coherence. It is consistent with (most of) the available evidence and the general knowledge available to judges and jurors… A good explanation demonstrates internal coherence such that, taken as a whole, the account of events makes sense. The best explanation can be distinguished from other explanations and is not compatible with these other explanations. Statistical reasoning has its place but only when integrated into an overall narrative or best explanation.
p. 212
In pressing the case for probabilistic reasoning, Philip Tetlock and Daniel Gardner, the appraisers of forecasting and architects of the ‘good judgment project’, argue that ‘For decades, the United States had a policy of maintaining the capacity to fight two wars simultaneously. But why not three? or four? Why not prepare for an alien invation while we are at it? The answer hinges on probabilities.’ No it doesn’t. There is no basis on which one can form probabilities of an invasion by aliens… The attempt to construct probabilities is a distraction from the more useful task of trying to produce a robust and resilient defence capability to deal with many contingincies, few of which can be described in any but the sketchiest of detail.
p. 294-295
The mark of science is not insistence on deductive reasoning but insistence that observation trumps theory, whatever the purported authority supporting the theory.
p. 389
Acknowledging radical uncertainty does not mean that anything goes. Look to the future and contemplate the ways in which information technology will be deployed in the coming decades, or consider the ways in which the growth of prosperity and political influence in Asia will affect the geopolitical balanc.e They are all things about which we can know something, but not enough; we see though a glass, darkly. We can construct narratives and scenarios to describe the ways in which technology and global politics might develop in the next twenty years; but there is no sensible way in which we can refine such dialogue by attaching probabilities to a comprehensive list of contingiences. We might, however, tlalk coherently about the confidence we place in scenarois and the likelihood that they will arise. As we have ephasised, the words ‘confidence’, ‘likelihood’ and ‘probablitly’ are often used interchangeably but they have different meanings. We do not enhance our understanding of the future by inventing facts and figures to fill in the inescapable gaps in our knowledge. We cannot rely on forecasts in planning for the future…. We are not afraid to answer these questions with ‘we do not know.’
p. 403
beyond the times 