The iron rule of explanation

Last year I posted about an Aeon article by the philosopher Michael Strevens, about the scientific method. That was based on his book The Knowledge Machine which I’ve since read. I’ve been posting a lot in the past year about theory vs. evidence and epistemology in general and I really recommend this book. Of all the books, articles, and courses I’ve looked at on the philosophy of science this is my favorite.

Strevens takes on several related questions: Why did it take so long for the scientific method to appear? What even is the scientific method? And is there any sense in which it is “objective” or is science an inherently subjective enterprise?

The core to his answer is what he calls the “iron rule of explanation”:

Here, then, in short, is the iron rule:

1. Strive to settle all arguments by empirical testing.

2. To conduct an empirical test to decide between a pair of hypotheses, perform an experiment or measurement, one of whose possible outcomes can be explained by one hypothesis (and accompanying cohort) but not the other.

p. 96

But the process of interpreting the results of empirical tests is subjective, he argues:

For these reasons, Popper is now thought by most philosophers of science to fall short of providing a rule for bringing evidence to bear on theories that is both fully objective and adequate to science’s needs. What kind of rule might do better? There is philosophical consensus on this matter too–and the answer is none. An objective rule for weighing scientific evidence is logically impossible.

p. 79

Interpreting evidence requires subjective assessment of the plausibility of both an explanation and its attending assumptions. Yet, despite this subjectivity, Strevens argues that science tilts, in the long run, toward “Baconian convergence” where scientists over time do agree more and more on the theories that best explain all the evidence they’ve created.

The iron rule works with four related innovations:

1. A notion of explanatory power on which all scientists agree

2. A distinction between public scientific argument and private scientific reasoning

3. A requirement of objectivity in scientific argument (as opposed to reasoning)

4. A requirement that scientific argument appeal only to the outcomes of empirical tests (and not to philosophical coherence, theoretical beauty, and so on)

p. 119

So what is the limited sense in which scientific publishing is “objective”?

When a scientific paper is written, the grounds of many of the experimenter’s crucial assumptions, being partially or wholly subjective, are cut away. What is left are only observation reports, statements of theories and other assumptions, and derivations that connect the two.

p. 161

There’s a ton more of interest in the book, including his argument that this whole process is, while useful, in some sense irrational. And a discussion of whether beauty is a useful criteria for theory. There’s a bunch of good material situating this whole argument within debates about both the philosophy and sociology of science, and some brief discussion of why science arose when it did.

But one of my favorite paragraphs is right in the beginning, right after he introduces the iron rule “compelling scientists to conduct all disputes with reference to empirical evidence alone.”

How can a rule so scant in content and so limited in scope account for science’s powers of discovery? It may dictate what gets called evidence, but it makes no attempt to forge agreement among scientists as to what the evidence says. It simply lays down the rule that all arguments must be carried out with reference to empirical evidence and then steps back, relinquishing control. Scientists are free to think almost anything they like about the connection between evidence and theory. But if they are to participate in the scientific enterprise, they must uncover or generate new evidence to argue with. And so they do, with unfettered enthusiasm.

p. 7

I have a lot more to say about the book and how it applies to social science as well as to the more practical reasoning employed by forecasters and other analysts, but I’ll leave it there for now. The book is worth your time.