Forecasting update

In February, I recapped my track record as a forecaster, going back to 2015. I’m a bit more than halfway through my first season as a “Pro” on the INFER forecasting platform so I thought I’d post an update.

  • 64 questions have resolved. I’ve forecast on 7 of them.
  • Of users who’ve forecast on at least five resolved questions this season (the platform’s default leaderboard cutoff) I’m ranked 66 out of 280 (76th percentile). My percentile is basically the same if you include anyone who’s forecast on at least one resolved question.
  • For the all-time leaderboard, among anyone with five resolved questions since 2020, I’m currently 73 out of 558, (87th percentile).
  • My best performance year to date was on a forecast of venture capital. My worst on Intel’s Q2 revenue.

Prediction and resilience

In Radical Uncertainty, the authors make a common argument: Instead of trying to predict the future, you should prepare for a wide range of scenarios, including ones you can’t even fully articulate. You should become more robust or resilient:

The attempt to construct probabilities is a distraction from the more useful task of trying to produce a robust and resilient defence capability to deal with many contingincies, few of which can be described in any but the sketchiest of detail.

This might make sense as an argument against certain types of forecasting exercises that organizations undertake. But it seems to me to still clearly require accurate prediction.

They’re essentially shifting the necessary prediction from the likelihood of a specific future outcome (will there be a recession, for example) to a prediction of the causal effect of a certain action across a range of unspecified outcomes.

For example, one way to be more resilient as a company is to hold more cash. Cash keeps your options open, so instead of trying to predict exactly where the economy is going, hold more cash.

But does that get you away from predictions and forecasting?

Not as far as I can tell. You’re basically predicting that, all else equal, your firm will do better (by whatever metrics) if you increase your cash holdings than if you don’t. You’re making a prediction about a causal effect, rather than about a specific external scenario. Prediction, though, is still a key part of the enterprise.

In their book Prediction Machines, Agarwal, Gans, and Goldfarb define prediction as “using information you have to generate information you don’t have.” They’re using the word more broadly than just predicting the future, but that definition has value even if we add “about the future” to the end of it.

The art of decision making isn’t about trying to predict everything that might have an influence on you. It’s that you use the information you have to generate information you don’t have; you make the predictions that seem most useful, that have the highest payoff.

Sometimes that is straightforward forecasting of a scenario: Will there be a recession next year? Sometimes it’s predicting the effect of a choice you’re considering, ahead of time: Will we be more likely to succeed if we do X to improve our cash holdings?

The difference isn’t that one of these involves prediction and the other says prediction is impossible. Both are bets about the future. The difference is in their potential payoff. Does the information you have give you any purchase on the question? Maybe your information suggests that recession forecasting is borderline impossible but there’s good evidence on the causal effects of increasing cash holdings. You’re likely to make a better prediction in the latter case.

Use the information you have to generate information you don’t have. That takes time and effort — to collect information, to analyze it, to apply it to the decision at hand. The question when you’re considering the value of a forecast isn’t Can I avoid making predictions here? It’s Is the effort I’ll put into this prediction the best use of my time given what I’m trying to achieve?

The German labor market

A paper by an MIT economist explains its unique features…

Germany has less low-wage work and less inequality than the US, but more flexibility and lower unemployment than France.

Germany—the world’s fourth largest economy—has remained partially insulated from the growing labor market challenges faced by the United States and other high-income countries. In many advanced economies, the past few decades have seen sustained increases in earnings
inequality, a fall in the labor share, the disappearance of “good jobs” in manufacturing, the rise of precarious work, and a deterioration in the power of organized labor and individual workers. These developments threaten to prevent economic growth from translating into shared prosperity. Figure 1 shows that compared to the United States, German organized labor has remained strong. Half of German workers are covered by a collective bargaining agreement, compared to 6.1 percent of private-sector Americans (BLS, 2022). Trust in unions is almost twice as high in Germany compared to the US. Employees in Germany work fewer hours, the country’s
low-wage sector is 25 percent smaller, and labor’s share of national income is higher. The German manufacturing sector still makes up almost a quarter of GDP (compared to 12 percent in the US). Germany has one of the highest robot penetration rates in the world (IFR, 2017)—yet in contrast to the US (Acemoglu and Restrepo, 2020), robotization has not led to net employment declines in Germany, especially in areas with high union strength
(Dauth, Findeisen, Suedekum, and Woessner, 2021). At the same time, relative to other OECD countries—many of which, like France or Italy, have maintained even higher collective bargaining coverage through more rigid bargaining systems—the German labor market features low unemployment and high labor force participation (though also a larger low-wage sector).

Bargaining is mostly (but not entirely) at the sectoral level rather than the firm level.

The first pillar is the sectoral bargaining system. In Germany, unions and employer associations engage in bargaining at the industry-region level, leading to broader coverage than in the US. Meanwhile, partial decentralization of bargaining to the firm level—through flexibility provisions in sectoral agreements, or direct negotiations between individual firms and sectoral unions—gives firms space to adapt to changing circumstances. However, this flexibility has also resulted in a gradual erosion of bargaining coverage.

Workers have multiple channels to share their perspectives in firms’ decision making.

The second pillar of the German model is firm-level codetermination. Workers are integrated into corporate decision-making through membership on company boards and the formation of “works councils,” leading to ongoing cooperative dialogue between shareholders,
managers, and workers. Overall, the German model combines centralized “social partnership” between unions and employer associations at the industry-region level with decentralized mechanisms for local wage-setting, dialogue, and customization of employment conditions.

There’s a tradeoff between flexibility and collective bargaining. Germany’s balance between them has evolved over time.

A recurrent theme in our discussion of the German model will be a tension at the heart of the model: between firms’ flexibility and workers’ collective bargaining strength. Since the 1990s, the model has become more decentralized and flexible. This evolution has arguably contributed to reductions in unemployment and increases in economic growth, but has
entailed a substantial erosion of collective bargaining and works council coverage (as Figure 2 illustrates) and a weakening of bargaining agreements. This erosion may explain Germany’s slowly increasing—and perhaps underappreciated—exposure to the afflictions suffered by
other developed-world labor markets: rising wage inequality and the spread of low-wage, precarious jobs.

Notes & quotes: ‘Radical Uncertainty’

I recently read Radical Uncertainty by the economists John Kay and Mervyn King. A few notes, then a bunch of block quotes that stood out to me…

Notes

  • I strongly disagree in practice with their argument against probabilistic reasoning. Only economists who’ve spent time in finance and business schools could possibly think that probability and expected value-based thinking were overvalued; in practice they seem far undervalued. Kay and King tell the story of Obama’s advisors telling him numerically what they think the chances are that Osama bin Laden is in the house — a scene Phil Tetlock describes in his book as a model case of probabilistic reasoning. Kay and King think this is useless and actively damaging: The analysts are using numbers to hide that they just don’t know. I think Tetlock has it right here, and that summarizes how I felt about most of the book.
  • That said, Kay and King’s basic point that sometimes it’s pointless to put a probability on something and we should just admit “I have no idea” — that seems right. What will US GDP be in the year 5,000? I’m not sure it’s helpful to try and put numbers and confidence intervals to that sort of question.
  • They also stick up for the art of reasoning to the best explanation (abductive reasoning), and they frequently come back to the question, borrowed from a business professor: “What is going on here?” Again, overall I’m mostly skeptical. The evidence seems to suggest this is an overvalued starting point — we’re more likely to zoom too far in than too far out, which is why it’s often wise to step back, look for data and take the “outside view.” But it’s also possible to go too far in that direction and pay too little attention to what’s unique about a single case (I’ve done it plenty). And they’re right that explaining individual cases requires judgment. Sometimes broader data is nonexistent; sometimes conditions are such that broader comparison sets aren’t useful; sometimes diving into the details is what’s required to truly understand a topic. “What is going on here?” is a good animating question.
  • Their dismissal of behavioral economics was unpersuasive to me, but the discussion of narratives in decision making was intriguing. They argue that people craft “reference narratives” about how they hope or expect their lives to go, and then they make decisions so as to bring reality as closely into line with the narrative as possible. I was left wanting more on this subject.

Quotes

Plato sought and found truth in logic; for him there ws a sharp distinction between truth, which was axiomatic, and probability, which was merely the opinion of man. In premodernt hought there was no such thing as randomness, since the course of events reflected the will of the gods, which was determinate if not fully known. The means of resolving uncertainty was not to be found in mathematics, but in a better appreciation of the will of the gods.

p. 54

At the end of the nineteenth century, Charles Sanders Peirce, a founder of the American school of pragmatist philosophy, distinguished three broad styles of reasoning. Deductive reasoning reaches logical conclusions from stated premises… Inductive reasoning … seeks to generalise from observations, and may be supported or refuted by subsequent experience… Abductive reasoning seeks to provide the best explanation of a unique event… Deductive, inductive, and abductive reasoning each have a role to play in understanding the world, and as we move to larger worlds the role of the inductive and abductive increases relative to the deductive. And when events are essentially one-of-a-kind, which is often the case in the world of radical uncertainty, abductive reasoning is indispensable.

p. 137-138

Kahneman offers an explanation of why earlier and inadequate theories of choice persisted for so long — a ‘theory-induced blindness: once you have accepted a theory and used it as a tool in your thinking, it is extraordinarily difficult to notice its flaws’. We might say the same about behavioural economics. We believe that it is time to move beyond judgmental taxonomies of ‘biases’ derived from a benchmark which is a normative model of human behaviour deduced from implausible a priori principles. And ask instead how humans do behave in large worlds of which they can only ever have imperfect knowledge.

p. 147-148

In many colleges, students of law are taught to follow a structure described as IRAC: issue, rule, analysis, conclusion. The impressive skill of a top lawyer is to identify the issue; to give structure to an array of amorphous facts, freqnetly presented in a tendentious manner — that is to establish ‘what is going on here.’ … IRAC is a useful acronym for anyone engaged in the search for practical knowledge. In the legal context it leads naturally to thetow next stages of effective practical reasoning — communication of narrative and challenge to the prevailing narrative.

p. 194-195

The legal style of reasoning, essentially abductive, involves a search for the ‘best explanation’ — a persuasive narrative account of events relevant to the case. The great jurist and US Supreme Court Justice Oliver Wendell Holmes Jr. began his exposition of legal philosphy with the observation that ‘The life of the law has not been logic; it has been experience… The law embodies the story of a nation’s development through many centuries, and it cannot be dealt with as if it contained only the axioms and corollaries of a book of mathematics.’

p. 211

A ‘good’ explanation meets the twin criteria of credibility and coherence. It is consistent with (most of) the available evidence and the general knowledge available to judges and jurors… A good explanation demonstrates internal coherence such that, taken as a whole, the account of events makes sense. The best explanation can be distinguished from other explanations and is not compatible with these other explanations. Statistical reasoning has its place but only when integrated into an overall narrative or best explanation.

p. 212

In pressing the case for probabilistic reasoning, Philip Tetlock and Daniel Gardner, the appraisers of forecasting and architects of the ‘good judgment project’, argue that ‘For decades, the United States had a policy of maintaining the capacity to fight two wars simultaneously. But why not three? or four? Why not prepare for an alien invation while we are at it? The answer hinges on probabilities.’ No it doesn’t. There is no basis on which one can form probabilities of an invasion by aliens… The attempt to construct probabilities is a distraction from the more useful task of trying to produce a robust and resilient defence capability to deal with many contingincies, few of which can be described in any but the sketchiest of detail.

p. 294-295

The mark of science is not insistence on deductive reasoning but insistence that observation trumps theory, whatever the purported authority supporting the theory.

p. 389

Acknowledging radical uncertainty does not mean that anything goes. Look to the future and contemplate the ways in which information technology will be deployed in the coming decades, or consider the ways in which the growth of prosperity and political influence in Asia will affect the geopolitical balanc.e They are all things about which we can know something, but not enough; we see though a glass, darkly. We can construct narratives and scenarios to describe the ways in which technology and global politics might develop in the next twenty years; but there is no sensible way in which we can refine such dialogue by attaching probabilities to a comprehensive list of contingiences. We might, however, tlalk coherently about the confidence we place in scenarois and the likelihood that they will arise. As we have ephasised, the words ‘confidence’, ‘likelihood’ and ‘probablitly’ are often used interchangeably but they have different meanings. We do not enhance our understanding of the future by inventing facts and figures to fill in the inescapable gaps in our knowledge. We cannot rely on forecasts in planning for the future…. We are not afraid to answer these questions with ‘we do not know.’

p. 403

My writing for Quartz

I recently left Quartz after ~2.5 rewarding years as an editor there. The most gratifying editorial aspect of that work was editing hundreds of interesting features from nearly every reporter on staff. There are too many of those pieces to try and select favorites. But I wrote a bit, too. And I wanted to link to a few of my favorite pieces I wrote during my time there:

I’ll always be grateful for my tenure there, working with some truly excellent people.

A.O. Scott on pragmatism and art

…Among my principal guides are Ralph Waldo Emerson and John Dewey. If there’s an implicit allegiance here, a school of thought in which I might claim membership, it’s some version of pragmatism. That is, I believe that our understanding of art emerges from our experience of it, and that our notions of beauty and value are the result of our arguments about them, rather than the conditions of such arguments. Truth is the lovely echo of our noisy, contending ways of being wrong.

And one big way to be wrong–perhaps one I should have addressed at more length–is to buy into the fantasy of critical authority, to confuse the decidedly democratic power of persuasion with other kinds of power.

“Better Living Through Criticism: How to Think About Art, Pleasure, Beauty, and Truth,” A.O. Scott, pg. 275-276

Edmund Wilson on journalism

“When I speak of myself as a journalist,” he wrote, “I do not of course mean that I have always dealt with current events or that I have not put into my books something more than can be found in my articles; I mean that I have made my living mainly by writing in periodicals. There is a serious profession of journalism, and it involves its own special problems. To write what you are interested in writing and to succeed in getting editors to pay for it, is a feat that may require pretty close calculation and a good deal of ingenuity. You have to learn to load solid matter into notices of ephemeral happenings; you have to develop a resourcefulness at pursuing a line of thought through pieces on miscellaneous and more or less fortuitous subjects; and you have to acquire a technique of slipping over on the routine of editors the deeper independent work which their over-anxious intentness on the fashions of the month or the week have conditioned them automatically to reject, as the machines that make motor parts automatically reject outsizes.”

“Better Living Through Criticism: How to Think About Art, Pleasure, Beauty, and Truth,” by A.O. Scott, pg. 228

Corporate social responsibility

There’s a lot of buzz and debate about ESG and corporate social responsibility lately, so I wanted to post something I worked on recently on this topic. I helped Quartz put together a page describing its mission to “Make Business Better,” and most of that reflects group work bridging a range of perspectives within the organization.

I’m pleased to take credit for one bit, though, where I tried to articulate my own thinking on the topic:

Econ 201

Milton Friedman’s theory of shareholder capitalism kept things simple: Companies should not break the law, and other than that they ought to serve shareholders. By contrast, “stakeholder capitalism” or “corporate social responsibility” can seem hazy, unfocused, and not rigorous. 

So here’s a short checklist for what a better form of capitalism would require of businesses, in plain English and in economic jargon:

Solve a real problem for someone (in econ terms, create “surplus”)

Share the rewards fairly (customers, workers, suppliers share in the surplus—and not just those who are members of privileged groups)

Don’t pollute (minimize negative externalities, or at least pay for them)

Follow the rules (follow the law in spirit, not just in letter)

Be a good corporate citizen (compete fairly, and don’t undermine the institutions on which you depend—including government and civil society) 

That’s it. A little more complicated, sure, and a little harder to write down as a mathematical model. But we believe it’s what’s needed in a world beset by inequality, climate change, and (one more econ term) countless market failures. We can’t just leave these problems to the market, and we can’t just leave them to governments or philanthropists either.

This draws on stuff I started to sketch out in my post “What are organizations for?” and this interview I did with an ethicist also in my opinion adds a lot of good context around these questions.

How economics thinks about technology and labor

A recent David Autor review paper sums up the evolution:

I began by asking what the role of technology—digital or otherwise—is in determining wages and shaping wage inequality. I presented four answers corresponding to four strands of thinking on this topic: the education race, the task-polarization model, the automation-reinstatement race, and the era of AI uncertainty. The nuance of economic understanding has improved substantially across these epochs. Yet, traditional economic optimism about the beneficent effects of technology for productivity and welfare has eroded as understanding has advanced. Fundamentally, technological change expands the frontier of human possibilities, but one should expect it to create many winners and many losers, and to pose vast societal challenges and opportunities along the way.

What are the policy implications of these observations? The question is so broad that almost any answer is bound to appear vague and inadequate. One can reliably predict that technological innovations will foster new ways of accomplishing existing work, new business models, and entirely new industries, and these in turn will generate new jobs and spur some productivity gains. But absent complementary institutional investments, technological innovation alone will not generate broadly shared gains. Autor et al. (2022) sketch a long-form policy vision of what form these investments may take, focusing on three domains: education and training; labor market institutions; and innovation policy itself.

Deference to experts

This is a good tweet:

The value of deferring to experts depends on the alternative. If the alternative is deferring to a market or the consensus of smart generalists with good incentives or to a carefully calibrated statistical model, then deference to experts might not look so good–or at least is likely incomplete.

But a lot of the time the alternative is leaning on your own biases or those of your group, or deferring to pundits or to prevailing views shaped by attention algorithms that no one fully understands. In those cases, deference to experts looks pretty good!

Ultimately, the value of deferring to experts is in tying yourself to the mast, in epistemological terms. You defer to avoid your own biases. But as always, getting it right is about choosing wisely whom to trust.