Forecasting

Let’s get one thing straight: I am not a “superforecaster.”

Over the past decade, I’ve written about forecasting research and forecasting platforms. And I’ve participated in them as well. In this post I’ll share some of my results to date. Though I’m nowhere near superforecaster level (the top 2% of participants) I’m pleased to have been consistently above average.

Here are my results:

Good Judgment Project (~2017): 23 questions, 68th percentile
Good Judgment Open (2015-2017): 9 questions, 60th percentile*
Good Judgment Open (2021): 4 questions, 76th percentile*
Foretell/Infer (2021): 2 questions, 90th percentile
Update Aug. 2022: On INFER first half of 2022, 7 questions, 76th percentile; lifetime INFER ranking 87th percentile.

The number of questions is not the number of forecasts: in many cases I made several forecasts over time on the same question. I’ve given percentiles rather than relative Brier scores or other measures because a) they’re more intuitive and b) the GJ Project setup I did was a market (no real money), and so the results were given in terms of total (fake) dollars made and where that scored by percentile. The latter is more comparable to the other scoring systems.

(*) GJP and Infer report percentile scores across an entire season so I used those above. GJ Open doesn’t, best I can tell, so in these cases I’ve averaged my percentile scores for each question, which is a bit different than percentile in total score.

Here’s another view, this one excluding my Good Judgment Project results because I don’t have percentile scores for each question.

For Good Judgment Project, not included in the chart, I “made money” (again: no actual money involved) on 17 of 23 questions, lost money on 4 and was basically even on 2.

Some of my worst scores across all of this involved the 2016 election (including primaries). One of my best involved venture capital. My impression is that, although subject matter knowledge is nice to have, time spent is the major limiting factor. Spending more time and updating forecasts more regularly pays off, even in areas where I’m coming in fairly fresh.

To close out, here is some of my writing about forecasting: