Thread by @page_eco

What are the chances that Biden will beat Trump in November?
That's an important question, but do we even know what it means?
A thread on what it means to forecast probability.

You find forecasts of the chances of Biden to win in many places.
@StatModeling's recently posted about his model which gives a 88% probability for Biden to win.
statmodeling.stat.columbia.edu/2020/06/12/ele…

Another prediction is given by betting markets. On Betfair, Biden's odds to win are 1.83. The odds are the inverse of the probability, so according to Betfair's market, Biden has 1/1.83=54.6% chances to win in November.

But what does it even mean to think of the chances of one event? There will be only one 2020 election. It's not like a die you can throw repeatedly when you say that there 1/6 chance to get a 6.

This probability has to be seen as a measure of our uncertainty. A measure very much in the same way as length measures distances. The probability only reflects our uncertainty. But then what is a good probability forecast?

Well, even if the election is a single event. We could look at your predictions for many single events. A minimal rule for your predictions to be considered good would then be that whenever you predicted a probability of X%, the event happened X% of the time on average.

We would say that your predictions are "well-calibrated". Predictions are often not well calibrated. For instance, psychologists have found that people tend to overestimate the probability of rare events.
researchgate.net/figure/Relatio…

In a study on betting markets, I have found that their predictions tend to be biassed towards 50% when the event is a fair way in the future.
academic.oup.com/ej/article-abs…

The bias is even larger on political markets, possibly because of people politically motivated competing to push the price on both sides, compressing it towards 50%.

In practice, given the two reasons above, the 54.6% estimate from Betfair may underestimate the chances of Biden.
But... underestimate relative to what?

The first answer is relative to a well-calibrated prediction. But a well-calibrated prediction does not mean a good prediction!
Suppose that there are 10 polls, but people on Betfair only bet after looking at one of them (the same one).

If they do this at every election their predictions may be well-calibrated (the poll is not biased), but these predictions would not be "precise". It would tend to be too close to 50% given the available information (9 other polls).

It is what we found in another study. The predictions are well-calibrated, but the betting markets do not incorporate all the information "out there".
👇 Calibration curve: Observed frequency is close to price.
Precision: Prices tend to be too close to 50%
sciencedirect.com/science/articl…

OK, so suppose that you use all the information available. What would be a good forecast? Decision theory says that you would want to form "rational" forecasts using Bayes rule.
You would use the available evidence to update your beliefs in a consistent way.

If your initial beliefs (priors) are not systematically wrong, your forecasts would be calibrated but they would also have the right degree of precision given the available information.

There is however a challenge, this kind of belief formation requires you to be in a world where you know all the things which may happen until November and how they could impact the election result.

But what about things like, the coronavirus, or the BLM movement which were totally unexpected?
In the real world, surprises happen, there are "unknown unknowns": there are things which will happen of which you are unaware they will happen.

So how can you form forecasts when totally unforeseeable events may occur in the future?

Well, you may be "aware of your unawareness" you know that there are things that you don't know which may happen, even if you don't know what are these things.

Recent work in decision theory shows that you may generalise the "rational" way of forming beliefs to this situation. In practice, you can give some probability to the possibility that something unexpected will happen.
econ.queensu.ca/sites/econ.que…

So, in that view, a probability forecast for Biden's chances to win reflects both your understanding of what you know about what may happen and your appraisal of possible unforeseeable events that may occur.
(In that light, a prediction just above 50% seems reasonable)
/End