Thread by @RadioFreeTom

So, this is a thread not about polling, but about experts, and being wrong. I have no idea if this guy is right, but I've seen this kind of expert stance before, and this is why "expertise" is more than a parlor game of prediction. /1

https://twitter.com/tripgabriel/status/1323455968781414400

Guessing and betting on outcomes is not the same thing as getting something right for the right reason. If you're playing blackjack, and you hit a 15 against a 5, and you pull a 6, you still stink at gambling even if you get away with it. Even more than once. (I've seen it.) /2

And if you don't reveal methods while predicting stuff, no one knows if you're good or just lucky. There's a political scientist at Stanford (I mention him in the book) who claims to have a top world events prediction algorithm - but only sells it to his clients. 🤷‍♀️ /3

No one really knows if this guy is a good predictor; he claims his clients are happy and he's sold his product to a lot of people. He's called a few things right. But his critics note that there's no way to know how much of that is brain work and how much is dart-throwing. /4

And it matters, especially over time, and especially if you're trying to build a body of knowledge about why things happen. "There will be no major war in Europe today" was a good prediction and it was only wrong twice in about 36,000 times but it's not helpful. /5

If Trafalgar calls this election correctly, no one will know if it was luck or skill. It's like Sovietologists who predicted every year that the USSR would not fall - and they were right 20 or 30 times in a row until they were wrong. But hey, until then, they were "right!" /6

The thing is, the Sovietologists were "right" for the wrong reasons. (I was not one of them, but I was too young for the "will the USSR fall" game.) Their predictions were "right," but in the end useless when it came time to foresee or explain a giant event. That's a problem. /7

Unfortunately, however, the public hates experts who caution that models have limits and exact prediction is a mug's game. (Think of how they react to meteorologists.) They want solid predictions with no caveats. Experts can't really provide that, nor should they try. /8

In the end, being right by guessing isn't expertise; it's just luck. Knowing *why* you're right, and refining your explanations, is how experts do stuff. Trafalgar is like the guy at Stanford; they can claim all kinds of wins, but we'll never know if it's just guessing. /9

If Trump wins, I'd rather have @NateSilver538 explain how the 1-in-10 shot that 538 said was possible came about than a guy who says "I have a magic box whose workings I won't reveal to you that foretold this answer." You'll learn more that way, imo. /10x