Coverage of the new multi-study replication project by @edyong209, who dives in to why the prediction market was so effective and what that means theatlantic.com/science/archiv…
One important thing that the prediction market did is change the cost-benefit analysis for expressing hunches as behavior
Normally, if a scientist looks at a study and thinks "hmmmm, that can't be right," there are costs to expressing that out loud. These costs come in 2 layers
Cost Layer #1 is that you have to invest time in coming up with defensible reasons to be doubtful. There is a norm that scientists can't just go around saying "meh that's probably crap" too loudly
BTW a norm requiring scientists to back up critiques with reasons and evidence can be a good thing for science. But only if we have what we need to turn hunches into arguments: *transparency*. Access to data, materials, code, etc. Otherwise it's just a norm against criticism
Cost Layer #2 is that the more doubt you express publicly, the more you put your reputation on the line. Particularly if an author is powerful or well-connected, or if the results are popular in the scientific community
Again, this could be a good thing if your reputation is just judged on the validity of your critiques, if it's actually a net for your reputation plus if you're right (so there are benefits that balance the costs of being wrong), and if being occasionally wrong isn't catastrophic
But those conditions don't hold. Error correction is viewed as second-class work. Prestige and power structures are very much in operation. And there is a general norm against rocking the boat
So as many people who've done criticism or error-correction work will tell you, there is not a lot of upside, and plenty of costs - even when you are right
But the prediction market changed all those incentives. People just quietly placed bets. Nobody was going to shit on you for expressing your hunches or half-formed arguments
And lo and behold, under those different incentives the prediction market nailed it 
What all this shows is that there are important benefits to science if we can improve the norms and incentive around substantive, critical discourse
Give scientists the information they need, create incentives to do criticism well, fight against prestige dynamics and other distortions. The end result will be better better science all around /fin
