Literally reproducing the model in the lab. As opposed to testing the key assumptions and implications of the model.
Let me explain what I mean. And why this is problematic.
But careful thought needs to be given to the question: what exactly, about the model, needs to be tested.
And what’s the ideal experiment to test that.
And what does this experiment actually test.
Instead, the experimenter does the easy thing. The obvious thing. The thing that superficially resembles a test of the model.
Namely, have subjects interact in a laboratory environment designed to look as much like the model as possible.
That doesn’t test whether the key assumptions of the model hold. Or whether the key implications do.
Rhetorically, it’s powerful. But scientifically, it’s dubious.
But before I do so, I want to preface w/ this is a *very* common problem. One that social scientists, especially in experimental econ, poli sci, and in the evolutionary literature (where models are prominent) are not trained to spot or avoid.
And I hope by highlighting and walking through some concrete examples, the problem will become more clear and hopefully more easily avoided.
Eg if everyone w/ preferences for policy x gets “gerrymandered” so that as many as possible interact w/ few who share their preferences, then the voting can be swayed against them, even if the majority prefer policy x.
How does one test this model? What exactly ought you to test? And what do we learn from the experiments actually reported in this manuscript?
That’s the question. *If* that’s true, the implications, and insight, of the model will follow.
Namely, putting participants in a setting where you have rigged it so that they have the preferences in your model, and information flows the way your model supposed.
So economists have studied what kinds of (bad) things this can do to markets. And how to ameliorate these effects.
But how best to answer this question? What exactly needs to be tested?
So you give player 1 the option whether to take action A or try for B, give player 2 the option whether to accept B, if that’s what 1 chose, or force him to A.
So basically you have given subjects the payoffs and information specified in the model.
Of course they *can.*
(Have you never lost interest on a date when you found out how interested the other was?)
Of course not.
(If the setting is rather new to me, or the procedure especially opaque, or I am not that motivated to think about it, or i am sleepy or high...)
What does the first experiment I described actually test?
I *think* it again tests whether people *can* in *soke context* optimize their payoffs, and incorporate the information flow, given to them by the researchers.
So I would start there.
Tbh I don’t know. It’s not always obvious.
And it’s not always the case a lab experiment is the best way to test a model.