Thread: sample size calculations by economists.
Economists, esp in dev and health econ, are increasingly doing sample size calcs. It seems weird to do from an economics of info perspective.
Instead of calculating minimum sample size, we should report minimum detectable effect.
Economists, esp in dev and health econ, are increasingly doing sample size calcs. It seems weird to do from an economics of info perspective.
Instead of calculating minimum sample size, we should report minimum detectable effect.
Usual sample size calculation requires calculating:
N = f(CI,study design,minimum detectable effect)
where CI is confidence. But economically this is telling you the cost you must incur (N) for a given level of quality (inputs into f, including MDE).
N = f(CI,study design,minimum detectable effect)
where CI is confidence. But economically this is telling you the cost you must incur (N) for a given level of quality (inputs into f, including MDE).
That has economic value, but is not really how studies are done or items are purchased. Consumers don't say, I have to have a car of this quality, just tell me what to spend. They see a vector of (quality, price) pairs & choose pair that maximizes U s.t. their budget constraint.
Econometricians are consumers of info. They should behave the same way. Should examine (MDE,N) pairs & choose one that is optimal. If we assume the unit price of N is p, then I can get a sequence of costs C associated with qualities MDE: (C(N),MDE).
So why do I say, report MDE's rather than N? The reality is very few of us pay out-of-pocket for studies. (Sometimes we have a grant budget for multiple studies. But my experience has been 1 budget=1 study.) But if I have a fixed budget, then C(N) is fixed, so N is fixed.
What I want to know is MDE = g(CI,design,N).
BTW, you could say show me the whole set (MDE,CI,design) compatible with affordable N. But anyone going through peer review kinda thinks CI & design are fixed too. ;)
BTW, you could say show me the whole set (MDE,CI,design) compatible with affordable N. But anyone going through peer review kinda thinks CI & design are fixed too. ;)
Also, MDE rather than N be the focus. Eg while I prefer a car w heated seats (being from Chicago), a car w/o heated seats has value. Usual sample size calcs seem to suggest if you can't hit N*, don't bother doing the study (or suggests the journal shouldn't publish your results).
Its true people can on the backside manipulate MDE or study goals. But that is not exactly what statisticians really want you to do when they ask you to do sample size calcs. Plus, we shouldn't be ashamed of those tradeoffs. Instead we should be open & ration about them.
I think one benefit of this that we'll get to focus on the social value of being able to detect the MDE, which is underexplored. (A nice study in this vein from @raogautam @HjortJ @dianamoreira_sb & Santini: gautam-rao.com/pdf/HMRS_revis….)
All this said, I can see the value of an economics literature that continues trying to work with
N = f(CI,design,MDE)
It would be great to see the price associated with causal inference (design). The debate over the value of RCTs is an example of this dort of analysis.
N = f(CI,design,MDE)
It would be great to see the price associated with causal inference (design). The debate over the value of RCTs is an example of this dort of analysis.
Endnotes:
1/This is'nt a novel idea. Logic flows from work on data markets, the economics of sample size. Eg Philipson, Desimone, etc.
2/Seems like this could be a good approach for grant funder to take. Think of it as evaluation evaluation as opposed to impact evaluation.
1/This is'nt a novel idea. Logic flows from work on data markets, the economics of sample size. Eg Philipson, Desimone, etc.
2/Seems like this could be a good approach for grant funder to take. Think of it as evaluation evaluation as opposed to impact evaluation.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
