A tip for folks doing empirical papers that are meant to advance soc sci theory— i.e., those that specify and “test” hypotheses.
Think of an hypothesis test as a proof of concept.
To elaborate a bit:
Think of an hypothesis test as a proof of concept.
To elaborate a bit:
Your intro & theory section should do 2 things:
a) identify a problem— how we’re confused about how (an important aspect of) the world works; &
b) develop a promising solution.
If you’ve done a good job with these 2 steps, the reader should be where you were when+
a) identify a problem— how we’re confused about how (an important aspect of) the world works; &
b) develop a promising solution.
If you’ve done a good job with these 2 steps, the reader should be where you were when+
you wrote your hypothesis:
Really motivated to make progress on the problem & really excited by the potential solution.... but also nervous. Am I really right? Would I really bet valuable resources on this solution?
That's why a proof of concept is valuable!
Really motivated to make progress on the problem & really excited by the potential solution.... but also nervous. Am I really right? Would I really bet valuable resources on this solution?
That's why a proof of concept is valuable!
If the hypothesis is designed such that it allows us to see evidence for and against the solution, results in favor should increase confidence that the solution is promising— & results against should make us rethink. Of course, it can't be definitive. But it’s a start.
Obvious? Maybe, but much too often, I find myself uninterested in the hypotheses around which a paper (including may published papers) is built before it even gets to the emprirics. This is either bc the authors have failed to identify a problem I'm interested in solving+
(there's no puzzle, just some new topic that has been "overlooked") or the proposed solution isn’t compelling (the hypotheses are "undertheorized" in that they may be plausible but it's easy to think of why they might not hold).
You might think that the evidence in support+
You might think that the evidence in support+
of the hypothesis should be compelling if the study is well done, but a prediction without a strong theoretical foundation is worth little; for one thing, it's impossible to know how to use the idea in slightly different conditions. & if there's no puzzle, why do I care?
And if one undermotivated/undertheorized hypothesis is bad, don't get me started on papers in which there are multiple such hypotheses & the authors try to conclude something from the fact that some hypotheses are supported & some aren't. UGH.
In short: Write good theory to resolve a difficult, important puzzle. If you do this, your readers will be excited by your idea before you even turn to data & they will be as eager-- & nervous-- as you are for proof of concept. This is the role of hypothesis tests.
I see this is blowing up a bit. Thx for the attention— & pls send feedback.
& if you appreciated the sensibility I shared here, you also might like this (bit.ly/2uu9Wv4) or this:
& if you appreciated the sensibility I shared here, you also might like this (bit.ly/2uu9Wv4) or this:
