SAMPLE SIZE IN QUALITATIVE RESEARCH
This is a thread on how to address criticism of low sample size in qualitative research. I hope you will read and share. #AcademicTwitter #QualitativeResearch #critqual #phdchat #phdadvice #educolor #PhD #acwri
This is a thread on how to address criticism of low sample size in qualitative research. I hope you will read and share. #AcademicTwitter #QualitativeResearch #critqual #phdchat #phdadvice #educolor #PhD #acwri
1. Any question about sample size must not be answered right away until you understand the context of the question. Otherwise, you will fall into the trap of justifying qualitative research by using the criteria for quantitative research. It's a set up for failure. Avoid.
2. To understand the context of criticism about small sample size, inquire about the concerns first. Is it because the inquirer thinks nothing can be learned from a small sample size, are they asking questions about generalizability, data saturation? What is exactly the concern?
3. Realize this. Not any of your explanation should include justifying how your sample size could be used to generalize somewhere down the road, or any other explanation where you pin qual against quant research. It's an oppositional conversation, without any decent end goal.
4. So, a script. "Your query makes certain assumptions of inquiry that are grounded in positivism/postpositivism. However, qualitative work isn't informed by those philosophies of inquiry. (cont'd)
5. So a trajectory for methodological processes in qual research has to be traced from its philosophies of inquiry." The key here is to focus on philosophical assumptions, something that might be outside the scope of the inquirer's knowledge, but you remain invitational.
6. Next, say the inquirer is unconvinced. Says that they don't believe in subjectivism, relativism, interpretivism, constructionism, etc. You're not in the business of EVER trying to convince anyone of anything that they are not willing to believe. It's an energy drain.
7. Respect the person's agency and state, "I understand that certain philosophies of inquiry associated with methodologies might not be the ones you would pick, but this is what guides the field, and I am going by established procedures of the field." Then flood with citations.
8. Again the key here is to invite them to your inquiry playground & remind them that this is an unfamiliar space for them. They can resist all they want, but you hold firm that you will not defend your methodological decisions using the criteria of goodness for quant work
9. Now, let's amp up the stakes a bit. Say someone is qual-friendly but barely crossed over the positivist/post-positivist threshold and thinks that unless your sample size is 20-100 people you aren't going to learn anything. They are having a heart attack with your n=5, or less.
10. Remind them that qualitative research isn't set up to generalize to the population from the sample that you are working with. Instead qualitative research is for the purpose of developing critical, analytical, in-depth insights. People can take up these insights in many ways.
11. As a qual researcher, you can make recommendations of how your work informs policy, practice, understanding structures of inequities, etc. but you, as a researcher, don't make arguments towards generalizability because you start with purposeful, criterion-driven sampling.
12. These forms of sampling would never be methodologically congruent to generalize to large populations even if you had 100 people in your sample, because as qualitative researchers, we seek people who can offer us information to build deep, rich, thick, narratives, & analysis.
13. One strategy to mitigate this that I use with my students is to do a data inventory chart. I make them list all data sources and the raw pages of data each source would generate for the duration of the study and count them up. For example, let's say your sample size is n=5.
14. 1-hour interview = 20 transcript pages. For 5 people it would be 100 pages for just one interview/person. But you shouldn't only want to do one interview/person because you are after depth. So you would want to do at least 3-5 interviews (if that's your method) per person.
15. Now you're at 500 pages of raw interview data for 5 people if you did 5 1-hr interviews. Then you add in observations, document analysis, journal reflections, member checks, & whatever else you're counting as data sources, & you will get to 1000 pages of raw data in no time.
16. I tell students to lead with the number of raw data pages they would have in their studies instead of the sample size. Then explain, of course, how that data would be generated. And you return to the argument that you are going for depth and not for generalizability.
17. Sometimes people use "data saturation" in qual world as the benchmark of good work. However, "data saturation" is approach contingent. Not all qual approaches go for data saturation. So here, you have to know how your theoretical & methodological frameworks inform your study.
18. Sample selection is also resource contingent (of the researcher). So the solo researcher is limited in time & access, & sample selection is reflective of that. If there was resources available, then sure, increase sample size if you can do justice to the work with depth.
19. But more does NOT mean better. More can mean superficial. Then not only do you not have generalizability, you're not even producing original insight/knowledge with superficial treatment of data. So the argument is always about depth. Have scripts prepared to talk about depth.
20. In summary then: a) don't argue with positivist/postpositivist criteria to justify your work, b) invite the critique to your playing field, c) remind them that you are operating with different assumptions informed by different philosophies of inquiry, (cont'd)
21. d) have clarity on your theoretical and methodological framework and how they shape your study and continuously invite people to engage you in your theoretical and methodological spaces, e) count the raw data pages you will produce and lead with that, (cont'd)
because if you're analyzing 1000 pages of raw data, it would take you a long time to cultivate critical, analytical insights, f) remind them that data saturation may not be your goal depending on your framework, g) you're motivated to create original knowledge through (cont'd)
in-depth inquiry & analysis, & most importantly h) hold your ground, cite your people, & demonstrate deep knowledge & conviction in your area of inquiry so that no one can unsettle you or make you feel that the work's unworthy because you aren't including the entire neighborhood.
Although it's implied, but if you're a minoritized scholar, in a space that's hostile, then lean heavily on culturally-situated philosophies that inform your work, & hold firm with your conviction. Say, "Your suggestion is incommensurable & culturally incongruent to my study."
