After reading our latest article 'Is Academic Freedom in Crisis? A New Report Suggests the Answer is Yes', a few people tweeted at us saying that the sample sizes in @erickaufman’s study are small. We reached out to Eric for his response. Here’s what he had to say:
1) Sample size is the total sample (in this case, 484) and *not* the minority in the sample who vary from the majority.
For instance, in a sample of 100 US voters, with 15 black voters, if the 15 are 95% Democrat and the sample is only 45% Democrat, the coefficient for black will be statistically significant if their difference from other voters is large enough – even with a small sample.
Surveys can credibly use a sample to stand in for the whole. Most election polls are around 1,000 to represent 60m or 350m.
And small samples have low margins of error in relation to the total population in question (our survey we estimate a 3% margin of error with a sample of 500 out of a total maximum population of 50k SSH academics: goodcalculators.com/margin-of-erro…).
Note: most psychology papers work with sample sizes in the hundreds to test effects.
We run statistical models throughout the report to test statistical significance – so even if you want to query the 82% you cannot query that the effect is statistically significant.
The predicted probability of under .2 (i.e. 20% chance of saying ‘comfortable’ if you are a Brexit voting SSH academic) appears below, even with controls. 
2) When you have multiple surveys from different sources saying the same thing, and this being confirmed in statistical models, critiques of single surveys lose force.
Notice that self-censorship was high also in previous studies summarized in tables 4 and 5 below. At this point those who deny the results are just science denialists, pure and simple:
• • •
Missing some Tweet in this thread? You can try to
force a refresh
