Thread by @mbw61567742

1/n My paper on the garbage dump that passes for causal analysis in Physics Ed. Res. (and I think in other education) has just been posted. Kudos to the editor for publishing this critique of papers from his journal! link.aps.org/doi/10.1103/Ph…

2/n
Newer critiques of 2021 papers here: arxiv.org/abs/2101.05647.
Start of broader discussion of what it means for all of us in science down-thread.

3/n It looks like the field of physics education research (NSF funders, editors, reviewers, funded researchers, National Academy panels …) has no clue about causal inference and little clue about even more familiar stats methods.

4/n Big policy decisions, e.g. dropping standardized tests, are being made based on “research” with comically bad errors leading to conclusions opposite of what the data imply. (See also arxiv.org/abs/1902.09442, youtube.com/channel/UCHEvL…)

5/5 Some of these papers have slipped into real dishonesty. How can we claim to be scientists and accept this crap? How much is this just the tip of the iceberg? How much is driven by ideology and how much by bureaucratic rewards? I dunno but would love to hear anecdotes.

4/n Big policy decisions, e.g. dropping standardized tests, are being made based on “research” with comically bad errors leading to conclusions opposite of what the data imply. (See also arxiv.org/abs/1902.09442, youtube.com/channel/UCHEvL…)

Some of these papers have slipped into dishonesty. How can we claim to be scientists and accept this crap? How much is this just the tip of the iceberg? How much is driven by ideology and how much by bureaucratic rewards? I dunno but would love to hear anecdotes.

On #GRExit & pseudoscience in physics education research, new trash paper has come out since my last tweet. New critique included here:arxiv.org/abs/2110.04266. I'll submit to journal after hearing reactions.
Do we really have to just accept this crap? & act as if we believed it?

finding the best student selection method is trivial in one limiting case. If we do not try to maintain minimal standards of competence and transparency or even basic logic in our treatment of data, then the optimum group of students whom we should be educating is the empty set.