If psychologists had to present and defend a DAG every time they did a mediation analysis, 10% of them would result in "sorry the effect we care about isn't identified" and the other 90% would be "sorry we don't understand this phenomenon well enough to draw a DAG"
(Apparently today is "be salty about mediation" day in my brain)
Real talk though, I think mediation analysis can sometimes be useful if it comes with a thorough, eyes-open discussion of all the things that must be true for it to be interpretable as mediation
It's just that that hardly ever happens
• • •
Missing some Tweet in this thread? You can try to
force a refresh
And now a thread of things that must be true in order to validly interpret a standard mediation analysis.
These are assumptions you must defend with outside knowledge. They are not tested as part of the analysis, and often they are not testable even in principle.
Hey psychology twitter. The journalist Jesse Singal is probably going to be showing up on your radar because he has a new book about social psychology. He has also written about trans people. I want to encourage you to read what trans people have said about that work
There is a lot of stuff out there and it can feel a little overwhelming. Here is one place to start, a collection of links to critiques of a very influential 2018 Atlantic article he wrote patreon.com/posts/19542136
This tweet and the thread it is a part of encapsulate some of what really sunk in the most to me about the criticism
Is who you follow on Twitter correlated with your mental health? Using machine learning, we find that the high-degree accounts that people follow - celebrities, public figures, etc. - can predict anxiety, depression, post-traumatic stress, & anger (out-of-sample R = .2)
Substantive upshot: A user's Twitter experience - what they see in their feed - is heavily affected by their decisions of who to follow. Those curation decisions, in the aggregate, are associated with mental health
Some notes and points of interest:
1. "Predict" is used in the machine-learning sense, think of this as a correlation - causality wasn't tested
2. The size of the correlation is neither huge nor neglible. We don't think it can be dismissed but it shouldn't be exaggerated either
Were you taught that you cannot interpret a main effect in the presence of an interaction? That interactions "supersede" main effects? That you have to use hedging language like, "the effect of A depends on B"?
Then I've got a little provocation for you. Thread...
Imagine the following study: People with depression are randomly assigned to get either drugs or psychotherapy. In addition, they are asked which they believe is more effective: half say drugs, half say therapy. Outcome is functioning after treatment (0-100). So it's a 2x2
The PI asks two grad students to each run an ANOVA. (What can I say, she likes redundancy.) She wants to know if psychotherapy is more effective than drugs. Both grad students go off and do their thing, and report back at the next lab meeting