I’m going to begin today with a bold claim: Being an applied statistician is a lot like being an ethnographer.
I say this both based upon years of experience working in collaborative projects and consulting and based on my experience studying ethnography. (Recall: before my PhD in statistics, I started and quit a PhD in sociology).
Very often a question asked is not the ‘real’ question at hand. Typically, the person asking has a sense of the problem, but may not know exactly how to ask the question.
For this reason, I like to ask people to back up, tell me more about their project, and then I ask them a lot of questions. I assume that it’s not straightforward – figuring out their question is a puzzle in and of itself.
A question is never really in the abstract. There are always constraints – some resource driven, and some socially determined. You have to elicit these as well – and some of them may be unspoken.
One constraint is disciplinary norms. As I showed earlier this week, economists like to use CRVE, while sociologists like to use MLMs. They both ‘get the job done’ in terms of taking into account clustering, but the approach – what is signal, what is noise – is different.
To be clear: I’m not saying your job is to reify norms. But they need to be acknowledged, as they affect how the person will need to write about their work.
Another constraint is what the person – and their team – knows how to do themselves. What software do they use? What methods are they familiar with? You simply can’t provide an answer without also providing a means to getting between here and there.
Finally, very often the job of a statistical consultant is to be an ‘outsider.’ As an outsider, it’s ok for me to ask a lot of questions. Much of the work is more about ‘thinking statistically’ than modeling or calculation.
For example, what are the goals of the project, the questions guiding the research? What is the study design? Why are you using this model and not another?
In summary: Like an ethnographer, be curious, listen carefully, and observe.
Try to really being intellectually engaged with the work – ask a lot of questions, think carefully about what is possible, and help them. Remember that statistics is one part of science, but not the whole of it.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Yesterday I tweeted about nested data, with multi-level models (MLM) versus OL + cluster-robust variance estimation (CRVE). This made me think about another confusion that arise, between what are called fixed versus random effects.
Let’s begin with a simple relationship between a covariate X and Y in nested data, e.g. students i nested in school j. We are interested in understanding the relationship between X and Y at the student level.
Approach 1: Assume the schools are fixed, but that students are a random sample within these schools. Assume the relationship between X and Y is the same in all schools. This often amounts to including a dummy variable for each school in the model. Here I use OLS to estimate β_1.
I work primarily with nested data. One example is in experiments, with students nested in schools. Another is meta-analysis, with effect sizes nested in studies. In this thread, I’ll focus on students nested in schools, but this applies more generally.
Question 1: Do you need to take nesting into account in your analysis? Our world is naturally nested – students in classrooms in teachers in schools in districts and so on. Does this mean we need to take all of these levels into account? No.
Nesting only needs to be accounted for if it is part of how our sample of data is generated – either how the data is selected (sampled) or the who gets an intervention being studied (assignment).
Hello everyone – I’m so excited (and nervous!) to get to tweet with you all this week. I’ll start by telling you some general things about myself.
I’m an Associate Professor of Statistics at Northwestern University and a Faculty Fellow at the Institute for Policy Research. I also Co-Direct the Statistics for Evidence-Based Policy and Practice Center. For more info see here: bethtipton.com
I call my field “Social Statistics” and I much of what I study has to do with the role of statistics in the creation and use of evidence for decision making, particularly in the field of education research.
The #DataFeminism book also made me look inward and examine my own biases, which I am exceedingly grateful for.
Namely, it forced me to reckon with some of my fundamental operating assumptions as a statistician & data scientist.
Examples threaded below...
In chapter 3, the authors discuss the role of emotion in data visualization, specifically calling out giants in the field like Edward Tufte and Alberto Cairo (no snitch tagging, please) for what is presented as an anti-emotion stance.
On Tufte: "Any ink devoted to something other than the data themselves ... is a suspect and intruder to the graphic. Visual minimalism, according to this logic, appeals to reason first. ... Decorative elements ... are associated with messy feelings ... and emotional persuasion."
For #ThrowbackThursday I thought I'd highlight some of the amazing women who have been mentors (and friends) to me. Without support from an amazing community of women in mathematics & statistics I would not be where I am today! #WomenInSTEM