Thread by @joshbudlender

This is outrageous - an RCT cutting off water to tenants in Nairobi "slums" to see if it induces their landlords to pay the utility bill. And reading the paper just makes it worse.
(1/9)

nber.org/papers/w27569

One issue common for RCTs in poor countries-
no informed consent. To me this is always serious, but sometimes people justify w/ experiment "does no harm" & allows potentially nb benefits for targeted population. In this case? *Massive* harms & completely unclear benefits

(2/9)

*Landlord* arrears of as little as $25 for one month mean *tenants* face water cut-off. Authors say *9 months* after the disconnection, many/most (unclear, & number not given) were reconnected. As if this is not so bad? Poor families with no water, some for >9 months.
(3/9)

And to learn what? That cutting off water induces landlords to pay bills? Uh, duh? They sell contribution as cut-offs not inducing "community activism", but every reason to think this result won't scale when you move from 97 randomly selected "compounds" to official policy
(4/9)

Also: "community activism" is basically made synonymous with "political costs", ie people protesting water cut-offs. Lack of "political costs" is portrayed as a good outcome. So we should be happy about lack of democratic engagement from tenants having services cut off?
(5/9)

How would authors feel re "community activism" in Joseph v City of Johannesburg where cutting tenant electricity due to landlord arrears found unconstitutional when tenants have no chance to challenge? Experiment'd be unlawful in 🇿🇦 on this basis btw (6/9) saflii.org/za/cases/ZACC/…

I suspect they'd think the SA Constitutional Court judgment was indeed undesirable: they seem skeptical of "positive" socioeconomic rights. They can hold whatever view they want, but their one-sentence hand-wave at this Q does nothing to allay the serious ethical concerns (7/9)

I used to be methodologically opposed to RCTs of all kinds. No longer, as I've found genuine insights from some & "gold std" claims seem to have waned. Just another empirical method. But ethical issues are generally serious (often under-considered) & this case is grotesque
(8/9)

I.m.o practitioners & those in favour of development RCTs on #EconTwitter need to speak up clearly about egregious interventions like this.

The idea of even one family being cut off from water in a Nairobi "slum" in order to produce this paper is horrific to me (9/9)

UPDATE (1/11):
a) this thread got way more attention than anticipated & I can't keep up with replies/comments. Apologies
b) MORE IMPORTANTLY: the authors have issued a statement which may have a substantive impact on how one evaluates the ethical issues: drive.google.com/file/d/1nVGR4q…

2/11
A few new points I think relevant, per authors:
- Nairobi City Water & Sewerage Company asked for World Bank help in addressing non-payment
- NCWSC was *considering* increased "slum" disconnections itself, & researchers discussed doing a smaller pilot study first with them

3/11
- The Q of whether researcher involvement *reduced* cut-offs compared to counterfactual is still unclear to me, as discussed below
- ALSO, they welcome the discussion about ethics & recognise they should revise the paper to include thorough discussion of this & the context

4/11
What to make of this? For some people this will significantly allay the ethical concerns. For others this won't do anything. For me the new info affects my evaluation but very serious concerns remain and I would like more info:

5/11
I'd like more on whether NCWSC would've implemented cut-offs w/out researcher involvement. 1) They may have stated intent, but actually doing so, worrying about "political costs", may be another story. Researchers can provide legitimacy/impetus to unpopular local actions

6/11
More directly, 2): differences between v1 & v2 of researchers' statement make it unclear to me whether researcher involvement did actually *reduce* cut-offs, as v1 claimed. I appreciate authors' efforts to revise & be precise, but does leave me unsure about what happened

7/11 Btw, I also don't think "it would have happened anyway" automatically resolves ethical concerns about researchers being involved in this kind of thing. This evocative example from @graykimbrough may help illustrate this for some people

https://twitter.com/graykimbrough/status/1292154950169104385

8/11
Regardless of this & NCWSC norms, I still think unethical to be part of project where tenants not given chance to challenge cut-offs from landlord arrears. Others may disagree, but based on experiences working @SERI_RightsSA & logic of Joseph v City of JHB, I think unjust

9/11
This relates to inadequate consideration of ethical implications of rights. NCWSC has problem: landlords don't pay. Here's "solution" to get payment: cut tenants' water. But tenants are people, not just policy levers. Imo bad (if normal in econ) not to discuss this properly

10/11
In general, that these ethical issues & context were not in main paper is indicative of broader problem. I don't know exact vetting process for NBER papers, but it is shocking to me that it was presumably vetted in some way (even informally) & these issues weren't raised

11/11
I'm glad the researchers plan to revise paper to include greater discussion of these issues, & also that they welcome public discussion. I'd like more info on what would've happened w/out researcher involvement, but even if they did reduce cutoffs, for me some issues remain