My Authors
Read all threads
Is applied behavioural science stuck at a local maximum?

My sense is that there is a set of technical and ethical limitations that are getting in the way of progress.

Here is how I'm currently thinking about these limits and ways to overcome them (when possible).
Firstly, the Technical Limitations:

1) Replicability
2) Unknown Boundary Conditions
3) Complementary & Crowding-Out Effects
4) Cultural Variation
5) With-in and Between-subject Idiosyncrasies
6) Channel & Data Access
7) Implementation Issues
8) Unforeseeable Second-Order Effects
(1/8) Replicability

A barrage of replication failures has called behavioural research findings, and the methodologies being used, into question.

Pre-registration, larger samples, field experimentation and research replication incentives are being used to overcome this 💪
(2/8) Unknown Boundary Conditions

Okay, this one's a bit more tricky. Interventions are more effective given the presence of certain conditions. These conditions aren't well understood yet.

Progress is made here by studying a variable across a range of contexts + meta-analyses.
(3/8) Complementary & Crowding-Out Effects

This is similar to boundary conditions I suppose. Practitioners often use a combo of interventions. Is more better than less? Maybe. Maybe not.

Studying the combinatory effects of interventions (e.g WOOP) is the way forward here.
(4/8) Cultural Variation

This one is close to home for me. I work in Africa, yet lean on US/UK research. The challenge? Cultural factors (e.g. norms) can mediate the effects of interventions in unexpected ways.

To solve: Treat findings as hypotheses and run experiments locally.
(5/8) With-in and Between-subject Idiosyncrasies

Individuals within a seemingly similar context are more or less responsive to interventions for unknown reasons, and their responsiveness changes over time.

Data access + ML and behavioural profiling are making some inroads here.
(6/8) Channel & Data Access Limitations

The effectiveness of interventions can be constrained by the channels available to the practitioner. In addition, data on the target behaviour can be hard to collect, leading to a reliance on proxy measures.

These are tricky to overcome🤔
(7/8) Implementation Issues

Even if a robust behavioural research finding is trialled and proven to be effective within a local context, it can still fail when implemented at scale.

The scaling scenario should be mapped out before the test, examined and resources identified.
(8/8) Unforeseeable Second-Order Effects

Interventions don't affect behaviour in a vacuum. There can be serious side effects and externalities. These are difficult to map, let alone measure.

Premortems and systems thinking tools are useful for mapping here. Measuring is tricky.
So robust research methods, larger samples, fieldwork across different contexts, meta-analyses, localised testing, channel expansion, individual-level tracking, ML and externality measures get us started.

However, these need to be considered alongside the ethical limitations.
The Ethical Limitations:

1) Preference Ambiguity
2) Intention Misattribution
3) Lack of Transparency
4) Pseudo-Reversibility
5) Unknown Externalities
6) Data Tracking and Privacy Issues
7) Broader Psychological Implications
(1/7) Preference Ambiguity

In some situations, it may be relatively easy to infer what an individual's preferences are. However, there are many scenarios where it's not so clear.

Asking people helps. But that's often not possible or practical given personalisation constraints.
(2/7) Intention Misattribution

Sometimes the overarching value of an individual is clear (good health), and this leads to the assumption that they intend to perform an action (getting a vaccine).

This kind of misattribution of intention means we can't rely on revealed values.
(3/7) Lack of Transparency

Some behaviourally-informed interventions (by their nature) work better when the targeted individual isn't aware of them.

Another issue here is that the intervention may be known, but the mechanism doing the work is opaque to the individual.
(4/7) Pseudo-Reversibility

Because behavioural interventions often operate outside of conscious awareness, even when optionality is available, the ability to reverse a choice may not always be possible, practically speaking.
(5/7) Unknown Externalities

Even if it is clear that an intervention has a positive effect on a behaviour (that an individual knowingly intends to perform), there may be second-order effects that outweigh the value created (e.g. pendulum or moral licensing effects).
(6/7) Data Tracking and Privacy Issues

Behavioural, linguistic and psychographic data can improve the impact of an intervention. However, this data can be (and has been) collected without the individuals' knowledge or consent, to influence mental representations and behaviour.
(7/7) Broader Psychological Implications

Feedback I often see is that the worldview presented by behavioural science leans more towards the paternal than the liberal. This psychological implications of this belief-set shouldn't be overlooked, given the research on the subject.
It is less obvious to me what the solutions to these ethical limitations are.

Additionally, some of them seem destined to be exacerbated in an attempt to overcome the technical limitations (e.g. individual-level data collection).
Understanding these limits is the reason I have decided to dedicate attention to exploring alternative approaches and intersections that may bear fruit with time.

The path that I'm the most optimistic about at the moment self-initiated behavioural science. More on this soon.
Am I missing any limitations or ways to resolve them? I'd be very interested in getting others thoughts on this.
Missing some Tweet in this thread? You can try to force a refresh.

Enjoying this thread?

Keep Current with David Perrott

Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!