**There was a lot more attention given to testing at #BX2018 , with RCT's being a key focal point here**
Yet RCT's aren't a perfect panacea, as @HarryGreenwell explained - spillover effects, attrition (participant drop off) and statistical power all need to be carefully considered when designing, setting up and executing RCT's.
I'd add temporary interaction effects to that list. We've had experiences where the treatment effects seemed to amplified by a temporary campaign that was initiated at the same time. Timing of trial execution needs to be considered.
It was good to be reminded by @alexgyani of the benefits of running RCT's and how intuition and basic pre/post evaluations can let us down. The Scared Straight campaign was again a great example of where a lack of evidence can lead to some seriously bad outcomes for society.
With our knowledge limitations, inability to accurately predict outcomes and overconfidence, it could be argued that it is perhaps more unethical not to test than it is to do so. Much more humility is needed.
Going one set further, John List introduced the PSP value and the idea of a 'voltage drop' - where a successful treatment is scaled, but the impact doesn't show up. He made a call for more replications and consideration of population and situational representativeness.
This would mean around 4 high-powered replications before scaling decisions are made. The concern ofcourse is that this pushes practitioners further away from evidence-informed policy and practice.
This point was made salient by David Gruen, with his elegant reminder that 'The truth is only one special interest group and it’s not particularly well funded'. We need to understand what we're up against.
Bridging this gap is a point that comes up often. In fact, it was the topic of a recent meetup we held at the Gravity offices here in Cape Town. The outcome? Compromise. You can read more about it here: medium.com/behavioural-de…
**Behavioural insights applied to financial decision making**
Financial well being - Mo Khalil from CBA presented a useful and simple set of rules of thumb for thinking about financial well being and individual's financial needs.
Financial well being - Mo Khalil from CBA presented a useful and simple set of rules of thumb for thinking about financial well being and individual's financial needs.
Mo then outlined how they will go about tackling financial well being - a really elegant framework that is customer-centric and guided by four pillars (measurement, collaboration, evidence, and innovation).
He also made the case for private sector organisations being producers of behavioural science literature as opposed to just consumers. This is being achieved through roundtables and partnerships with academics and the public sector.
A key challenge is to quantitatively define financial well being so that their teams have something to work towards. Carol Comerton-Forde is doing some great work in this space, looking at a combination of reported and observed data and the difference between the two.
According to her data, there isn't a perfect correlation between people's reported financial well being and what is seen in the financial data - e.g. a subset of people in poor financial situations self-reported high levels of financial well being.
Anuj Shah's work was mentioned previously, but one of the big areas of discussion was how practitioners should intervene at moments of slack (higher levels of cognitive bandwidth) as a way to improve financial decision making.
**How to set up and grow a behavioural insights team (nudge unit) within an organisation or government**
It still seems necessary to start with quick wins (low hanging fruit), to provide local evidence of impact. The evidence gathered can be pivotal in persuading decision makers to invest in larger, more complicated challenges, where the impact is realised over a long timeframe.
Alex King from the NSW BI unit made a nice point about 'helping your masters' solve their problems using BI. This helps to ramp up the buy-in required for operationalisation.
There was also a lot of emphasis on having a clear vision and mission for the team. This should then be accompanied by clear objectives and guiding principles. @TaraOliver_BETA shared a succinct overview of how @beta_gov_au is thinking about this currently.
Relating to this @wrmailer neatly outlined the trades off that need to be made when setting up a unit. Emphasizing the value of 'slowly' crafting a high-quality team that has the NB infrastructure and is tackling the right challenges. This approach seems to be paying off for CBA.
The services that public and private sector units are offering seem to overlap a fair amount. Unsurprisingly RCT's are still at the core focus, but there are also support services such as internal capability building, diagnostic reports, optimisation and strategic advice.
@mariamchammat , who is in the very early stages of setting up the French BI unit, shared some useful insights from her experiences in the trenches.
Behavioural theory is becoming close to just redescribing the behaviour in more complicated terms (e.g hundreds of cognitive biases) - no good for anyone. We need to move to more general theories that explain the observable decision making accurately - e.g modelling attention.
There is also a need to focus more on 'nudging nudgers'. Choice architecture for choice architects. The @WorldBank and @B_I_Tweets both seem to be investing alot of attention here. Hopefully this will lead to more @beapplied -like initiatives surfacing.
