Thread by @jeuasommenulle

A few thoughts on the A level debacle because I think it shows very important trend - especially when compared to what happened in France, which obviously faced a similar situation. How did both country do?
This is quite stunning.

Basically the UK tried to use a very sophisticated algorithm with rather fancy maths - the purpose was to be fair, not only for bad grades but also good ones (I'll set aside claims that all the problems were actually intended- I don't believe government is smart enough for that)

Of course it was never going to work for all, of course there were going to be problems , errors, but it was an impossible task - no one can reproduce what the exam would do without actually doing the exam! Algos are not wizards

Compare to what France did: they took all the grades you had during the year and used them for the Baccalaureate results.
Which is crazy because schools don't grade the same way ! And even within one school, teachers can have different grading style

E.g. a top school will give you bad grades because they want you to be prepared for highly selective schools while struggling schools will give good grades to encourage students

Both are understandable. But still it sounded weird to base Baccalaureate on those considering the discrepancies!

So they used a super simple technique: multiply each pupil's grade by the average grade for the school Baccalaureate for the past years divided by the average grades of the school this year. A sort of school multiplier. Can't make anything simpler!!!

But what about schools with a multiplier below 1 ? Wouldn't the pupils feel cheated especially if they had excellent grades all year !? That would feel harsh ! So what's the solution ?
Again: no fancy maths, just straightforward approach : the school multiplier can't go below 1

This is a heads I win tail you loose situation for students !!!

Unsurprisingly nobody really complained. A few articles about how easy the Baccalaureate was this year but that's it.

Compare with the shitstorm the more sophisticated UK approach has created - it's crazy!

So what lessons to I get from this?

1) a point @JolyonMaugham has made : this wasn't the year to be too smart and accepting a little bit of "good grades for everyone" was a sensible approach

2) it also shows a different approach to A levels /Bac. In france the government has pretty much given up on its relevance. Nobody cares anymore and it's now the job market's work to select talented people. I think the UK still gives more importance to A level (good thing imho)

3) is maybe the most important and it's about our collective reaction to algorithms. What this showed is that stupid algo that do not do individual, easy to identify, harm but have potentially long term dramatic consequences are much more socially acceptable than smart algo....

That work for the greater good but make individual errors that can (sometimes badly) harm people.

The obvious example is autonomous cars and that's why, technical challenges aside, it's going to be a bumpy road to get there.

It's going to take a lot of lives saved from less accidents to accept the few inevitable errors that will lead to deadly crashes! And that's really an extremely important topic that will also pop up in medical diagnosis, justice system, and countless other fields !

Our tolerance for algo errors is incredible biased towards anonymous, long term fatalities as opposed to short term, individualised cases!

(Technically it was a smoothed moving average over 3 years but it doesn't really change anything)

UPDATE : the UK goes French

https://twitter.com/jeuasommenulle/status/1295378684946329601?s=19