A quick thread on fairness in relation to the A Level results. Fairness sounds simple and straightforward, but it really isn’t. Who do the results have to be fair for, and fair in relation to what? This is multidimensional... 1/n
When we talk about ‘grade inflation’ we’re worrying about fairness in relation to prior years. If this year’s A is easier than last year’s A, it’s unfair to the person who got an A last year. 2/n
We also need to worry about fairness to people in other schools: if one school grants easier As than another, it’s unfair on the students in that other school. 3/n
And we have to worry about fairness to employees and universities: they need to know that an A is really what they expect, so if grades are too high the student may not be as good as the employee or university expects. 4/n
We need to be concerned about fairness between subjects: an A in Maths should mean the same as an A in English, in History, or in Media Studies. 5/n
Almost all *these* forms of fairness can be determined statistically. Looking at all the past records in subjects, in schools, in areas, over the years, can help us to understand these things on a population level.... 6/n
...and this looks as though it’s been the way that Ofqual has been doing its work. Trying to juggle all these different dimensions at a statistical level, to ensure fairness on all these dimensions. BUT.... 7/n
...through doing this, they forgot the most important dimension of fairness of all. Fairness to the individual students themselves. Fairness in relation both to their ability and level of work, but also in terms of the *impact* that the decisions of the algorithm makes. 8/n
Both these things matter, and matter *more* than the other dimensions of fairness mentioned at the start of the thread. More, because they have a direct impact on the life chances of the individuals. We’ve seen that as kids have lost university places unfairly through this. 9/n
This should not be a surprise. Algorithms working on ‘big data’ are good at overall numbers, good at keeping trends in line, fitting to curves, but in order to do so they sacrifice sensitivity at an individual level. That doesn’t matter when you’re analysing trends.... 10/n
...but it matters immensely, fundamentally, at that individual level. Lowering a set of marks by one grade can correct a curve nicely, but it means each individual is impacted upon directly and potentially catastrophically. 11/n
...and that’s what we’ve seen. Kids losing life-chances so that overall stats look OK. That’s why this kind of approach should only ever be used with *great* care, with *detailed* oversight, and with robust correction systems *at an individual level* 12/n
Ofqual don’t seem to have done any of these things. They’ve prioritised the wrong dimensions of fairness, paid insufficient attention to the impact on individuals, and not provided anything like the right kind of oversight. 13/n
It’s important to understand that this *shouldn’t* have needed the appeals system to be that important - though it matters a great deal (!) - because if the right kind of attention to detail had been applied *from the start* the appeals shouldn’t be needed 14/n
That is, they should have screened for most forms of unfairness and corrected for them *at an individual level* before the results came out. They do not seem to have done this at all. 15/n
The whole plan seems to have been cack-handed from the start. I can’t see an easy way to correct for it other than abandon it entirely - but they’re likely to be too pig-headed to see that. They basically don’t give a toss for the kids. /ends rant
