Thread by @dalebassett

With A-level and GCSE results just round the corner, a brief reminder of how the system works and what you can (and can't) read into results... 🧵

Results in each subject are set using a principle called 'comparable outcomes' (although Ofqual tends not to use the term any more). The idea is that a pupil achieves the same result this year as they would have done last year.

This is done so that results are 'fair' to pupils from year to year. It's important because pupils' performance can vary from year to year due to extraneous factors, like harder exam papers, new specifications, or pandemics.

This year's standards (and last year's) are broadly anchored to 2019, so the cohort is protected from lost learning as a result of Covid: even if performance is lower, grades are maintained.

So, how do exam boards achieve these 'comparable outcomes'? Grade boundaries are set every year, AFTER exams are sat and marked. Because the difficulty of exam papers varies each year, grade boundaries move up or down to compensate for this.

So if the paper was harder, boundaries will be lower, so that the same proportion of students will get an A (for example) as would have done if they'd sat last year's (easier) paper.

Grade boundaries are also tweaked to adjust for the cohort, using prior attainment data (KS2 data for GCSEs; GCSE data for A-levels) as a proxy for ability.

So if for example this year's GCSE cohort is slightly more 'able' than last year's (i.e. they had slightly better KS2 results) there will be more top grades awarded. Again, this helps to ensure that a grade 6 (for example) is comparable from year to year.

This is done at a NATIONAL level. Individual pupils' prior attainment has no bearing on what grade they get, which just depends on how many marks they got and the grade boundaries for the qualification, which are the same for everyone who takes it.

So, the 'starting point' for this year's standard is essentially last year's standard, tweaked to reflect the ability of the cohort. There are other factors that can then move the standard - although this happens rarely, especially in large-entry subjects.

The most important of these is senior examiners' judgement. If senior examiners see evidence in pupils' scripts that the standard has noticeably moved (i.e. if students are genuinely performing better than in the past) they are able to argue that the standard should be moved.

Ofqual also runs a National Reference Test to gather independent evidence of Year 11 students' performance in English and Maths. It could use this to adjust the standard, although it never has done so far (the NRTs have been in use since 2017).

Sometimes there are intentional tweaks to the standard. This year, grading will be slightly more generous in GCSE French and German (to move standards in line with Spanish) and Computer Science (to keep standards consistent after a partial reform of the qualification).

And, of course, there was a deliberately different standard in 2022, as Ofqual sought to incrementally move back to the pre-pandemic standard, after the cancellation of exams in 2020 and 2021.

However, in most cases the standard is last year's standard, tweaked for the cohort. Because of this, all you can really read into changes in national results is that the cohort is different to last year's. Inevitable media coverage about standards going up or down is nonsense.

The standard is the same as last year; if national results are slightly up or down, that reflects the nature of the cohort. Not that exams are easier or harder. Not really even that pupils performed better or worse. It just tells you that the cohort taking the exam has changed.

An example: there are more entries for GCSE Spanish this year, probably because of the EBacc policy. We might assume that many of the 'new' pupils doing Spanish are likely not to be high-performers (since if they were very good at languages, they might have chosen them anyway).

If this is the case, the average performance in Spanish will have decreased this year, and national results will go 'down'. But this tells you nothing about the standard of GCSE Spanish, which won't have changed - it only tells you that a less-able cohort took the qualification.

Finally, remember that a GCSE or A-level grade tells you little or nothing about SPECIFIC skills. They are compensatory assessments (you can ace one part, flunk another and come out with an 'average' grade) so you can't infer specific knowledge or skills from a grade. (🧵 ends)