Thread by @JohnnySRich

@ucu and @NEONHE have today published report supporting the proposals to move to a #PQA system for #UniAdmissions.

But are they right? ucu.org.uk/media/11479/PQ…
1/

I almost never disagree with Graeme Atherton on supporting disadvantaged students and @DrJoGrady is fantastic on so many things, but, on this, I think they're wrong...

...unless any shift to PQA is accompanied by a raft of changes & assurances that aren't currently planned.
2/

I have two key issues with the report:
▶️what they think the problem is
▶️what they think the solution is

The problem to solve is NOT the use of predicted grades.
And, as a solution, PQA would only make the real problem worse.
3/

As their report says, most predicted grades are 'wrong', but only as predictions of what the actual grade will be.

They are not necessarily wrong in terms of being fair assessments of the student in question.
4/

Predicted grades are the expert view of a teacher who knows the potential of a student based on their past performance across a course.
5/

Actual grades are an assessment of how a student did in an exam on a particular day when their dog might have just died, they're on their period and the room's too hot.
6/

What's more, actual grades aren't even an accurate assessment of that. As @nookophile has shown, second examiners regularly disagree with the awarded mark and, as OfQual's head Glenys Stacey told @CommonsEd, the grades are only correct give or take a grade.
/7

Saying that predicted grades are a good prediction of actual grades is like saying that a calculation of how many sweets are in a jar based on weight is a bad predictor of how many you might guess.

It is, but that doesn't mean it's less good if you want to know the number.
/8

But there's no denying it: predicted grades are on average higher than actual grades and, if they are on average less generous to disadvantaged students, then the UCU report has a point that their use is unfair and a problem. So, is that the case?
/9

It would seem not. Or at least, not always. @Markcorver's excellent piece for @HEPI_news's collection of essays on admissions suggests the opposite is true. hepi.ac.uk/2021/03/18/new…
/10

There is however a pattern of high attaining disadvantaged students whose predictions lead them to 'undermatch' – ie. pick an institution that is less selective in terms of grades than they could have achieved with their actual grades.
/11

This assumes they've made a reluctant choice to 'settle' for those institutions, which is disparaging to both the students and institutions involved. It's perfectly possible for someone capable of entry to, say, Oxford, to make a positive choice to study somewhere else.
/12

They have the option of UCAS adjustment if their choice was indeed made reluctantly. Very few people choose that option.
/13

In any case, when we refer to 'undermatched' disadvantaged students, we're talking about c1,000 students a year. Overhauling the entire system in favour of an untested model for the sake of 0.2% of the cohort is a massive risk...
/14

..particularly given that the unintended consequences of such a change may not only damage the other 98.8% of applicants, but may make matters worse for the very students we want to help most.
/15

Don't get me wrong. The current system is flawed. Deeply. And it's a flaw that affects more than 10% of the cohort.

In order for unplaced students to find places and unfilled places to find students, the current system requires the Clearing process.
/16

It's called a 'process' but it's more of a free-for-all. Decisions on both sides are made in the heat of the moment in the face of rising desperation.

Unsurprisingly, the matches made through Clearing are often sub-optimal.
/17

In fact, there's a correlation between the numbers arriving at an HEI through Clearing and its drop-out rate.

Correlation is not causation, but we all know the saying 'act in haste, repent at leisure'.

Dropouts are just the tip of an iceberg of bad matches.
/18

During the summer and after the exam results, it's difficult for applicants to access good careers advice, to take time to absorb it, to research, to explore options (visit unis) and generally make good choices without undue pressure.
/19

Students with the greatest disadvantages (socioeconomic or other) are most in need of support (because their need may not be typical of more traditional students) and guidance, but are least able to get it.
/20

Schools can't help: Various studies have shown that careers advice is weakest in the schools/colleges with the lowest progression rates.

Parents can't help: by definition, these students come from backgrounds with less experience of HE.
/21

It is not surprising then that students from disadvantaged backgrounds are the ones most likely to use Clearing.

So, if you're looking for a problem in the current system that stacks the odds against disadvantaged students, look no further than Clearing.
/22

Which brings me to what's wrong with the proposed #PQA solution. In practice, it does away with the use of predicted grades and putting a small proportion of students into Clearing by, in essence, putting EVERYONE into Clearing instead.
/23

All decision-making would be made over the Summer when teachers, uni administrators, careers advisors and families were all hoping or expecting to be on holiday.

Slow, considered, advice and guidance would disappear in favour of grab-it-while-you-can approaches.
/24

If you think the current system has encouraged some HEIs to use unfair competitive practice like 'conditional unconditional' offers, just wait to see what they come up with when their entire recruitment is based on what they can achieve in a massively compressed time-frame.
/25

Of course, it wouldn't be ALL their recruitment. Some places would be taken by international students who would need to be offered places earlier in the cycle (or we run the risk of compromising competitiveness on the international HE market).
/26

The most selective universities would be keen to use this as an opportunity to fill as many spaces with international students as possible, leaving fewer spaces for the very disadvantaged students we want to help.
/27

Indeed, unless regulation steps in (requiring legal changes), those unis might also offer early secured places to UK students on an international fee basis using – you guessed it – predicted grades.
/28

To many privately educated students and their schools, those fees may not look so bad as the price of a guaranteed place.

With that certainty, they can also get to the front of the queue for housing options, even part-time job opportunities (should they want or need them).
/29

Anyone going through PQA though will have to sort out their accommodation, funding, care for dependents, etc all at speed. Speed, in this case, is not only stressful, but often means poorer quality at a higher cost.
/30

PQA looks attractive at first glance, but the unintended consequences – especially for disadvantaged students – might herald a wild west era of admissions with higher costs, fewer opportunities and worse outcomes for the most disadvantaged students.
/31

We should only – I repeat, ONLY – get behind the idea if it comes with:

1) guaranteed ways to embed careers advice and support into the process from an early stage (Yr9 and beyond)
/32

2) Greater regulation to avoid unfair competitive practices (and such regulation may have its own damaging effects on institutional autonomy that on balance would mea PQA wouldn't worth it anyway).
/33

3) Penalties for failure to maximise wider access and equity of opportunity, including perhaps recruitment targets for students from disadvantaged backgrounds and reserved places on courses.
/34

4) Changes to L3 assessment processes to ensure they recognise potential as well as summative performance
5) Changes to the HE academic year (January start?) with financial & career support for disadvantaged students from the period when school ends until uni term begins.
/35

Those are big asks and not one of these 'only ifs' is included in the current proposals for PQA, so, for those reasons, count me out.

Very interesting research here from @BrilliantClub showing that the reported enthusiasm of students for PQA really isn’t all it’s cracked up to be. They only like it until they know what it is.

https://twitter.com/brilliantclub/status/1387728530470023171