So yesterday (Monday 17 August), the Education Secretary Gavin Williamson announced that this year’s A-Levels and GCSEs results would be based on Centre Assessed Grades (CAGs, frequently referred to as “teacher’s predictions” or similar). This is due to a massive outcry and sense of injustice after many students got lower (often much lower) grades than they were expected to. It followed a similar u-turn by the Scottish Government the previous week; Wales and Northern Ireland announced similar measures.
This is a mess, a complete and utter fiasco, one that could have been avoided had the government(s) done some thinking and planning ahead. Many, many young people have been put through enormous trauma and stress and many are still going through those things.
Of course, social media has been alight with fury, mostly righteous fury, about what’s happened. But the trouble with that happening is that often the full truth, the small but important details, get lost. The story essentially became “Evil Algorithm Denies Students Their Rightful University Places”; but this devastating threat would be thwarted by “Teachers’ Grades/Predictions”.
I’m no expert on secondary and further education other than having been through them too many years ago and having one daughter in secondary school now. But I really wanted to try and get my head around what had happened and what might happen next, something that the heat and fury of Twitter et al can obfuscate.
Health warnings when reading this:
1) This isn’t “A-Level Results: The Truth!” – I’ve no place writing that kind of piece, I don’t have enough knowledge. These are just a few of the things I discovered and/or that seemed important to me;
2) This piece isn’t in any way an attempt to deny or diminish the stress, trauma and anxiety many students have gone through and are going through. I just think it’s important to try to get to the bottom of what’s happened and why.
So:
1) The idea of “teachers’ grades/predictions” that has been spoken of a lot in the media abut this is not 100% accurate. As this document from Ofqual explains (PDF), the Centre Assessed Grades were drawn up by teachers and heads of departments. They then had to be signed off by 2 teachers in each department, including the head of department, and then the head of the centre (school/college etc.), who appears to have some discretion over whether to accept the teacher’s initial grades. See this tweet (and the replies) for more on this, including many examples where schools downgraded teachers’ initial grades.
1a) As that tweet suggests and the guidance from Ofqual linked to above states, setting the CAGs involved a lot more than simply picking a grade for each pupil; it had to be evidenced, discussed and moderated by departmental and centre heads. This wasn’t a case of “evil algorithm vs good teachers’ predictions”.
2) The CAGs weren’t the only things that centres had to supply, though. They also had to put students for each subject in a “rank order” of secureness for each grade from those most likely to those least likely to attain it. This was crucial for the central process that followed.
3) CAGs and predicted grades are not always that accurate. This report from 2016 (PDF) suggests that only 16% of grades were accurately predicted; some were under-predicted, the majority were over-predicted; this post, and the report linked to within it, backs up that stat and also suggests there may be socio-economic disadvantages resulting from this. This isn’t to diminish the work of teachers, just to point out that these predictions aren’t flawless.
4) There was no one, big, central “algorithm” that decided every single grade to award, in the sense of a computer program/routine that inhumanly decided people’s grades without any human input at all. See this blogpost for more details. There was a statistical model that was used to process the information sent from centres. But this didn’t just automatically spit out grades for each student; human input in the form of the CAGs and the rank order, was crucially important.
5) What seems to have been the intention was that this statistical model was to be used for “standardising” the CAGs across the country. There’s a huge and very technical document from Ofqual here (PDF) explaining why they thought this was necessary, how it was come up with etc. To me, this makes sense: A-levels are national exams and for many (most?) people who take them are used for entry to universities, competing for places and then studying with people from other centres in other parts of the country. Not every centre will have used exactly the same process to come up with their CAGs: some will have been more lenient, others stricter and so on. You need some way of “levelling the playing field”, ensuring that students are all more-or-less competing on roughly equal grounds. Normally, exams do this function; this year, that wasn’t possible.
6) It was in this standardisation process, I think, that it all went so badly wrong, where the injustices took place, where people lost out. I don’t fully understand how; perhaps it was “quirks” in the ranking orders, perhaps it was the smaller cohorts in some subjects in some centres that skewed things. But something went badly wrong there and caused all the problems that we’ve seen.
7) It now appears Ofqual was warned by the Commons Education Select Committee that there may well be problems with the model they decided to use. The Committee also asked for full transparency about the model and the process to parents, students and teachers, especially with regards to the flaws they found.
8) Side note: There were rumours going round that no one from Eton was downgraded. That’s not true.
9) Using CAGs in place of the grades originally awarded may well have been inevitable, especially after Scotland did so. But that doesn’t necessarily mean that it was the best solution. See, for example, this article from Schoolsweek, which sets out some of the problems the decision has brought up; there’s also this article in the Guardian which sets out many of the same problems. Essentially, though, it seems this is causing massive problems for universities: for some, especially Russell Group universities, it’s having to suddenly find extra space for new students; for others, it’s about the loss of students who would’ve come to them had the original grades stood, which brings with it a loss of income etc.
9a) As the Schoolsweek article points out, it also leaves current year 12s, who will take their A-levels next year, in a very precarious position, especially if large numbers of year 13s defer university for a year. There’s a real danger of, in the process of righting one injustice, creating a new injustice.
10) There are also concerns about grade inflation and the problems this might have for future years.
As I say, this isn’t the full story, or a comprehensive account of everything that went wrong. It does seem a lot more complex than many of the media and social media stories suggest. And there may well be errors; please let me know if there are and I’ll be glad to correct them.
But what sticks out for me is that this is something that Ofqual and the government could have planned for. The decision to close schools and cancel exams was taken in March. Yes, Ofqual put a plan in place and there may have been good reasons for that particular approach to have been used.
But there doesn’t seem to have been any consideration about the knock-on effects, no contingency planning for “what if it goes wrong”, no thought about what the appeals process might look like.
Then there was the stubbornness of Gavin Williamson and the government stubbornly refusing to admit the system was anything other than “robust”, before u-turning late yesterday. They had the benefit of a whole week after the Scotland experience to look into whether something similar might happen in England and, if so, what they might do about it. But as so often with this government, they didn’t.
This all led to the screeching u-turn happening. Personally, I think in general governments should be afforded some leaway with u-turns: they’re not immune from making mistakes and getting things wrong, and to jeer every time they change their mind just makes it harder for politicians to admit their mistakes. But when they’ve had the experience of another system running into major problems, time to look and learn from them, and then a whole weekend of government people saying how robust the system is and how they’re not going to change their mind – the inevitable u-turn just looks weak.
Plus, as noted in points 9-10 above, there are serious drawbacks to switching to using CAGs at this stage; it just pours even more fuel on the flames and creates new problems for new people. Many celebrated it as a triumph; but I’m not so sure. It’s the least worst option, at best (if you see what I mean).
There was no need for this to happen; this situation could’ve been dealt with in a fair and reasonable way. That way might have been tricky to work out, but that doesn’t mean it shouldn’t have been looked at. What we’ve ended up with is a complete farcical, damaging mess that somehow ends up hurting nearly everyone involved in it in some way, at some point during all that’s happened. Just because it didn’t play out as Twitter often made out, just because it wasn’t a simplistic as it was sometimes made out to be, doesn’t mean it wasn’t deeply damaging and troubling.
And yet Gavin Williamson is still education secretary, at the time of writing this…
Wandering Thoughts II - the Sequel 