The Score and The Unaccountability Machine

A bit of a ramble through some recent reading

Among many brilliant things in Middlemarch is the image of perception as being like a candle held up to a reflective but scratched metal surface. No matter where you move it, the mad hatching of random marks catch the candlelight in concentric circles, with the flame at the centre. Thus we look at the world, and see only confirmation that we are the centre of it.

Sometimes I read things, and I find myself mentally crying out “Yes, that’s what I was thinking!” Sometimes the pile of reading by my bed coalesces and seems to all tell me the same thing, the thing that I was already thinking. Sometimes this is illusion, but other times it all seems too much of a coincidence not to be speaking to some moment or change.

I mentioned The Score last week. It’s a fascinating book. It’s not about education per se – neither of the books I talk about in this post are – but it seems to me to articulate something I was reaching for in Exam Nation. It takes as a central premise the question of games and metrics. Why is it that rigid scoring systems in games lead to joyful play, while rigid metrics in public life lead to stifling and joyless institutions?

In answering this, Nguyen takes us through a fascinating analysis of the ways in which we interact with games. He talks about the way the scoring system provides a goal that we focus on with huge intensity during the course of the game, then discard without thought once we finish. The goal is a McGuffin: the true purpose that the focus on the goal enabled was the ‘striving play’ during the course of the game. Slowly he pushes and probes at how we can attain this pure, playful, process-focused state of mind outside the world of games. As I read this seemed to me to encapsulate the dilemma I feel when talking about exams. The scoring system – the goal of the grade at the end – should be simply a navigation aid, while the true educational purpose is being accomplished en route.

I highly recommend the book. But from my educational perspective, it seems at times to glide over the most obvious reason we sabotage this sense of playful striving in education. Any game becomes deadly serious when something of great value rides on it. I might enjoy Scrabble of an evening to relax, but I would probably bring a very different attitude to it were the penalty for losing lifelong penury.

Of course it’s more complex than this. And the book’s answer is subtle and interesting. But in its discussion of how metrics and therefore outputs guide behaviour, it took me back to Dan Davies’ The Unaccountability Machine, which I finished in October. This one wasn’t the easiest read at times. It’s an account of management cybernetics, and the way the structures of managing large organisations can result in ‘accountability sinks’: instances where no one takes responsibility, and simply defers to mechanistic processes. ‘The communication between the decision-maker and the decided-upon has been broken – they have created a handy sink into which negative feedback can be poured without any danger of it affecting anything. Essentially, its all about a shrug of the shoulders and the response ‘there’s nothing I could have done.’

Anyone who has gotten lost in the maze of automated phone line or chatbot responses has experienced a version of this. The crucial element, though, is not just that individuals duck responsibility. It’s that large systems are designed to avoid the possibility of it. Think of the financial crisis of 2008, and the way in which no one in the banks that undoubtedly bore the blame could be held to account.

There were two crucial things that stuck with me from the book. One was the mental discipline of thinking of organisations as a series of black boxes. Input goes in, output comes out. Which, of course, as a teacher of a certain age got me thinking about Paul Black and Dylan Wiliam’s ‘Inside the Black Box’. The thing is, Davies describes the way we should judge the mechanism on the basis of the output. He says ‘The purpose of a system is what it does.’ (A caveat: he means the habitual operation of a system. In his words, a broken air conditioner that emits thick smoke does not have the emission of thick smoke as a purpose. You have to look at the day to day operation.)

This could bring us to some very interesting conclusions about education. If we treat the exam system as a black box, there is no doubt that it has as a purpose ‘to sort and rank people.’ There is also a case for saying the purpose of it is ‘to tell people what they can’t do.’ In fact, one could also argue that the output of the exam system is ‘to frame thinking as compliance to authority’.

No matter how you phrase it, though, the narrowness of the metric we use for output – the grade – makes it impossible to see the system as having as its purpose that far more vague and amorphous concept of ‘education’. And this is where the link to Nguyen’s book becomes fascinating to me. He talks about the seductive power of metrics as being all about spurious clarity, and, in his phrase, ‘laundered objectivity’. They take the vague, contradictory and complex qualities of life and give them a falsified objectivity by slapping a number on them.

The decisive stage that he identifies, though, is that of ‘value capture.’ This is when, instead of holding your own internal values, you farm them out to the metric. Nguyen uses weight as a really good example. You want to get healthier – so you use your weight as a handy metric by which to measure it. But weight on its own does not equate to health. Only, the seductive clarity of it as a number that can be seemingly objectively measured makes you vulnerable to being captured by it. You obsess about weight, until eventually, your health starts to suffer – exactly the outcome you were trying to avoid in the first place.

Sometimes it can seem to me in education that we are in an accountability sink, where teachers just shrug our shoulders and say ‘what can we do – the system is as the system is.’ The necessity of grades for our students leaves us no option but to teach to the grades, and the process leads to a wholesale ‘value capture’ whereby we talk about numbers as directly equating to education.

But this is only one version of our system. Because, of course, just because we label grades as the key metric, doesn’t mean they are the only output. Laughter and tears on results day are an output of our exams black box just as surely as grades are. My chapter on Paradise Lost in a recent book is a distant output of my study of it at A-level. And, sadly, the parent who tells me in a meeting that they can’t help their child with homework because they’re ‘thick as shit’ is also an output of their long distant time at school.

There is a difference between the value capture of our large scale educational institutions by the exam metrics – and the more nuanced reality of being a teacher. In practice, all teachers with their classes perform a version of what Nguyen calls ‘Reflective Control.’ This is the process of playing with rules and scoring systems, adopting and discarding them according to need and judgement. While I might in one sense be pushing for the grades as the goal for my students, I will also be thinking about this one girl who will never pass but who desperately needs to feel better about herself in the wake of significant trauma, and this other boy who is going to get a nine no matter what but really, more than anything, I want him to stop worrying so much and take pleasure in the texture of poetry because I think it will give him solace throughout his life. On results day, I won’t be simply congratulating those who got the highest grades – I’ll be speaking to those who went on journeys, who gained most, who discovered something about themselves.

Despite the pervasive metrics, we can actually choose our values. We don’t need to be captured. But the second insight I took from Dan Davies’ book gives some really important context to this.

This was about the vital importance of layers of control in systems. In the book it was complicated, and involved some weird diagrams. But, in as simple terms as I can make it, Davies describes the way in which you need five systems that sit on top of each other in order to manage a complex organisation.

You need the operations – the bit that does the stuff. You need a regulatory system that checks the operations (like a thermostat for a heating system). You need a system for optimisation, which integrates the different elements together as they work in the “here-and-now”. And you need system 4 which gives you strategic leadership, the capacity to look forward and backward and make decisions about direction.

In a school this might equate to:

System 1: the classroom

System 2: assessment

System 3: middle leaders

System 4: senior leaders

But I said there were five systems. And the fifth is fascinating. Davies describes it as philosophy. It is the system that provides the rationale, the belief system, the deep sense of purpose and identity. In schools, we have a place where this should be. Most schools and MATs label it as a ‘Vision’.

What is striking is that this piece of corporate-speak comes from precisely the analysis that Davies gives. It is a product of a world where structures defined by management cybernetics have been applied to schools. It’s easy to see this as an empty and destructive process, and to rail against the very concept of something as soulless sounding as ‘management cybernetics’ having a place in schools. And yet I think we have to ignore the poor examples of ‘Visions’ we might see, both in the corporate world and in education, and notice the clarity and insight of the analysis it springs from. Because Davies, and Stafford Beer (whose analysis forms the basis of the whole discipline of management cybernetics) show that of course we do need to have a vision, a philosophy, a guiding sense of identity to our institutions. Its just that we run the risk of that philosophy being captured by the narrow metrics of the exam system and Ofsted.

So, in a round about way, this is where I end up – having bent these two (non-education) books around the flame of my own perspective. Yes, if we see the outputs of our exam system as narrow metrics, we can experience a form of value capture that destroys the nuance and subtlety of what true education means. But the existence of those metrics doesn’t make that value capture inevitable. There are other philosophies available. It’s just that we need to think properly about what our schools are for – not only as a nation, which I have been advocating for some time. But also as individual schools, and individual teachers.

Let’s not settle for the bland. Teachers and schools need to openly discuss, with nuance and contradiction, their purpose and philosophy. We need to keep an eye on the score insofar as it provides us with a goal. But that isn’t the same as a purpose. We can exercise reflective control in the way we respond to the individuals in our care by thinking about when that score is important, and when another value system might be better.

Most importantly, we need to bring this into the open. We have to stop talking about education and school as singular, as measured on a neat scale, as uncomplicated. Leadership should be about expressing the contradictions of our system openly, and plotting a way through that allows a deep philosophical understanding of what education is. Instead of empty bromides about excellence, I want to see vision statements on school websites that say:

“We believe in the fact that that knowledge and understanding are more important than grades, and yet that grades open doors; that all experiences teach us something, both positive and negative; that challenge and risk are vital ingredients, but come with the real danger of damage and failure; that exams sometimes get it wrong; that school doesn’t have all the answers; that life’s both a deep joy and a bit shit; and that fairness is not always possible, but yet the deep understanding that it isn’t always possible can give us access to a more profound and complex sense of meaning.”

I mean, you have to dream, right?