It’s been popping up on Twitter quite a bit today, and all over the national press it seems. The Guardian claims that “Half of all teachers in England threaten to quit as morale crashes”, whilst similar headlines can be found from The Telegraph, The Independent, The Daily Mail and the BBC.
Whilst the NUT press release does not use the word “all”, it certainly implies it.
Now, I’m certainly not suggesting that there isn’t a morale crisis in state schools at the moment. And I’m not suggesting that I know for certain that 53% of all teachers aren’t thinking of leaving. But this survey and, more specifically, the way in which it has been reported are seriously misleading.
Before I go any further let me make something clear: I am not a statistician. I don’t really do numbers. In fact, numbers and I cross the road to avoid each other. Numbers is the language that Satan uses to confuse mortals. So, please do help me to see things differently if I’ve got all this terribly wrong.
Using the numbers given from the NUT (this is an .xlsx file) we see straight away a problem. The total number of participants is 1020. This may sound like quite a lot, until we see that, according to the most recently available data from the government, “in November 2011, there were 438,000 teachers in state-funded schools in England on a full-time equivalent basis”
I realise, of course, that that figure will have changed. But even if the teacher shortage crisis means we’ve lost 38,000 teachers since then, that would still be 400,000 teachers in England. Using the published number, the NUT survey is dealing with a sample of 1020 out of a population of 438,000. This is 0.23% of the teaching population who responded to the survey. Less than 1 whole percent.
Of those 1020 participants, the NUT data table tells us that 53% said they were thinking of leaving the profession. That’s 53% of 0.23% of the total teaching population in England. Incidentally, the number given in the data table is 536.
The data also tell us that of those 536 teachers who are planning to leave the profession, around a third of them are aiming to retire (34%).
I’ve been told, and have read, that such a sample size for such a population is deemed to be quite good. I find this, in itself, quite staggering, and reason enough to doubt the efficacy of such surveys and such an approach to social science. But, it is only deemed good if the sample is random.
The NUT website gives no indication that I can find about how the participants were recruited to the survey. And so far I’ve been unable to locate any reference to it on the YouGov website. My suspicion is that the survey was conducted online and probably via a link sent to members of the NUT via email, or perhaps through other member communication. This in itself would raise questions for me about how securely warranted any claims that this survey reflects “all teachers” might be. There is an assumption, if I’m right, that the NUT membership is genuinely reflective of the general teaching population. Furthermore, this survey only reflects the views of those NUT members who could be bothered to take the survey. What views are such teachers likely to hold?
I’m not saying there isn’t a genuine issue at the heart of this. There probably is. But can anyone out there tell me how this survey can really tell us anything? Is it typical of social science? If so, educational research is buggered.