More media statistics misuse

There was a fair bit of media coverage this week about the results of a survey, published by the Association of Teachers and Lecturers, which suggest that nearly three out of every ten teachers had had a false accusation made against them by a pupil. For this to happen to a teacher is undoubtedly horrible; for it to happen to so many of them would suggest that problem is endemic, and can only be the result of a hysterical child protection culture¹.

As always with these sorts of figures, you have to ask, does it sound realistic? Can it really be true that nearly one-third of all teachers have had a serious allegation made about them by a pupil? And, as always when it comes to surveys and the media, the answer is “no”. The ATL survey tells us that:

The survey was completed in May 2009 by 1,155 ATL members working as support staff, teachers, heads of departments and school leaders in state and private schools around the UK.

Have you ever had a false allegation made against you by a pupil? (1,155 responses)
*  28% said yes
*  72% said no

That’s 28% of the 1,155 responses that the ATL got, of the 1,155 members who actually completed the survey. It’s not three-tenths of a random sample of one thousand teachers: it’s three-tenths of the 160,000 ATL members who actually went to the trouble of filling out the survey. The ATL’s Mary Bousted admitted as much on Radio 4’s Today programme (see the interview at 7:50am).

The problem is that it’s more likely that teachers who have had false accusations made against them will care enough to complete such a survey. They will have much more motivation to do so than a teacher who has never encountered such problems. This is a form of response bias, and basically means the 28% figure is very likely to be larger than the real figure. As very often is the case with the statistics used to drive media stories, it turns out to be meaningless upon even a cursory examination.

¹ The Daily Mail, showing their usual disdain towards consistency or accuracy, has trouble deciding how many teachers have gone through such an ordeal. While the headline of the story suggests 1 in 4 teachers, the title of the web page increases this to 1 in 2!