Consider the following three statements:
- “Converting secondary schools to academies improve GCSE attainment”.
- “Peer coaching is an effective form of teacher CPD .”
- “Learning to learn strategies lead to improvements in pupil attainment.”
These are all claims that have been made about education. The question is, do we have any empirical basis on which to decide whether these claims are true or false?
One option is just to say “no, we cannot”. For some, the social world is so complicated that any attempt to compare an intervention in any unique environment is doomed to fail. Every school is different; every teacher is different; every pupil is different. Therefore, we have no way of deciding whether or not any given change we make is effective, and so it is just down to everyone in their own context to work out what works for them.
There are some serious philosophical defences of this position, but it is worth noting that some people try to cut it both ways. If you are claiming that we cannot claim that ‘direct instruction’ is best practice because of the complexity of social world, then similarly you cannot claim that making a school an academy does not improve results: either we can measure the efficacy of an intervention, or we cannot.
Alternatively, your view might be “it is possible to decide on the efficacy of an intervention, but this requires a clear statement regarding measurable outcomes and a very carefully designed experiment”. This is probably my position.
My concern with empirical work in educational research is not that it is impossible to compare the efficacy of interventions: I do think it is possible to compare the same intervention in different contexts and to reach an overall generalisation about its efficacy. My concern, rather, is that empirical educational research is often poorly conceived, with weakly-bounded and loosely-defined interventions being used , with the result being that no one is really any the wiser by the end of the study. Any study into the efficacy of ‘group work’ or ‘teacher-led instruction’ is doomed to fail, as these terms can mean a wide range of things, and as such replicating them is very difficult to do.
If we’re serious about empirical research on ‘what works’, we need to be far, far more specific in terms of what we are trying to find out.