Dr Drew Miller, a Senior Lecturer from the University of Newcastle, recently presented at the Forum on the use of RCTs in Education. We spoke with him leading up to the forum, and in this article, we address some frequently asked questions from delegates about the use of Randomised Controlled Trials (RCTs) in school settings.
Randomised Controlled Trials (RCTs) are a widely-used research design involving some form of intervention being evaluated against another form of practice. The results obtained are able to indicate whether an intervention has made an impact and, therefore, if a new practice produces more desirable results than the practice already in use.
‘Schools are messy places to do research,' Miller tells Teacher. ‘...At the moment it's a new thing and it can be confusing and the trials are not always conclusive and all those kinds of things. We're really going through a rapid learning, expansion of learning in terms of research and what's happening in schools … that will only benefit the students down the track.'
An RCT works things out from a ‘what doesn't work' perspective, rather than a ‘what works' perspective, Miller explains. ‘So, in essence, what that's doing is giving us lots of information about things we need to do better in schools.'
Different research frameworks
According to Miller, the most common question delegates of the forum had was seeking clarification on the difference between efficacy and effectiveness trials.
‘In an effectiveness framework, we go into schools, we get a large number of schools involved in a trial; and in essence, what we're measuring is what's the effect between intervention and control,' he explains.
From a school perspective, this kind of research trial could be achieved by a school adapting an intervention program by fitting it into their particular timetable, and also their school improvement plan.
‘Effectiveness trials tend to be a lot bigger, because there is a lot more noise in what you're doing, and so you need more data,' Miller shares.
On the other hand, efficacy trials tend to be much smaller. Because of this, all possible conditions within the trial need to be controlled for. This is so you can deliver the best possible result. And, the more specific the trial designer is about the subject group, participants and sampling, the more chance a trial is going to give conclusive information.
‘And so there was this real kind of desire for people to get into this, but they were like, “but do we need to trial this with 200, 300 schools? Because we just can't do that.” So that was the main point of difference, and what people wanted to find out,' Miller says.
Considerations for school leaders
The designers of the research trial will take care of all the ethical elements of the RCT, Miller assures. They have a set of ethical requirements that they must fill, and a lot of this information will be passed on to schools.
He stresses it's important to carefully read all of this information, as oftentimes school staff will not read a document thoroughly and not be quite sure what they've signed up for.
‘From the school's perspective, to participate, look at the burden of the study,' he suggests. ‘And if, as school you think that it is too burdensome for you, then just don't participate.'
According to Miller, the fact that there will always be trade-offs and compromises between the trial designers and the school is also something to remember. So, he says, it's important for schools to weigh up their individual needs and requirements and then decide if it's a valuable trial to be a part of.
Dr Drew Miller says it’s important for schools to weigh up their individual needs and requirements and then decide if a trial is valuable for them to be a part of.
Consider this idea with your colleagues. In your school setting, for what purpose could an RCT be useful?
As a school leader, how does evidence inform your decision making when it comes to school improvement?