Captioning technology to support PD

Voice recordings made by teachers during their lesson delivery are being used to give professional learning feedback in an effort to promote effective classroom practice.

The recordings form the basis of a professional learning tool developed by a team from the University of Melbourne's Graduate School of Education. Researchers worked with real-time captioning experts at technology company Ai-Media to create the Visible Classroom system.

Teachers upload the voice recordings to a personal portal. They are given transcripts of their lesson, which are coded against a rubric to produce a dashboard view of how they performed in certain key areas – such as time spent on teacher talk, the types of questioning used, and incidences of positive behaviour management – and a detailed feedback report.

Lead researcher Associate Professor Janet Clinton, says a developmental trial in the UK funded by the Education Endowment Foundation (EEF) has shown the approach leads to a change in teacher practice. ‘In the EEF trial it was [used for] five hours a week for four weeks, and brought about dramatic change,' she tells Teacher, adding the challenge now is to ensure the change is long-term.

‘[Even] if they use it for one hour a week and it's just willy-nilly use, you see instant change but it doesn't hold up. For sustained change you need to give the teachers a mechanism that's going to support them through. By kind of embedding this process of evaluation and making it all about teacher voice, teachers are the champions here … and I think that's a critical thing with this.'

In the UK pilot study, involving eight primary schools, teachers wore a microphone in class and their speech was relayed back to a transcriber, who produced live captioning. Students were able to access this real-time transcript using tablets. The students also completed online surveys about their experience of the lesson.

The transcripts were coded against a 16-point rubric and the data fed into a dashboard – which teachers could access after the lesson – and the educators were given a detailed, individualised feedback report twice during the trial.

An independent evaluation of the pilot study reported mixed findings. It found that the approach was feasible and most teachers were positive about it, believing it had the potential to change their practice. The online dashboard and feedback report were also seen as valuable tools for teachers.

However, the report found that students didn't seem to follow the live transcripts available to them in class and it recommended further development of the tool before a full trial.

Clinton says a multi-state focus group study in the US is currently underway, working with teachers on how to adapt the tool to better suit their needs. In the UK, a larger trial is planned to test the efficacy and cost effectiveness of the model.

The academic explains the approach used in the UK pilot study grew out of an initial, small-scale trial in Melbourne involving 30 teachers and students with disabilities.

‘[We used] real-time captioning for students who had some form of disability – deaf and hard of hearing or ASD (Autism Spectrum Disorder). We recognised that teachers were looking at their transcript and going “Oh, guess what, I just talked for 40 minutes” and altered their practice, without any feedback, without anything. … They were dealing with students who were deaf and hard of hearing, so they realised they were kind of excluding kids who had some kind of hearing difficulty.

‘We said to the teachers at that point “what if we gave you feedback on your teaching?” and, in that setting, they said that would be good. So, we developed an evidence-based rubric about effective teaching based on John Hattie's Visible Learning work and the SOLO taxonomy … very practically based.

‘What we were able to do is then take the transcripts and code them against those rubrics. Things like: How long are you talking for? Are you giving feedback? How many questions did you ask? How many student questions were there? What was the nature of those questions? How fast did you talk? What were the incidences of positive behaviour management? … And getting into the deep factors like incidences of deep teaching, making connections, incidences of collaboration.

‘There were actually 29 variables that were all factors of effective teaching that, if you look around the world, everybody would say these are the things that you need to consider. So we just used what everybody else is using to be honest, although we did systematically look at the evidence and bring everybody's evidence together and look for the strongest commonalities.'

Researchers looked at the transcripts over time from the Melbourne trial teachers, particularly those working with deaf and hard of hearing students, and started providing feedback.

‘We realised, wouldn't it be great if we could give them real-time feedback as well as them getting their transcripts? So, we decided to take our information and provide the teachers with the transcript and a dashboard of some very basic effective teaching mechanisms.

‘The teachers, within an hour of doing a lesson, have access to their transcript and their basic dashboard that has a number of factors – but the one thing that we did that made the difference is we included questions from the students. It's not about the teacher, it's about the teaching and how the student sees that in terms of their own experience: Did my teacher explain difficult concepts? Did I get an opportunity to learn from my mistakes?'

Researchers then went one step further, taking a selection of transcripts to look for common patterns to give the teachers deep feedback about specific aspects of effective teaching. ‘And, because we've got now tonnes of transcripts, we can compare how that lesson went, or that series of lessons went, with the norm.'

Clinton says the big thing, from a professional development point of view, is that the approach is based in the classroom, teachers are doing it in context, and they're working with cold, hard evidence.

‘And we probably argue that most importantly … we can or [the teachers] can assess their progress themselves right from when they start, right through to retirement. I mean, they can just keep working on something, keep doing it.'