Thank you for downloading this podcast from Teacher magazine – I'm Jo Earp.
Results from the latest cycle of the PIRLS assessment (that's the Progress in International Reading Literacy Study) show the literacy achievement of Year 4 students in Australia has improved. However, there are significant achievement gaps by gender, Indigenous status, socioeconomic status and school location. The Australian Council for Educational Research has just published the details in PIRLS 2016: Reporting Australia's results and the lead author of that report, ACER Deputy CEO (Research) Dr Sue Thomson, joins me now.
Jo Earp: Welcome to The Research Files. Before we delve a little deeper into those Australian findings, most people will probably be very familiar with the PISA international assessment for 15-year-olds – that looks at scientific literacy, reading literacy and mathematical literacy – but can you just start by giving us an overview of PIRLS?
Sue Thomson: PIRLS is, as you said, the Progress in International Reading Literacy Study and it's a study of reading literacy of students at Grade 4 level. It's conducted on a five year cycle. It's conducted at Year 4 level because that's the age at which students move from learning to read, to reading to learn. So, it looks at children's reading literacy, in terms of literary experience and it looks at their reading skills in terms of acquiring and using information. So those are the two key types of reading that it looks at. It's conducted in a similar way to PISA and TIMSS, for example, it's a sample-based survey within Australia, we sample students and schools. It's quite a large sample, it's about 6500 students, and we test them on this and then they give their feelings and their attitudes about reading, about school, about learning, those sort of things, in a survey afterwards.
JE: And there are slightly less countries involved in this than PISA aren't there?
ST: There's a different range of countries involved in PIRLS than is involved in PISA. PISA is primarily OECD countries and then others who also want to do it. There's a lot more developing countries participating in PIRLS. And PIRLS is growing over years to measure a wider range of abilities of students. So, we used to have what they call prePIRLS, which is for developing countries that are not actually up to this standard. Now that's [been] wrapped into it so that those countries do a simpler version of this but it's measured on the same scale. So, it provides us with quite a lot of information about that.
JE: Great. As you mentioned there the latest test cycle was in 2016 and Australia also participated in 2011, so this is the second time. As we said in the intro, you're the lead author of the report on the Australian results. So, what are those main results, firstly in relation to where Australia sits internationally?
ST: Internationally, in 2016 Australia was outperformed by 13 countries. Some of these are the top performing countries that we would imagine would be there – so, Singapore, Hong Kong, for example. But other countries, such as Ireland, Northern Ireland and England, also outperformed us. However, we have done better than we did in 2011. In 2011 we were outperformed by more countries than that and some of those we've now gained equal ranking with, such as Canada and the United States for example. So, our results in that way have improved and it's been a significant increase in our score, so that's really good.
But I guess the downside to it is that we still have about 19 per cent, almost one in five our students, who are not performing at what's been put down as the Australian benchmark, which is the Intermediate benchmark – compared to countries such as Russia (which is about 6 per cent not achieving it), Hong Kong (7 per cent) and down to countries such as Ireland, in which there are only 11 per cent not achieving that benchmark. So, whilst we're doing better, that better is generally at the top end of the achievement profile and not at the lower end.
JE: You mentioned the benchmarks there. What would students typically be able to do at each of those levels – Advanced, High, Intermediate and Low?
ST: Okay, the benchmarks are designed so that we can describe student learning on a continuum. They range from, as you said, the Low benchmark to the Advanced benchmark. So, if you think about it in terms of that, the Low benchmark is a very basic level. We're talking about Grade 4 students, so when reading simple literacy texts students are able to locate and retrieve explicitly stated information. So, if they're asked a specific question about a piece of text they can locate that in the text and provide an answer to it. They can make straightforward inferences about events and reasons for actions, and they show that they can begin to interpret story events. So, same with informational texts [for the Low benchmark] they begin to locate and reproduce that explicitly stated information.
It ranges from there up to Intermediate, where they're starting to read a mixture of different, simpler and more complex tasks, and can start to independently locate and recognise things that we ask them to do, interpret obvious reasons and choices for things – it's still fairly straightforward, but it's a little bit more complex.
Moving up that spectrum to the Advanced level. In the Advanced level, students can interpret story events and character actions to describe reasons, motivations, feelings and character development with full support from text – so they'll be able to say ‘this is why they did that' and provide the evidence why. They begin to evaluate the effect on the reader of the author's language and style choices, so they begin to think about things like that. Remember, there won't be many kids at Year 4 who can do this, but we need to be able to make it so that we are testing the top kids as well. With informational texts [at the Advanced level], students are able to read relatively complex tasks, interpret that information from different parts of the text and integrate information from across the texts. So, you know, there might be a little bit here, and a little bit in the middle and a little bit at the end and they'll be able to draw that together. It's quite a sophisticated level of reading, especially for Grade 4 children. The texts are set for children at that level, but it is quite a sophisticated understanding of how language and text work.
JE: As you mentioned there right at the beginning, you described the Low and the Intermediate and you were saying that Australia, one of the things we should focus on is moving up from Low to Intermediate – 19 per cent I think you said?
ST: Yes, 19 per cent are either at Low or below Low. We do have a proportion of students who are below the Low benchmark, which is even more of a worry – 6 per cent of our students are actually not even reaching that Low benchmark. That is quite a concern, because that is a very basic level of literacy for a Grade 4 student. The issue with that is that if we have students in a classroom who are ranging from below Low, barely able to read and make sense of their text, to students who are reading at this Advanced level, it is very difficult for a teacher to be able to teach to a class with that wide a variety of literacy levels, and the problem is that these children who are not reaching the Low benchmark or are at the Low benchmark are, if you like, starting the race quite a long way behind all of these other kids. The theory is that yes we can get them to catch up, but while they're trying to catch up the kids who are already achieving at that high level aren't stopping still. They're already reading, interpreting, learning far more than the kids who are still struggling to read. So, that's the importance of being able to place kids on that scale.
JE: You and your team have done further analysis beyond that international placing, for the results for the Australian students. What does PIRLS 2016 tells us there? Are there particular strengths or areas for development? And what about those achievement gaps that we mentioned?
ST: Our students in Australia do reasonably well, and have done better overall on all of the different aspects of PIRLS that we measure. So, they have improved on each of the different processes and each of the different purposes. To me, the biggest issues are the achievement gaps, as you said, and the achievement gaps are still there, according to socioeconomic background – disadvantaged students are still achieving at a much lower level than advantaged students or affluent students – and Indigenous students are still performing at a much lower level than non-Indigenous, and rural and remote students are still not achieving at the same levels as metropolitan students.
So, those gaps are there, we know they're there from last time, we know they're there from PISA, but they're still there. Whilst there has been a lot of programs that have addressed issues of literacy they're not really having an effect in terms of the proportion of disadvantaged students who are doing better.
JE: So, what are the implications for educators then, policymakers as well, but particularly educators?
ST: The implications for educators I think come back to what I said before. We need to tackle these issues, I think well before Grade 4. If they're already apparent at Grade 4, and these issues are already there at Grade 4, then they're not going to get any better from this point on. It really shows to me that we need to tackle, particularly reading literacy, very early on in school and we need to make sure that we address that and make sure that every child learns to read and learns to read well in the early years of primary school. These kids [for PIRLS] are tested at the end of Grade 4, they're going into Grade 5 next year – that's almost secondary school really. If they're not reading well at that stage they're already missing out on so much learning and so much information that they could be getting. It is really critical that students are quite literate, I would venture to say that it's probably more critical in the 21st Century than it ever has been, because there's so much information available so easily and anybody can put anything on the internet, we all know that, anybody can put anything on the internet, it doesn't have to be true or not, nobody necessarily factchecks things, [so we need] students to be able to read things and read things intelligently enough to know whether they're true or not. In terms of Financial Literacy, we need students to be able to read and understand about superannuation and taxation – all of those sort of things that maybe we didn't have worry about so much in the past. Those things are really critical for students and they need to be able to read really well.
JE: Just finally on that gap and starting early, that's been backed up by lots of studies hasn't it, it's very difficult to close that gap once they start into school.
ST: Yes it is, and so we need think about a range of things. People are thinking about these things, I'm not suggesting for a moment that nobody's thinking about this, but we need to look at early childhood education, we need to look at pre-school education. One of the things we found in the report was that having some basic literacy before school made a huge difference to students' achievement in school. [Also] students in affluent schools are overwhelmingly more likely to have had that basis in their education than students in disadvantaged schools.
So, again, you're starting off with this gap when they start school – you've got your affluent kids and your disadvantaged kids, one of whom has far more exposure to literacy, books, learning, all of those things, and the other has far less, and that's reflected in the results completely.
JE: Now, there's lots and lots of information, stats and infographics and all kinds of things, some example questions and so on in the [PIRLS 2016] report and we'll put the link to those on the website. But for now, Dr Sue Thomson, thank you very much for sharing your expertise with The Research Files.
ST: Thank you.
That's all for this episode, to keep listening or to download all of our podcasts for free just visit acer.ac/teacheritunes or you can head to soundcloud.com/teacher-ACER. Of course, you can also check out the full transcript of this podcast and the related reading – including those links to the full report – by visiting teachermagazine.com.au, where you'll also find the latest articles, videos and infographics for free.
The PIRLS 2016: Reporting Australia’s results report is available from the Australian PIRLS website.
