Global Education Episode 22: Technology in education – a tool on whose terms?

This podcast from Teacher is supported by MacKillop Seasons, whose Seasons for Life project supports schools with loss and grief following a suicide and other loss event.

Thanks for downloading this podcast from Teacher – I’m Jo Earp. At the end of last month we brought you news of the release of UNESCO’s 2023 Global Education Monitoring (GEM) Report, Technology in Education: A tool on whose terms? This major international report draws on analysis of over 200 education systems, highlighting the benefits, opportunities and the challenges of technology in education. In this episode I’m joined by GEM Report Director Manos Antoninis to discuss the 6 key messages delivered by the team, including what governments, systems and schools should be thinking about when planning to bring technology into the classroom. So, let’s dive in.

Jo Earp: Hi Manos - it's good afternoon from Melbourne actually and a good morning to you there in Paris. Thanks for joining us here at Teacher for this Global Education podcast. So, we're going to be talking about the data, but can you start, first of all, by explaining to listeners what the Global Education Monitoring Report actually is, and its purpose?

Manos Antoninis: Thank you for the invitation. The Global Education Monitoring report was established in 2002, so it has a history of more than 20 years, to monitor progress on education in the International Development agenda. At the time that was called Education for All. These were targets set for 2015. And it is quite an unusual publication because it is an editorially independent report – of which there are not that many in this field – that is hosted and published by UNESCO. So, there was a kind of relationship of mutual trust that UNESCO could continue playing the monitoring function that it has, but it would assign responsibility for the publication of a quasi-annual report to an editorially independent team.

This experiment in 2015 was kind of rewarded by this time a global mandate from the world's governments to continue monitoring progress, this time in the new development agenda – the 2030 agenda for Sustainable Development, and in particular SDG4 [Sustainable Development Goal 4] (the education goal), all the way to 2030. And we produce a report that usually has 2 parts, because our mandate is twin: to monitor progress on education, and that means quantitative monitoring of the indicators that are in the SDG4 framework; but also monitor the implementation of national and international strategies to achieve SDG4 – and that is with the objective of holding all partners to account for the commitments they made at the beginning of this process. And that's why every report has a theme that follows one of these set of strategies that are identified in the road map for achieving SDG4, it’s called The Framework for Action.

JE: And the theme this year, this particular edition, was technology in education. You collected data from more than 200 countries. I'm interested on the timing and the demographics of that. So, first of all, when did the data collection take place and who were the participants? And then, I was thinking, you know, 200 countries, that's a massive undertaking.

MA: I mean, first of all, as I mentioned, the report has 2 parts and the monitoring part (the more quantitative, hard quantitative part), there's a process behind that that has been going on for 20 years. The main source of the comparative data is the UNESCO Institute for Statistics, but we also use several additional sources of information to inform our global audiences of progress towards SDG4.

But when it came to the issue of technology, apart from the classic literature reviews that we do and selected background papers that also contribute to the evidence base, it's been now 3 report cycles that we also collect additional information through what we call the PEER [Profiles Enhancing Education Reviews] website, and I will explain that in a second.

Essentially, we realised that even though the report usually covers about 150-200 pages of material on a theme, still there was a bias. Many countries are covered much more than others – for all sorts of historic political, economic, curiosity reasons. Other countries are left out and we felt that we really needed to fill that gap. So, we started a process whereby on the theme of the report we prepared, from our team (in a few cases with external collaborators), country profiles on selected issues or upon laws and policies that address this particular theme. And then we compile the information in selected indicators that give us a bit of a sense for that point in time. What is the global status? How is the world moving? We can disaggregate that by region and by country income group to give a general sense.

We don't report yet this information by country because, as you rightly said, it is the massive amount of information collection. It is always possible that countries may contest individual conclusions we reach for their own country. We don't want to enter into that particular challenge. We just want to give a general sense of where the world is going, and, so far, this has served us well. In any case, information is available on our PEER website, which is at https://education-profiles.org/

JE: Thank you, and I'll put a link into the podcast transcript, which is on the Teacher website for those who want to have a look there. I'll also put link into the full report, the full GEM 2023 Report – that's got a wealth of information, there's lots of examples from around the world. There's also a summary version – although that one is still 34 pages, so it really does have some great insights in there! So, I'd recommend that as well.

Before we dive in then, I want to read this overview from the report. It says: ‘The adoption of digital technology has resulted in many changes in education and learning, yet it is debatable whether technology has transformed education, as many claim. The application of digital technology varies by community and socioeconomic level, by teacher willingness and preparedness, by education level and by country income. Except in the most technologically advanced countries, computers and devices are not used in classrooms on a large scale. Moreover, evidence is mixed on its impact. The short- and long-term costs of using digital technology appear to be significantly underestimated. The most disadvantaged are typically denied the opportunity to benefit.’

So, I just wanted to read that to give listeners a bit of a context to that. In that Summary Report, that I mentioned, you've identified 6 key messages, and the first one is: Good, impartial evidence on the impact of education technology is in short supply. What have you found on this point?

MA: OK, first of all, it's good that you remind the audience that this is a global report. We cover countries from those that your audience may find it hard to believe, have no schools with electricity, to of course countries that already are applying – maybe on an experimental, but maybe on a wider basis – facial recognition technology in classrooms. So, we have all the extremes and it's of course quite difficult for us to strike the right balance and convey messages that can apply to all these countries.

In terms of the evidence, it's true that our report generally tends to deal with themes that are relatively established – we know we have a good evidence base and a clear sense of how they contribute to the achievement of the global education goal. Technology is not one of those. Technology is evolving rapidly, and every product is estimated to have a life cycle of about 3 years, then there's a new cycle. And of course, there are multiple technologies with very limited time therefore to evaluate, and with concerns that the main providers of evidence are the providers of technology itself – and that generates some concerns as to the impartiality of the evidence.

I think the one thing that is particularly striking, is that we found hardly any evidence that governments have units that evaluate the usefulness and effectiveness of technology. Perhaps one of the rare examples of not exactly one would say an evaluation unit, but at least a repository, is that of the Clearing House of the United States, of the Federal Government. And an investigative report evaluated that there were about more than 10,000 products and applications listed, of which only 2% had some evidence that they were moderately effective.

That, in itself, I think should serve as a signal. Of course, it's very clear that it's impossible for every government to have such a unit because the costs of setting it up are high. But one would expect a bit more collective action on the side of the largest users of technology – which is governments around the world – some common actions and some more lesson sharing, and some more process, not just leaving it to academic research, which inevitably sometimes may be focusing on relatively narrower issues.

I think that's one of the problems we encountered. Often the questions of research are, let's say, formulated in a rather narrow way. You know, a very specific product, a very specific target group, a very specific learning outcome. And yes, we do find evidence that in such tightly controlled environments, yes, there is some moderately positive impact. But the repercussions of technology much broader, and we could not find good evidence that really looks at all of these repercussions at a broader scale. Because, as we know, technology today is ubiquitous and it's difficult to identify precisely what is the contribution that technology is making.

JE: And, as you say, it changes so quickly and by the time you've done that evaluation … but like you say, again, you'd at least hope that there was some thought going into, or questioning of: What is the impact of this? What is the evidence behind this? I want to talk about access and equity. That's a big theme running throughout the report, obviously, and that's the second key message that: Technology offers an education lifeline for millions, but excludes many more. And this is something again, throughout the report, that's kind of that tendency to look at just the challenges, but there are some positives here, but there are also the negatives, aren’t there?

MA: Well, again, it depends how we define technology. There is a tendency, and in fact I could make a small diversion. The report inevitably focuses on the impact of digital technology on education, but the theme of technology is much broader, and the report also covers the impact of other technologies. For instance, not ICTs but construction, energy, transport – these also have impact. It also covers, only one chapter – even though it is as important, if not more important than the question of the impact of technology in education – the impact of education on technology, and that is perhaps the one issue that is the elephant in the room. Because without education, you cannot achieve technological development.

But, leaving that aside and returning to the question of access to technology, there's no question that technology has offered multiple advantages to student populations. Australia, of course, is one of the examples of countries that have applied distance education technology for its remote populations for decades with success, with many lessons learned and shared all over the world. Other countries have also benefited and continue to benefit for their remote populations through low-tech approaches; based on radio, for instance, or television.

There is, of course the very particular group of learners with disabilities, for whom technology is more than just a lifeline. It completely changes their interface with learning opportunities. And we know that people who are starved for time and other resources also can continue being adult learners through the help of technology, and there are several examples of that kind. Of course, the peak was the opportunity that technology afforded us all during COVID-19.

However, COVID-19 is also to some extent the unravelling of what technology is offering to some and not to others. And we know for sure that at least one-third of people on the planet did not have any opportunity for any type of distance education technology. But even among those who did, there were wide disparities in terms of the and quality of the connection that they had, in terms of the support and resources they could have at home, the availability of devices that could facilitate the learning.

And these are issues that continue existing because always the technology is advancing, the requirements become higher and higher, and they're always people who are left behind. So, I think there's a wide consensus that the digital divide is a major concern.

Of course, we know also that education and technology need not be limited to distance education. The question is: What is happening in the classroom? And there, there are other issues here. Technology, in principle, offers a promise for some types of learning – I think it's important to stress, always, some types of learning – for those who may be lacking sufficient support at home and maybe getting extra support by teachers in the classroom. But there's not that much evidence to prove the point that this is indeed happening, and I think that's another point that the report is making.

JE: And just that, as you mentioned earlier, the divide – the huge divide between some countries and schools who are using adaptive technology and high bandwidths and so on, and then you've got huge chunks of countries that do not even have access to electricity, let alone devices. The third key message that you make in that summary report is (and you've touched on this): Some education technology can improve some types of learning in some contexts. So, what does the evidence say here?

MA: Yes, I think it is an important message. There is a tendency, again mainly by technology providers, to present technology as a blanket positive solution to all problems of education. And yet, technology, generally speaking, has not been developed for education purposes. What we often see is people who are quite optimistic about the potential – and that's why this report struggled, because we're not really used to talking about the potential things, we try to look at proof and evidence that things are working in particular ways. So that has, time and again, proven to be false promises of transformation. You mentioned at the beginning, the issue of the fact that education has not yet been transformed. We used the evidence from one of the main comparative learning achievement surveys – the 2019 TIMSS [Trends in International Mathematics and Science], which is focusing on science and mathematics in, mainly upper-middle- and high-income countries. And there we found that only 10% of learners had at least one hour of exposure to the use of computers in the science and mathematics classes in grade 8. Australia was, of course, one of the exceptions because it was one of the top 3 countries that were using technology in classrooms, alongside a couple of Scandinavian countries.

But, overall, we see that technology has not yet been integrated in a systematic way around the world. And probably that is a matter of cost. The investment that it takes to make that happen, probably a matter of training the teachers. It's probably a realisation, I think that's the most important thing, of the huge steps that would need to be made in pedagogy, because that has been the classic of problem of most technology interventions – there has been an emphasis on digital inputs and yet very limited thinking behind: How do these inputs work in practice? How do they need to change the way a classroom is organised and managed, the assignments are given, and so on?

And I think that's where the education systems struggle, because they have not made that investment, yet, that necessary supplementary investment. And it looks as if, often, people are moving a bit in the dark, and that's why we don't even have the research that would be needed to support an investment in that direction that would be systematic, organised. Again, the examples of countries that have done that systematically are very limited. And we found, to our surprise, that even some of the most advanced in that respect have really only done a patchwork of individual interventions that work in particular subjects or in particular grades, or in particular … there hasn't been no systematic attempt to digitalise education internally. And that's because, I would presume, that governments are not yet convinced. They like to think of the need to transform their societies in a digital direction, but they haven't thought through the implications of that for education.

JE: This, you mention in the report about, for example, you know this idea of some countries just bringing in the tech, thinking about the tech, not thinking about the teachers, not thinking about the students, not particularly thinking about what the outcomes are going to be. And one of the cases you mentioned is of I think it's Peru, isn't it, where they brought in a one-to-one laptop program, but they didn't essentially train the teachers in it. And so obviously, that had not much impact on student outcomes. Like I say, it's interesting what's happening around the world. Interesting that you haven't found anybody doing that really effectively and consistently. OK, the next area then I want to talk about is the pace of change. We've touched on this as well, about how quickly products evolve; the difficulty, I think, for students and teachers as well, and then of course systems in trying to keep up with and adapt to how quickly technology is developing. What are the findings around that key message?

MA: I think there is an important divide that the report tries to bring up. The report is not sceptical about the usefulness and importance of technology. Every learner should be exposed to technology, should learn about technology. There's a chapter on digital skills, and the report makes an appeal for all governments to think through, carefully, what it means to be digitally literate and have the right competencies to navigate the digital world. We find, for instance, the risk there because there's a tendency – again, often led by commercial providers – to narrow down digital skills to individual, specific competencies that are related to the world of work, but quite narrowly defined, often leading to some certification that is also offered at a price.

We stress the importance of the approach that European countries have taken in defining digital competencies for citizenship and thinking more broadly about what one needs to have to be able to do that; and it's also important, in the sense that, if you dig deeply, what you find is that, ultimately, learners who have better reading and mathematics skills are far less likely to be, for instance, duped by phishing emails or misinformation on the web. So, it's interesting that if you dig down, you ultimately find that to be a digital citizen, ultimately, you need to be well educated, you need to have the foundations that ultimately lead you to develop the competencies and skills that help you navigate the digital world. So, and overemphasis on technical skills is potentially misleading and diverges away from the real purpose of education.

But I mentioned that the report is, of course, in favour of a good discussion on how to teach students about technology. It’s linked a little bit to that other relationship I mentioned – how education can affect the technology, rather than how technology affects education. But teaching students about technology – and we also know that students tend to know more about it than teachers or their parents for sure – is different to insisting that teaching needs to be done through technology. And that is a clear distinction that is, of course, somehow difficult to convince policymakers about. And technology providers like to see the 2 as one. But that would be a mistake because there are many ways to teach about technology; and as I mentioned before, we have a relatively limited proof, because of the constant changing of products that are being offered as solutions. Whereas we know for sure that no product offers a solution, the solution is pedagogy.

The solution is how you teach, how you integrate different types of resources and how you facilitate the flow, critical flow of information and the exchange between teachers on their experiences, building confidence, being clear about things that do not work, having a say in decisions being taken, not being offered packages of such solutions that are externally imposed and not sufficiently consulted.

So, the report is basically trying to make 4 recommendations in that respect, asking policymakers before they apply technology – and we don't go into the specifics of any individual technology because it would be unfair, given the amount of technologies that exist – to ask these 4 questions: Is the technology appropriate? That means is it context specific and is it leading to improved it learning?; Is it equitable? As we just discussed before; Is it scalable? Because many of the examples, as I was saying, are very narrowly construed and when you really need to scale them up, you need to be fully aware of the total cost and the evidence behind that; And fourth, is it sustainable? And that is not just economically sustainable but also socially sustainable. We know of all the difficult side effects that threatened wellbeing, safety and privacy, for instance, but also environmentally sustainable. Because, if everyone in the world jumps on technology today, if that's what we're aspiring for, the implications in terms of energy and materials are quite vast, and I don't think anyone has stopped for a moment to think what it is that we're asking for.

We’ll be back with more from my chat with Manos Antoninis after this quick message from our sponsor.

You’re listening to a podcast from Teacher magazine, supported by MacKillop Seasons, whose Seasons for Life project supports young people affected by suicide and other loss events throughout Australia. Free for Australian high schools and based on the strong evidence-base of the Seasons for Growth change, loss and grief education programs, the Seasons for Life project builds wellbeing, resilience, social and emotional coping skills, and strengthens supportive relationships.

JE: The next key message is certainly something that's featuring heavily in discussions around AI at the moment, and I find this area fascinating, and it plays into that point that you were making about digital literacy, information literacy. Where is the content coming from? Who's regulating that? Your message is: Online content has grown without enough regulation of quality control or diversity. Can you give some examples of what this might look like for K-12 education?

MA: I think that this message was perhaps more related to the non-English speaking world, which is the majority, but the types of resources that are available globally these days are dominated by English language providers. And that, in itself, or, you know, maybe some of them are translated, but the translation of a resource does not make it necessarily context relevant. And I think it is an alarm bell that we ring.

We launched by the way the report in Uruguay, Uruguay has been considered as one of the countries that have championed the use of technology. It originally fell into the same category as Peru. It invested in one laptop per child. It realised very soon that it had failed, even though huge political stakes had been placed there. But rather than abandon the effort, they did try to say ‘OK, what can we do to correct, improve the situation?’ It's not that education has been digitalised, but they have introduced, selectively, platforms that try to keep it more context specific. And I think they have been trying to reverse engineer a little bit. And they call that ‘digital sovereignty’.

It's an interesting term, because these days we like to think of more multilateral solutions and not think of country specific interests or isolationism, but I think digital sovereignty in their definition is more interesting because essentially it tries to ask ‘OK, all these externally available resources, what can they do for our learners? How relevant are they?’ And I think very few countries have gone down that route. Yes, there's a wealth of information at the tip of our fingers for all learners, regardless of the language we speak (and of course, English is becoming ever more spread and understood by millions and billions) but, still, for the vast majority of learners, English language resources are not relevant.

And there needs to be a process whereby countries study what is available and don't just have repositories where you can find them, but start curating, thinking through what is really relevant, what needs to be produced in country and how this is going to improve learning, etcetera. This is a very, very long process and I think very few countries have embarked on that path.

JE: That's fascinating. And it has impacts as well on research and evaluation and, yeah, the publishing language of that and the accessibility and the opportunities for people to do research, other than in English. The final key message then in the summary report, as I mentioned there are 6, is: Technology is often bought plug a gap (I think there will be a lot of people nodding their heads there, a lot of teachers and leaders out there, nodding their heads) with no view to long-term costs, and this is not just a financial cost, it also includes cost to children's wellbeing. And that's the message actually that's really grabbed the headlines from this report, isn't it, the recommendation that all schools should ban smartphones. Can you comment a little bit on the long-term financial cost aspect? You've kind of touched on that earlier. And then also on the smartphones, what you think about that?

MA: I mean, the report has a history of being interested and primarily concerned with the world's poorest countries and our cost implications relate to them. Because, still, in the international development agenda, and among financiers, they still desire to invest in technology. And so we basically remind these stakeholders that we have estimated that, for countries to achieve the national SDG4 targets – this is a process that the GEM reports has been working on with the UNESCO Institute for Statistics, to encourage countries to set national targets for selected SDG4 for indicators; that's a very, very important process that other sectors don't have, targets for 2025 and 2030 – and we have estimated that these 80 low- and lower-middle-income countries face a financing gap of almost $100 billion per year between now and 2030 to achieve their national targets.

Investment in technology, depending on the scenario; a scenario that is very, very meek – offline solutions for low-income countries and school connectivity for lower-middle-income countries – would add, essentially, another 50% to that cost. And we have, so far, found no evidence that any of that cost could substitute any of the resources that are currently missing. Maybe potentially some savings, more savings in textbooks could be made if we had the resources online, but given the difficulty of people being online – and therefore the continuing importance of printed textbooks for these learners – even that potential saving is not likely to come anytime soon.

So, we do a raise the alarm in that respect, that we need to be very cautious in how much we invest when children lack teachers, they lack teaching and learning materials, they lack classrooms that are in conditions that are suitable and appropriate for learning, before we invest in other, more advanced perhaps approaches to teaching and learning.

We, of course, as you say, mentioned and have an entire chapter on the need for regulation for a range of impacts on wellbeing, privacy and safety. The issue of the mobile phones attracted perhaps excessive interest compared to, as you understood from this conversation, the very wide range of topics we covered. It seemed to have touched a raw nerve. It's true, we identify that a large number of teachers are concerned and affected; they believe that the presence of phones in the classroom is distracting. There is also some academic evidence that is emerging that suggests that the presence of phones has a negative impact on learning.

Again, one should not be applying a blanket approach. Some people argue well, yes, in poor countries, having a mobile phone and resources accessible could be helpful given that other resources are not available at all. That's also possible. But for countries where people do have access to phones widely, which is the majority, there seems to be a consensus that the presence of the phone in the classroom is a problem. In fact, we don't make an explicit recommendation on the ban. We simply make the recommendation that any investment technology should improve learning. Now, if there's evidence accumulating that having phones in the classroom does not improve learning, then indirectly that can lead to this conclusion. But we know that countries differ far too much in how they conceive bans. There are some countries that are a bit more, let's say, authoritarian – they like the concept of having a law that bans and then everyone follows. There are many countries where decisions are taken at the school level, and schools ultimately have the decision what to do or not to do. So, I think we're very, very clear about that.

But the reactions that we have found all over the world has taken us by surprise, because it suggested that that was an issue that needed to be higher on the agenda, and we're happy that that it has led to these discussions.

JE: It certainly prompted a lot of discussion. I've just got one final question, actually, which I was thinking about. There are lots of – I want to make the point that there are lots of really positive impacts that you mentioned in the report, it's not all sort of doom and gloom. There are some focuses on the challenges there and, like you say, there's a tendency to do that and ring the alarm bell. My final question is then: What are you hoping for in terms of the impact of this report, and the separate country profiles as well that we talked about? What would you like to see happening now in terms of policy and school discussion?

MA: I think our preference would just be to help policymakers reflect. The report starts with 7 case studies from different parts of the world. Some are well known champions, like I mentioned Uruguay, or Estonia, for instance. But there are also other countries: there’s Egypt, Rwanda, Nepal, Samoa, Singapore. Examples all over the world that seem to suggest that technology is seen by policymakers as a signal of an intent to bring progress to education.

And what we're trying to say is that, not all that shines is gold. Yes, we are embarking on digital transformation in so many ways. The discussions on AI accelerate even though we retain some scepticism how much that is applicable to education, despite what is being said or how much that could change. The policymakers need to pause and ask these 4 fundamental questions, which we have found their absence. There's an enthusiasm to apply the latest gadget or the latest technology, but no enthusiasm to invest in the research that will prove that any of these tools are useful. And that's what is reflected in the title of the report, the report is called Technology and education: A tool on whose terms?

We have to see technology as a tool that can be applied well or can be applied poorly. That discussion is missing, and we would like to see it. And, on whose terms, it's the terms of the learner – technology should be applied in the learners’ and the teachers’ best interests, it should not be imposed as a solution of doubtful applicability in quality. So, to the extent, that as part of our launch in our campaign, which is called #TechOnOurTerms, these discussions can be had, then of course we would be very happy about the impact of the report.

JE: Well, there's plenty of food for thought there. It's been a fascinating discussion. It’s been great speaking with you today. Manos Antoninis, thank you so much for sharing your expertise with Teacher.

MA: Thank you so much for inviting me.

That’s all for this episode. If you want to keep listening, you can access more than 280 Teacher podcast episodes from our archive at teachermagazine.com, or wherever you get your podcasts from. Before you go, I have a quick favour to ask – please take a few moments to review our podcast; it helps other people like you to find it and it’s a big support to the Teacher team. Thanks.

You’ve been listening to a podcast from Teacher, supported by MacKillop Seasons, Seasons for Life – supporting schools and young people affected by suicide and other significant losses. Visit mackillopseasons.org.au.