ChatGPT: Education assessment, equity and policy

Much like the smartphone and personal computer that came before it, there is much talk about the impact that artificial intelligence-powered chatbots like ChatGPT will have on students, teachers and traditional classroom learning.

ChatGPT by OpenAI launched in November 2022 and uses data from OpenAI’s language model, GPT-3, to function. It is part of a new wave of AI that can generate highly cohesive, human-like responses to questions or prompts posed to it.

For example, a teacher could ask ChatGPT to produce a lesson plan for a specific year level or grade, and the platform has the capacity to respond with a range of activities, as well as any materials you may need. Students can ask ChatGPT to write an essay on a particular topic or provide written responses to homework prompts. The platform can produce remarkably clear, well-organised, long-form answers to complex questions or topics.

A key concern amongst those working in education is the potential for it to be used for writing assignments or for plagiarism. Since its launch in November, ChatGPT has been banned in Western Australia, New South Wales, Queensland and Tasmanian public schools.

Dr Catherine McClellan is Deputy CEO of Research and Assessment at the Australian Council for Educational Research (ACER). She says that while initially there is some fear or anxiety about the way ChatGPT will impact education, we are currently in a transition period that will pass.

‘I do think that transition period always happens and it’s always a little shaky. And it's scary because we don't already know. It goes outside our established boundaries. That's always anxiety-provoking – the fear of the unknown. What is this going to do?’ she tells Teacher.

The impact of ChatGPT on assessment

A specific concern about ChatGPT is the potential impact on assessment. Dr McClellan says we shouldn’t necessarily be worried. For instance, she shares that some initial studies have asked if teachers can differentiate between a ChatGPT written piece from a piece written by an anonymous student.

‘They are interesting, because to me, sort of the wrong question … I think it might come out differently if you said: “can you detect ChatGPT from your students’ writing?” Where you know how they write, you know their style,’ she says. ‘You know these things as a teacher because you have a relationship with that student.

‘I think that might be the thing, is people are sometimes ignoring that learning is best, most effective, occurs at the most rapid rate, is the most stable, when a student feels supported, is safe and it’s okay to make mistakes. They have a relationship. You’re never going to have a relationship with ChatGPT. You’re never going to be friends with it. It’s not going know how they did at the football game last night. So, I believe that piece is always going to be human to human.’

As for the future of assessment, Dr McClellan says the arrival of ChatGPT will cause us to rethink assessment.

‘And that happens in cycles all the time. Like when things were first available on computers, first it was “take paper, make on screen” – it was just a direct translation. But you could make it adaptive, you could make it select items, you could do all these other things.’

Chat is just one function of the GPT 3 corpus, she says, adding that there will be thousands of others very quickly. In fact, an assessment-tuned version is already being built.

At the moment, these tools can be thought of as a collaborator. ‘You can work with it. It supports the work, it makes the work more efficient, it takes some of the drudgery out, but it still requires your expertise to refine and target what we're trying to do. And that's probably not going to change anytime in the immediate future. So, again, it’s a tool. We will find ways to use it. We've already started with one, undoubtedly there will be others.

‘We’re going to adapt to this. We’re going to change with it too and it will change as we change. Tools always do. They will move forward as we move forward.’

Addressing equity in education

Given its open access, does Chat GPT have the potential to help address issues of equity in the classroom? According to Dr Pasi Sahlberg, a professor of educational leadership at the Melbourne Graduate School of Education at the University of Melbourne, it’s not about simply having access to ChatGPT, but rather it’s about understanding what this technology is capable of.

‘I do not believe that just by all of a sudden having ChatGPT or any other similar device that education would become more equitable, even for some people,’ he tells Teacher. ‘It is more about what extent our students would have adults around them or other people around them who will help them to understand what it is.’

In fact, Dr Sahlberg says that ChatGPT has the potential to make education more unequal, as happened with smartphones.

‘Our own research here shows how the smartphones and digital technologies have affected young people. It’s a highly unequal situation between those 2 extreme groups of children … and I think the same thing is going to happen with the ChatGPT. That’s why the role of the schools is such an important thing in making sure that everybody, all the students understand what it is,’ Dr Sahlberg says.

‘ChatGPT is not like a search engine, it’s not like Google where you can search things. It operates in a completely different logic and if the young people don’t understand how this thing actually operates, where these essays and responses come from, they’re going to be in trouble.’

Dr Sahlberg says that educators have promised to prepare young people to live and prosper in the world, and that world is increasingly run by technologies.

‘This means, to me, that rather than ban these technologies in our schools, we should think much harder how to teach our young to understand what these technologies are, what we can do with them, and how to live and work safely with them.

‘Removing these technologies from the lives of schools by external bans won’t make us keep that promise and often makes things worse, especially for those with various disadvantages.'

Creating school policies for ChatGPT usage

While bans are in place across some schools and jurisdictions, it’s unlikely these bans will stay in place permanently.

‘It was more a pause to figure out what this is. In some ways you can understand why it was initiated so and it'll be reversed eventually. All of them will be, because you can't ban a technology like that very readily,’ Professor George Siemens from the University of South Australia, tells Teacher.

The academic is the Director of the Centre for Change and Complexity in Learning, a multidisciplinary centre covering a range of areas from healthcare to computer science education, human and artificial cognition, and other related areas.

When looking to the near future, to a time where schools will be grappling with using ChatGPT and other generative AI, Professor Siemens says there are a few key things to keep in mind when beginning to develop schoolwide policies around the usage of these tools.

The first is for leadership to look at policy development from an adaptive stance, and to be flexible and responsive. The second consideration is multi-stakeholder engagement – in other words, involving parents, students and teachers. Finally, Professor Siemens says the best policy is one that takes a no harm stance.

Creating and reviewing policies will become increasingly important as new tools are right around the corner. ‘There’s actually a lot of resources being developed ... there’s over 300 companies now offering products and services on GPT-3,’ he shares. ‘They fine-tuned the model itself – so, for essays or for writing social media campaigns, they’ll be optimised for that kind of activity.

‘So that means there’s a lot more technology coming … It’s going to be a never-ending deluge of new tools and resources.’

Stay tuned: Tomorrow we will be publishing a podcast about ChatGPT that explores the themes of assessment, equity and policy in further detail. We also pose some questions to ChatGPT itself.

Have you tried using ChatGPT in your planning or in the classroom yet? How has it impacted the way you deliver your lessons and administer assessments? What are some of your key concerns about the platform?

Dr Pasi Sahlberg says it’s important to teach our young to understand what these technologies are, what we can do with them, and how to live and work safely with them. How will you be increasing your students’ digital literacy when it comes to ChatGPT? What support do you need to deliver these lessons?