The Research Files Episode 76: Embedding playful assessment into learning

Thanks for downloading this episode of The Research Files from Teacher. I’m Dominique Russell.

At Teacher, we’ve shared content on the impact of playful learning and game-based learning on student outcomes, and what this might look like in the classroom. But how can we extend these same principles to assessment, and why is important to think about assessment in this way?

In this episode, I’m joined by two international guests from the United States: Louisa Rosenheck, Director of Pedagogy at the game-based learning platform, Kahoot!, and YJ Kim, Assistant Professor of Design, Creative, and Informal Education in the Department of Curriculum and Instruction at the University of Wisconsin-Madison.

Louisa and YJ are presenting at ACER’s Research Conference this year. The online conference kicks off on the 22^nd of August, and at the time of publishing this episode, you still have time to register, so I’ll leave a link to the conference on our website, teachermagazine.com, in case you’re interested in finding out more.

In our conversation, you’ll get a taste of their conference presentation, and we’ll also discuss what playful assessment looks like, how educators can embed it in the classroom, and the impact of this type of assessment on student outcomes.

Before we jump into this episode, it would be great if you could take just a few moments to give a rating of our podcast if you’re listening to this audio on Apple podcasts or Spotify. If you’re listening on the Spotify app, just click on the three dots, then ‘rate show’, and if you’re on the Apple podcasts app you’ll find the rating section by scrolling to the bottom of our podcast channel page. On Apple podcasts, you’re also able to leave a short review for us. Leaving us a rating or review helps more people like you to find our podcast, and is a really big support for our team.

Thanks for taking the time to support the work we’re doing. Let’s dive into this episode.

Dominique Russell: Louisa and YJ, thank you for joining us for this episode of The Research Files. It's great to have you on. You'll be presenting at Research Conference 2022 on the topic of playful assessment. And it's something that we haven't really looked so much at Teacher before. So I'm keen to get a better understanding in this episode. We've spoken about playful learning quite a bit, but not playful assessment. So YJ, can you describe what you mean by playful assessment for us?

YJ Kim: Sure. Playful assessment is an application of a lot of things we already know about playful learning and how we can apply the design principles of playful learning in the context of assessment to really increase students’ agency and engagement and enjoyment in the context of evidence generation.

And also helping them to understand that assessment doesn't need to be really scary thing. And often I think students don't even realise that they are generating evidence, but by engaging in these playful activities that are well-designed, are aligned with the constructs that are meaningful and educationally valuable, we are hoping that students can see that. Like, ‘oh, assessment is a good thing’. And also for teachers, that assessment doesn't need to come as this particular format that we're familiar with, but it can feel like an interesting and playful thing that students are doing.

So it's really the application of a lot of playful learning design principles in the context of assessment, but really building up on the innovative practices around more performance-based kind of assessment, where assessment is performance-based, ongoing and embedded in the context of learning.

Louisa Rosenheck: And I would just add that, one thing to keep in mind for why playful assessment matters, is because what we know [is] playful learning is so good for so many different kinds of outcomes, different kinds of skill building, right?

It's not the same content knowledge or procedural skills. It's future-ready skills and the 4 or 6 Cs and social-emotional learning and lifelong learning skills and all of these important things. And if you're not doing assessment in a way that's aligned, then you're not really measuring those things that you care about and those amazing things that you're getting out of your playful learning.

So it's really important for the learning and the assessment to not only be aligned, but really be part of the same experience.

DR: And so before we get into speaking in much more detail about the concept of playful assessment, I thought it would be good to take a little bit of a step back and hear about the work that you've both done in this field so far. So Louisa, you've spent over a decade researching digital games and creative learning. Can you tell me a little bit about this research?

LR: Yeah. So I worked at MIT [Massachusetts Institute of Technology] in a couple of different labs – the education arcade and the playful journey lab – for the past decade or so. And there, we were really thinking about how can technology and playful learning and games and creative learning transform learning so that it's not just doing the same kinds of learning, but digitally? It's really letting students explore and conduct inquiry and iterate and experiment and try on new identities.

And so in that work, we've designed a number of different games. Some that were very kind of open-world MMO [massively multiplayer online] style games, some that are more like puzzle games as well as alternate reality games and AR [augmented reality] games. So lots of things coming out of the lab that have been really interesting. Anyone who's interested in that can check out the book called Resonant Games that a bunch of us from the lab helped write a few years back.

And I think some of the really important things that that have come out of that research are the design principles of playful learning and game-based learning. And that it's not just good for content knowledge.

It's this science practices and math practices and collaboration and communication. And, you know, using evidence to make a claim and all these kinds of things you can do in a game, but it doesn't just happen in the game. This is something that I always like to emphasise. It happens in the teaching around the game and the reflection on the game, talking to other students about your strategies and how you persisted and how you figured stuff out and understanding that ‘oh, somebody else actually approached this in a whole different way’. So all of this kind of metacognition and reflection is really important to get the most learning out of game-based assessment.

DR: And YJ, your work centres on innovative assessment and working closely with educators to think more innovatively about assessment. Can you tell me a little bit about this work and maybe give an example of how you've worked with a classroom teacher, or a group of teachers, recently to think innovatively about assessment?

YJK: Sure. So, unlike Louisa, who's coming from a more playful running background, I'm coming from more assessment, design and development background.

So for the past 12, not 12, around 15 years, as well, I've been applying evidence-centred design, an assessment design framework established by Bob Mislevy and his colleagues at ETS in the United States in the context of more social-cultural learning context.

So a very specific example that I've been working on a lot is game-based assessment. So similarly, kind of, how can we think about affordances of a game as a learning context, but not just learning, but how can we use it as an assessment tool?

But for my work, so I started as a very kind of kind of kosher-style application of evidence-design in a game environment, then kind of eventually moved on to thinking about inviting some of these principles of learning and principles of play in the game and taking that outside of the game environment and helping teachers to think about their assessment practices by applying those.

So, one specific example that I have with my recent work, I've been working with – currently I'm in Wisconsin, the state of Wisconsin in the United States – I’ve been working with two Computer Science teachers who are thinking about ways to teach AI [artificial intelligence] literacy concepts that are not really coding-based. So these are teachers like my students, they're not really coders, they don't see themselves as coders, but can they still learn about AI literacy and some fundamental concepts of AI?

So we've been creating these playful activities that are essentially game-like, but instead of like digital games, the students are kind of role play as like code and AI agents and label and data and they are trying to understand, by kind of playing these roles in this game, they get to understand, ‘OK, this is how the machine learning process works’. But also because they are playing this game, it also generates evidence about how they’re understanding these fundamental concepts. So we've been kind of working with teachers to create those kinds of assessments.

Another example of us working closely with teachers is with the Shadowspect game, which is the game we developed when both of us were at MIT. We did a year-long code-design process with teachers to be able to think about how, metrics or assessment machinery development process, how we can engage teachers as active designers, as active code designers along that process? And how can that help teachers to understand different learning outcomes that games can support better? And how can we basically make the process more democratic and come up with assessment machineries that actually make sense to teachers better? So we've been looking at that as another kind of angle thinking about affordances of playful assessment as well.

DR: And so YJ, would you say that playful assessment is a relatively new field of research or how long have we really been talking about this idea of playful assessment?

YJK: I don't know if we can call this a field yet, but I want to point out that this idea of games as a learning system, right? Like the game-based learning as a field emerged like early 2000s. So, from that kind of beginning people start understanding affordances of game for assessment. So I'll call that, kind of late 2000s. And that's when I started working in the context of game-based assessment and I'd say playful assessment, as a genre within that kind of field, I'll say it emerged around 2015/16.

I don't want to call that as a field yet, because I think in terms of theory and in terms of how playful assessment actually works in classrooms, I think we have very limited understanding yet. And as I pointed out earlier (or maybe I haven't) game is a very structured form of play and we're really trying to, in a way, move forward or move beyond game to apply those playful learning principles in the context of assessment.

But the examples, I think, in the field are still kind of limited. And I think how researchers and practitioners have been thinking about playful assessment also kind of looks different from context to context. So I think there's a lot that we still need to figure out. A, what is playful assessment and B, application and implementation of that in classrooms, how that benefits learners and teachers? And C, how it offers beyond what game-based assessment has offered? And I think that's something that we're still trying to figure out.

DR: And so as I mentioned in the beginning of the episode, you're both giving a presentation at Research Conference 2022 and that presentation is titled Playful assessment: Don't stop the fun. So, Louisa, can you give our listeners a bit of a taste of what you'll be sharing with delegates at your presentation?

LR: Yeah, we're excited to be part of the conference. So the title of the session, Playful assessment: Don't stop the fun – the reason we called it that is that when you're doing game-based learning or you're engaging in playful learning or creative learning, it's fun. It's engaging. Students get in the flow. And unfortunately a lot of times the way assessment works is you stop that fun and that engagement and then you bring out the assessment, which is a completely different format, often a more conventional traditional kind of experience and that's not what we want.

We want to continue with the same kind of learning, the same kind of experience, and even have them integrated together so that the assessment is happening, you're recognising and celebrating your learning while you're doing the project or doing the learning experience.

So what we're going to share in the session is we're going start with some background on the theory and what we know about learning science principles around playful learning and also about assessment and some of the qualities of a good assessment. And that will include things like psychometric qualities (validity and reliability), but also some of the qualities that are so important for good learning and deep learning. Things like being authentic and timely and social and student-centred and co-created and things like that. So we'll provide some background on that.

And then we'll give some examples of what playful assessment can look like. Some examples from our work, from maker education as well as game-based assessments.

And then it is going to be an interactive session. So we're going to have some discussion or some kind of active learning activities where we we'll share some playful assessment principles and we'll encourage participants to think about those in their context, and how can they gather evidence and help their students gather evidence, of their learning from, whether it's games or play or creative activities?

So we hope that people will come away with a new perspective and some action items that they can take to their own practice.

DR: And so as you've just touched on there as a little bit, Louisa, you've both said that a key part of playful assessment is really embedding it into playful learning experiences. So YJ, for classroom teachers who are listening to this podcast episode, can you expand a little bit on why it's important to really aim to embed it rather than simply swapping out a more traditional assessment at the end of a teaching unit with a game-based assessment? Why is the embedding element really crucial?

YJK: So when we started doing this work, we talked to a lot of kind of playful, creative teachers who’ve been doing this kind of assessment in classrooms. And when we discuss this idea, I remember one teacher saying, ‘Oh, you call that playful assessment? I've been doing this all these years.’

And the reason why I'm bringing up that teacher’s anecdote is because I feel like that good teachers, thoughtful teachers, always been already doing this. They blur the line between learning and assessment. They create a engaging, authentic, playful context that is easy for all students, not just the high-performing students can participate and feel safe.

And also because it helps students to think about, right, evidence and things like that in relation to these activities unlike these task scores, where they see just a number, right, like there's a concrete evidence that they experienced, that they can reflect on and have a discourse on with teachers.

And I also think that it really helps teachers to kind of change the focus of assessment from grading. I think often a lot of education unfortunately, especially in United States, when they think about assessment, they think it's a grading, but those are two different things, right? So playful assessment can help teachers to think about ‘what are really the outcomes that I care about as a teacher?’. And then ‘how can I foreground that in this authentic, playful experience?’ and then they can think about, ‘OK, how can I make sense with what I observed with this?’.

And I think that those are the kind of affordances of playful assessment and that the part of kind of embedding that assessment piece in the playful activity is the key piece of this.

LR: Yeah. And just to emphasise a little bit, one part of that. You know when students are doing, for example project-based learning or maker projects, it would not feel authentic to do a whole big project and show these skills and then have to do another assessment where you have to show those skills again, but in a more contrived setting, right?

So in the real world you don't stop and then take a test, your work is demonstrating your skills, and so that's what we want with playful assessment and with student-centred assessment as well.

DR: And so just finally, we’ve touched on it a little bit throughout our conversation, but I thought it would be good YJ if there's anything that you wanted to add to this on what impact this type of assessment does have on student outcomes and also how can it improve teacher practice?

YJK: I think one immediate – we don't have any evidence that playful assessment, for example, like, increased their performance on standardised tests, you know, two points or whatever, we don't have that kind of evidence. I don't know if you particularly care about that kind of evidence either – but the one core element, or the difference that we see with playful assessment is students’ reflection and student metacognition and understanding of the learning process. And that’s better than the more traditional assessment.

So that piece I think is really crucial and that's something that we have more evidence than, you know, increasing on learning. But I will also say that in terms of the quality of assessment, what we believe is that because playful assessment really does reduce anxiety associated with the task and things like that. And we are hoping that playful assessment can actually provide a better understanding of what students know and can do than more traditional forms of assessment.

LR: ... One other thing I'd like to say just in closing is we've talked about some of the research that we've done, and this is in the context of research work, but these ideas about playful learning and playful assessment, they really need to go way beyond the research field, right? We need research to show what some best practices and identify design principles, but my perspective is always then, how do we take those design principles and bring them to life?

And sometimes that's in teacher’s classrooms, in teacher practices, and also sometimes that's in EdTech products and curriculum design. So since leaving MIT, I've actually made the switch into industry, and I now work leading the learning design at Kahoot!.

And that is just one example of how I think it's so important to take some of these ideas about student-centred learning and playful learning and expand them out to some of the products and tools and platforms that are reaching so many teachers and students to get these ideas out there and help people both try new things and see different ways that they can use tools and influence the EdTech sphere. And also, like YJ said, to validate what a lot of teachers are already doing, and say ‘yes, this is good, and this is actually known that it's working’. It's helping students and it's helping transform learning to prepare students for the future.

That’s all for this episode. Thanks for listening. If you found this topic interesting, you might like to listen back to episode 51 in our Research Files series where we looked at game-based learning. Just search for ‘The Research Files Episode 51’ in your podcast feed to access the episode.

Don’t forget to subscribe to our channel so you never miss an episode. We’re so excited to be launching a brand new series next week, so keep your eyes peeled.

Related reading

Klopfer, E., Haas, J., Osterweil, S. & Rosenheck, L. (2018). Resonant Games: Design Principles for Learning Games that Connect Hearts, Minds, and the Everyday. MIT Press. https://www.resonant.games/