Student writing – comparative judgement and feedback

The process of Comparative Judgement (CJ) for schools in the first ever Australian project resulted in student data that provided a writing age and an assigned NAPLAN band. The question for the Year 3/4 Team at Templestowe Heights Primary School (THPS) was: What would we now do with this information?

We could see clearly, when comparing students to each other, the differences between those who understood how to craft a sequential and meaningful text, and those who wrote with stream of consciousness using sentence fragments and basic vocabulary.

As teachers, we know that asking students to ‘go and make it better’ or ‘use more descriptive words’ is unhelpful feedback when students don’t know what they don’t know.

CJ provided the opportunity to analyse and review our explicit preparation of the way we were developing our students’ fluency to accurately construct sentences. Equipping them for precision with vocabulary, parts of speech and complex use of language, seems a much easier planning sequence when the gaps are made more visible, such as with the CJ data.

Writing and working memory

Cognitive Load Theory and the limited storage available in working memory has been well researched and written about. According to Nat Wexler (2021), co-writer of The Writing Revolution, the writing process places a heavy burden on working memory, where several mechanisms including punctuation, grammar, paragraphing and use of dialogue are a complex juggling act. The gaps in students being able to integrate all these processes in a fluent and accurate text creation, were obvious in our CJ data.

Challenging students to create something new may be just beyond their reach until writing mechanics become part of their long-term memory retrieval.

This was important growth for us at THPS and it is well-supported by research in how novices become experts. As Dan Willingham (2010) states, in Why Don’t Students Like School?, posing tasks that require the creation of something new (like writing about less familiar topics or crafting sentences in a new way) is a task beyond the students who need more background knowledge and time to have in-depth practise and retrieval.

Sequencing writing lessons based on the data

How does writing fluency happen, then? THPS began to modify our learning sequences to address sentence level work and spaced practise of specific skills. We are using techniques from The Writing Revolution to embed sentence mechanics, including the use of contractions, how to develop more complex sentences and steps in planning out a text.

The delivery of feedback to our students became an integral part of how we structured our lessons. Since teachers found the CJ process of comparing samples for levels of quality so useful, could this also be a key act of influence for the students?

In our research, we discovered that the use of comparison as a technique (Bouwer et al., 2018) had shown improved performances for students in that they develop a schema (Sherrington, 2020) assisting them to distinguish between present and absent features in different quality writing pieces.

Contrasting quality exemplars with their own, and then allowing students to revise, edit and retrieve this process again and again across multiple contexts, has allowed us to space out key areas of learning in sentence work.

Whole class feedback

Instead of individual conferences with multiple sets of written feedback, we had a better sense of what was lacking for the whole cohort and we addressed these accordingly. An example of one technique we used can be seen in Figure 1.

[Figure 1: Lesson slide using student examples for feedback and practise]

A quick survey was also conducted to establish how students found the whole class feedback sessions targeting the use of comparisons, misconceptions and exemplars.

In learning through examples (Christodoulou, 2017) did it help:

• what they need to improve in their planning
• techniques used to construct a paragraph

Over 70 per cent of students found that writing comparisons demonstrated what they should add to improve; 60 per cent of students stated that the process helped them reflect on their mistakes, editing and the readability of their writing and they were more likely to do it differently the next time.

We were pleased to see students interact with this process and we believe that over time, techniques that utilise whole class feedback, common themes and misconceptions, and multiple opportunities to practice sequencing small steps, (Sherrington, 2021) will equip the students as writers more effectively than individual conferencing.

As we continue to utilise this data, we are collating a bank of feedback practices to provide the benchmark for students creating and evaluating their own writing. Using the data to intervene and change the outcome for this cohort of students is the exciting and open space that we look forward to comparing in the second CJ process in September.

CJ provides a lens for feedback across every individual student, but also reveals the collective misconceptions, allowing teachers to target point of need.

Providing feedback with comparative judgement – the UK context

With traditional marking, teachers often write lengthy comments at the bottom of a piece of work. These comments give the feeling of personalisation, and they are certainly very time-consuming, but how effective are they?

One major problem with this kind of feedback is that it is all too easy for it to become ‘true but useless’ – stating a problem with the work but not providing a helpful way forward for the student.

In his book Embedded Formative Assessment, the assessment expert Dylan Wiliam gives an example of this:

I remember talking to a middle school student who was looking at the feedback his teacher had given him on a science assignment. The teacher had written, ‘You need to be more systematic in planning your scientific inquiries.’ I asked the student what that meant to him, and he said, ‘I don’t know. If I knew how to be more systematic, I would have been more systematic the first time.’ This kind of feedback is accurate – it is describing what needs to happen – but it is not helpful because the learner does not know how to use the feedback to improve. It is rather like telling an unsuccessful comedian to be funnier – accurate, but not particularly helpful, advice (Wiliam, 2018, p.140).

To be effective, feedback has to be helpful, not just true. Ideally, it should provide students with a clear, concrete next step. Traditional written comments are quite bad at doing this.

So, what can we do instead? One technique that has become popular in England in the past few years is whole class feedback.

Here’s how it works: a teacher will read all of their class’s work, but instead of writing comments at the bottom, they will plan their next lesson based on the common strengths and weaknesses they have seen across all of the students’ work. And when they are planning, they will focus on designing activities that provide students with a clear next step.

For example, suppose that you’ve read a set of narratives that all have some issues with inconsistent tense. The traditional way of addressing this problem would be to write a comment at the bottom of every piece of work along the lines of ‘you need to improve your tense consistency’. This falls into the ‘true but useless’ trap identified by Wiliam.

With whole class feedback, you wouldn’t write any comments. Instead, at the start of the next lesson, you might display the following two sentences, which are typical of the tense inconsistency we see in younger students’ writing.

The spaceship buzzed around the sky and then landed. Two aliens get out and threaten the locals!

You could then ask the class to identify the error in the sentences, correct the error, and see if they can spot a similar error in their own work.

You might then give them an example of the opening sentence of a story written in the past tense, and then the same sentence in the present tense – and then ask them to write their own opening sentences in the past and present tense.

This kind of whole class feedback can be used in lots of ways, and it doesn’t depend on CJ. But lots of the schools we (at No More Marking) work with in England have chosen to integrate the two approaches. They will judge their students’ writing together as a staff group, discuss it after they have finished judging, and use that discussion to adapt their subsequent lesson plans. We provide templates and case studies to support this approach.

If your school is interested in participating in the second Assessing Australia’s Primary Writing project with nomoremarking.com and THPS, I encourage you to make contact and investigate the CJ assessment technique. On offer is also a project for secondary schools and a Writing Network with Think Forward Educators where members can explore writing practice in more depth.

References

Bouwer, R., Lesterhuis, M., Bonne, P., & De Maeyer, S. (2018). Applying criteria to examples or learning by comparison: Effects on students' evaluative judgment and performance in writing. Frontiers in Education, 3(Article 86). https://doi.org/10.3389/feduc.2018.00086

Christodoulou, D. (2017, November 5). Giving specific feedback. The No More Marking Blog. https://blog.nomoremarking.com...

Sherrington, T. (2020, October 4). Teaching for understanding: Schema-building and generative learning. teacherhead. https://teacherhead.com/2020/1...

Sherrington, T. (2021, January 17). Applying Rosenshine’s Principles to specific contexts. teacherhead. https://teacherhead.com/2021/0...

Wexler, N. (2021, January 31). The Puzzling Gap In Research On Writing Instruction. Forbes. https://www.forbes.com/sites/n...

Wiliam, D. (2018). Embedded Formative Assessment. Solution Tree Press.

Willingham, D. T. (2010). Why Don't Students Like School?: A Cognitive Scientist Answers Questions About How the Mind Works and What it Means for the Classroom. Jossey-Bass.