Decoupling summative and formative assessment

The following post is my seminar notes from a session I ran with Christine Counsell on why ‘summative’ and ‘formative’ assessment need to be decoupled, particularly in terms of not principally using summative marking criteria to inform teaching. I am particularly grateful to the history teaching community (particularly the so-called ‘History Pizza Assessment Group’ in Cambridge) and Daisy Christodoulou for helping me clear up my thinking on this matter. 

The current prevailing model in formative assessment is to give pupils a task (an essay, an exam question, a piece of music to play) and then to judge their competence at that task using a series of levels, often based on either task-specific or generic descriptions of competence. Having ascertained how well a pupil performed on that task, we then identify what was absent from the performance: what should have been done that was not, or what could have been done better? The feedback we offer to pupils is then based on this analysis of deficit. Feedback is then framed in terms of ‘reaching the next level’: comments appear to take the form “in order to do this task better you needed to…” Let’s call this model of feedback ‘coaching’.

And, in some ways, this is fine. It is a particularly effective form of feedback when the person you are coaching already has the wider domain of knowledge and skills they need in order to make sense of the meaning of feedback.

The ‘coaching’ model of feedback is however less useful when the person you are teaching does not know enough to make sense of the feedback. To the contrary, the feedback you offer might actually be more confusing than helpful.

In order to help a pupil in this situation, you need to identify where the gaps are in the knowledge base of the pupil: your assessment is less one of “how well was the task performed?” but rather one of “what are the possible causes of this task being performed poorly?”

But herein lies the problem. The knowledge of your pupils cannot directly be seen: it can be inferred only from their performance in a task. But, paradoxically, the task you want the pupils to perform well in (the essay, the exam, the piece of music) might not be the most suitable task you need for identifying the causes of failure in that task. This means that (a) feedback you give on the performance (‘coaching’) might not be understood by the pupil and (b) you are not diagnosing what it is you need to teach or re-teach in order to fill the gap in the pupil’s knowledge base.

By way of example, a doctor might see that a marathon runner has collapsed half way through a marathon. But simply noticing the failure does not explain why the runner has failed. It would certainly be pointless at this stage to begin giving feedback to the runner on how to run better. Instead, the doctor is likely to carry out some tests: lung capacity, blood-sugar levels, tests for an infection, and so on. These tests don’t look anything like the final performance. What the doctor gets the runner to do (e.g. modifying diet before the run) also does not immediately look like the final performance. But this is precisely the kind of help the runner needs to do better next time.

So what might a diagnostic approach, rather than a coaching approach, to formative assessment look like? In short, it might involve the following process:

  • what are the common causes of weak performance in the subject I am teaching?
  • what tests can diagnose those causes most accurately?

All of this might be helpful when working with a class who are close to completing the final performance – e.g. a class of Year 11 pupils with a looming exam. But what about younger children? It would be even more pointless to start coaching younger children on exam performance: their knowledge base is even smaller, and therefore they are likely to be even more confused by the feedback. It is here that we need our curriculum to help us out. A curriculum sets out what pupils ought to know and, although a curriculum is not written simply to prepare children for tests, it nevertheless is the case that pupils who have learnt the curriculum well should be able to perform well in tests based on that curriculum. For younger children, the crucial question then is simply

  • have the children learnt what is on the curriculum?

Here again the tools you need have to be fit for purpose. Some tools are good at determining whether factual knowledge has been learnt. Some tools are good at spotting misconceptions. Some tools are good at seeing whether knowledge of one thing has been sufficiently well connected to knowledge of another.

So what are the key take-away messages here?

First, there is something to be gained by coaching pupils on performance in a task, but this is most likely to be useful when they already have a good knowledge of what is being learnt. Feedback given that is based on assessment criteria therefore has a place, but its place is limited, and it can be counter-productive.

Secondly, diagnostic formative assessment is instead a matter of identifying what the causes of poor performance are, and then testing to see which of those causes is responsible for poor performance. These kinds of assessment will not necessarily look like the final performance: indeed, they might well look very different.

Advertisements

5 Comments on Decoupling summative and formative assessment

  1. Tom Burkard // 2 November 2017 at 10:51 // Reply

    In all honesty, I think the caravan is moving on, However useful the concept of summative assessment may have been in the 1990s when teachers were so busy trying to teach a vastly over-presecriptive curriculum and hence had no time or inclination to do any kind of assessment at all, the focus on assessment is increasingly being focussed on routine tests of declarative and procedural knowledge. Largely, this is due to changes in SATs and GCSEs, where more points are to be had from simple recall of knowledge, but it also reflects the publicity surrounding Michaela and other schools which have achieved outstanding results by routinely testing knowledge.

    The advantages of routine knowledge testing are manifold–pupil behaviour and motivation is transformed, especially with the least-able pupils, Knowing that they are going to be tested, they are far more attentive; even more important, they are presented with well-defined and achievable goals. I think Heather Fearn put it beautifully:

    “…the progress of my children has stunned even me. How is it they missed out on SO much work on understanding while accelerating far ahead of their peers? . . .I’ve realised the reason I’ve never had to invest significant time in exercises to build understanding. It is because when my children are given a new sort of problem they can already calculate the separate parts of that problem automatically. All their working memory is focused on the only novel element of a procedure and so it is very quickly understood. Understanding is just not a biggy. Identify the knowledge necessary to calculate the component parts of a problem and get fluency in those and generally activities for understanding become a (crucial but) small part the maths diet.”

  2. I have been teaching for twenty odd years now and your description mirrors the approach I was taught to adopt and have indeed adopted.

    I use Bloom’s/Anderson and krathwol’s taxonomies and would not think of assessing higher order skills without ensuring first that students had sufficient knowledge. Knowledge forms the basis of everything I teach and I assess knowledge and understanding typically 2-5 times a week for students who I see for 5 lessons in a week.

    I believe in 99% of cases there is no difference between formative and summative data gathering, it is smply the interpretation and use of the data which differs.

    What does surprise me however is that so many teachers assume that teaching and assessment transfers across subject disciplines.

    I should also say that while I would typically spend 60-70% of my class time directly interacting with students, explaining and modelling I would be regarded by many as a “progressive troll”.

    Some people seem to inhabit worlds that are very different to mine.

    The formative vs summative issue is one of the most important issues in education for me.

    Nice post giving cause for reflection.

  3. With all the different subjects taught in history, with all the unique knowledge and concepts, it is, in my opinion, necessary to define as precise as possible, what it means to become better in history. The next thing to do is tell children what/how they have to perform and the end of a module to show that they have reached the attainment targets. This suggests that the attainment targets and succescriteria are told to the students at the beginning of each module (see the Backward Design Principle)..

  4. Great summary of the arguments – I particularly like your extension of Daisy’s marathon metaphor: the image of sending a collapsed marathon runner for another run (or telling them to shave five minutes of their time) is a vivid way to convey this idea.

    My one suggestion would be to consider using a word other than ‘coaching’ for the kind of feedback you’re describing, simply because this risks drawing you into a separate distinction made in the literature on feedback: whether feedback should be directive (i.e. tell students what to do) or facilitative (i.e. offer hints). What if you called it ‘assessment-focused feedback’?

You can offer your thoughts here:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: