Gerald Ajam

Measure the change, not the state

A class scores eighty per cent at the end of a unit. That sounds like learning, and it might be. It might also be what the class would have scored on the first day. One score cannot tell those apart, and most reports only ever collect one.

Status Position

Published November 2026

All notes

In short

A score at the end shows where a learner is, so any claim that something taught them needs a measure of where they started.

The shift it turns on

The usual reading

The score as the result

A high mark at the end means the teaching worked.

The reading it moves to

The difference as the result

Only the distance from the start says what the teaching did.

01The situation

One snapshot, read as a story

Most evidence about learning products is a single snapshot. An end-of-module quiz. A mastery percentage. A mark on the final task. Each describes the learner at one moment, and each is routinely read as a story about what the product did.

A story needs a second point. Kirschner, Sweller and Clark (2006) put it bluntly: “The aim of all instruction is to alter long-term memory. If nothing has changed in long-term memory, nothing has been learned.” Change is the word that matters, and a single score cannot show change. It shows where someone is and leaves open how they got there.

Coe (2013) makes the same point from the classroom. His list of poor proxies for learning includes students giving correct answers, with the caution that some of them already knew how to do it anyway.

02The tension

Where were they at the start?

What if many claims that a product works are really claims about who used it? Learners who arrive knowing more finish with higher scores. Schools that adopt a product early differ from schools that wait. A strong class looks strong at the end of anything. Without the starting point, a high end score credits the experience with whatever the learner brought in.

So the useful unit is a difference: where the learner was, where they are now and, where it can be had, the same difference for a comparable group who did something else. Each extra point costs something. Each one also removes an explanation that the single score left standing.

Without the starting point, a high end score credits the experience with whatever the learner brought in.

03The move

Collect the second point

None of this needs a trial. It needs the habit of measuring twice.

  1. Ask before you teach

    Put two or three questions at the start that match the ones at the end in kind, with different surface details. The start-of-unit check is cheap, and it changes what the end score means. It is also a small learning event in its own right. Trying to answer before being taught can improve later learning (Richland, Kornell and Kao, 2009), which is a reason to use one, and a reminder that it is no neutral measurement.

  2. Report the movement

    Show where learners started, where they finished and how many moved. A class that went from forty to eighty and a class that went from seventy-five to eighty share an end score and tell very different stories.

  3. Look at who moved

    Averages hide the shape. Ask which learners moved, which stayed put and whether the ones who moved were the ones who most needed to. A product that lifts the strongest learners and leaves the rest where they were can still raise the average.

  4. Find a comparison where you can

    A difference shows change. It cannot show the cause, because learners also change through the teaching around the product, the homework and the weeks passing. A comparable class that did something else, even informally, shows how much change to expect anyway.

04Where it breaks

Research suggests

Change is harder to measure than it looks

Gain scores have an awkward history. Cronbach and Furby (1970) argued that raw differences between two scores are often less reliable than either score on its own. The difference carries the measurement error of both, and the more closely the two scores track each other, the more of the difference is error. Small groups make it worse.

Then there is regression to the mean. Learners chosen because they scored low at the start will tend to score higher next time even if nothing happened, because part of a low score is bad luck. A programme for the lowest scorers will appear to work even when it does nothing, unless someone checks it against a comparable group.

None of this rescues the single score. It means a difference should be read with care, and next to what the teacher saw.

05Where it lands

Compared with when?

The first change is that a claim about what a product taught now comes with a starting point. Without one, it is reported as a description of where learners ended up. Both are honest. Only one is evidence of learning.

The second is a question I put to any figure offered as an outcome: compared with when? A mastery percentage with no starting point is a state. It may be good news. It is not yet news about the product.

The third is that the start-of-unit check earns its place twice. It makes the end score mean something, and it shows the teaching what it needs to do before the teaching has done it.

The point

What transfers

The reason this page exists, rather than the story that produced it.

  1. A score shows where a learner is. Only a difference between two points can show what an experience changed.
  2. Ask before you teach. Two or three questions at the start change what every later score means.
  3. Report where learners started, where they finished and who moved. Averages hide the shape.
  4. A difference shows change. A comparison group shows how much change to expect anyway.
  5. Read gains with care. Raw differences are noisy, and learners picked for low scores will rise on their own.
Singapore