Gerald Ajam

Why delayed measures matter

The easiest moment to measure learning is the end of the lesson. It is also the moment a measure is least likely to show it. What a learner can do five minutes after practice and what they can do a month later are different quantities, and the conditions that raise the first can lower the second.

Status Position

Published November 2026

All notes

In short

Learning is what is still there after time has passed, so the measure that matters most is one taken weeks later, once forgetting has had its turn.

The shift it turns on

The usual reading

Measure at the end

Test when the teaching finishes, while it is fresh.

The reading it moves to

Measure after forgetting

Test when time has passed, and see what is left.

01The situation

Measured while it is fresh

Almost every measure in a learning product is taken during learning or just after it. The practice score. The end-of-module quiz. The mastery badge awarded when three items in a row come out right. These are convenient, because the learner is already there, and flattering, because the material has not yet had time to fade.

Teachers know the pattern from the other side. A class that handled a topic easily in March cannot do it in May. The March result was real. It was taken at the point where performance is highest and says least about what will last.

02The tension

Research suggests

What if the timing decides the verdict?

What if the timing of a measure can reverse what it shows? Roediger and Karpicke (2006) had students either reread a passage or take a recall test on it. Five minutes later, the rereaders did better. Two days later, and a week later, the testers did. Measured early, the weaker strategy won. Measured late, the order flipped.

Soderstrom and Bjork (2015) describe this as the gap between performance and learning. Performance is what can be seen during and just after practice. Learning is the lasting change, and it can only be inferred later. Usually the two move together. When they part, the immediate measure points the wrong way, and so do learners’ own judgements. Kornell and Bjork (2008) found learners rated massed practice as more effective than spaced practice, even after their own test results had shown the opposite.

Judgements improve with time as well. Nelson and Dunlosky (1991) found that judgements of learning made a short while after study predicted later recall far better than judgements made straight away. Even a learner’s own sense of what they know is more accurate after a pause.

Measured early, the weaker strategy won. Measured late, the order flipped.

03The move

Put time into the measure

  1. Add a look back to every check

    Every quiz carries one or two items from a few weeks earlier, unannounced and unmarked. It costs a minute. Over a term it builds a record of what lasted, alongside the record of what was practised.

  2. Keep immediate and delayed results apart

    Report the end-of-lesson score and the later score in separate columns. A product whose immediate scores rise while its delayed scores stay flat is improving performance, and the report should say so in those words.

  3. Wait for the delay before judging a design

    A change that makes practice smoother will usually raise the immediate numbers, because smooth practice is what immediate numbers reward. Hold the verdict until the delayed result is in.

  4. Let the delayed check teach

    A delayed check is retrieval practice as well as a measure. Recalling something strengthens later recall, which is what the tested group in Roediger and Karpicke’s study were doing. So the delayed item does two jobs: it shows what lasted, and it helps more of it last.

04Where it breaks

Open question

How long is long enough?

There is no agreed delay. Studies use days, weeks and sometimes months, and the right gap depends on how long the learning is meant to last. A delay long enough to show retention is also long enough for other things to happen: later teaching, revision, a tutor or a holiday. The delayed score measures all of it together, so it is better at showing that something lasted than at saying why.

There is a practical cost too. Delayed results arrive after the decision they would have informed. A pilot ends, a report falls due and the product moves on. The signals that arrive in time to act on are the ones furthest from learning, and I do not think that has a clean answer. A small delayed check that arrives late still changes what the next decision rests on.

05Where it lands

A second date on every result

The first change is a second date. Every result carries two: when the learning happened and when the measure was taken. The gap between them decides what the result can claim. Minutes after practice, it is performance. Weeks later, it starts to be learning.

The second is patience with designs that feel worse. A practice routine that makes learners struggle, and scores lower on the day, deserves a delayed check before anyone drops it. The figures that would have condemned it are the early ones.

The third is one question for any claim of impact: how long after the learning was this measured? Most claims cannot answer it. That tells you what kind of claim it is.

The point

What transfers

The reason this page exists, rather than the story that produced it.

  1. Learning is what is still there after time has passed. A measure taken minutes after practice records performance.
  2. The timing of a measure can reverse its verdict. Rereading beats a recall test at five minutes and loses to it at a week.
  3. Add one or two unannounced items from a few weeks back to every check, and report them in their own column.
  4. Hold judgement on any design that makes practice smoother until the delayed results are in.
  5. Ask of every claim of impact how long after the learning it was measured.
Singapore