Gerald Ajam

What counts as evidence of learning?

Every platform can say what a learner did. Very few can say what a learner can now do. Most claims about learning go wrong in the gap between those two sentences, and four questions asked of any record show which side of the gap it sits on.

Status Position

Published November 2026

All notes

In short

A record counts as evidence of learning when it supports one sentence about what the learner can now do, on their own, later and somewhere new, that they could not do before.

The shift it turns on

The usual reading

Evidence as data

Whatever the platform recorded, counted and charted.

The reading it moves to

Evidence as a claim

A record that supports one sentence about what changed in the learner.

01The situation

The data arrives before the question

A learning platform records everything it can: logins, clicks, views, attempts, submissions and minutes. None of it was designed to show learning. It is what software produces as it runs, and it piles up long before anyone has asked it a question.

So the question usually gets asked backwards. A review opens a dashboard and asks what the numbers say about learning. The numbers were never built to say anything about learning, and the room reads them that way anyway, because they are the only numbers in the room.

I find it more useful to start from the other end. What sentence about a learner would I want to defend, and what would I need to see to defend it?

02The tension

My interpretation

What if evidence belongs to the claim?

What if evidence is a property of the claim, and the record is only raw material? Assessment researchers have argued this for decades. Messick (1989) placed validity in the inferences drawn from a score. Kane (2013) turned that into an argument to be built: state the claim, then show what supports each step from the performance to the conclusion. Validity is always validity for a particular use.

Applied to a platform, that changes the question. A completion log is excellent evidence for the claim that a learner reached the end of a module. It is very weak evidence for the claim that they can now solve the problems in it. The record is the same in both cases. The claims differ, and so does what each one needs.

Claims about learning need the most. The question this domain starts from says why: what can the learner now do that they could not reliably do before? Hidden in that question are four tests, and most records pass one of them.

A completion log is excellent evidence that a learner reached the end. It is very weak evidence that they can do what the end was for.

03The move

Four tests before a record counts

Before a record is called evidence of learning, it has to answer four questions. Most records answer one, which is fine, provided the report says which.

  1. Was it there before?

    A score at the end says where a learner is. It says nothing about how far they moved. A class that scores well after a unit may have scored well before it, and without a measure at the start the end score gets the credit for whatever learners brought in. Measure the start, and the claim gets much stronger.

  2. Was it the learner or the support?

    Much of what a platform records happens with help still on the screen: the hint, the worked example, the retry that waits until the answer is right. Performance with the support in place measures the learner and the support together. To claim something about the learner, take the support away and look again.

  3. Did it last?

    Soderstrom and Bjork (2015) separate performance, what can be seen during practice, from learning, the change that lasts. The two can come apart in both directions. Practice that feels smooth can leave little behind, and practice that feels hard can leave a great deal. A measure taken at the end of the lesson captures performance. Learning shows up later, if anyone checks.

  4. Does it travel?

    Recall shows that something was stored. Using the idea on a problem the learner has not seen shows it was understood well enough to move. So the test should be a little unlike the practice. A check built from the items the learner rehearsed mostly measures the rehearsal. Kulik and Fletcher (2016) found that intelligent tutoring systems showed far larger effects on locally developed tests, matched to what the tutor taught, than on standardised tests: a mean of 0.73 of a standard deviation against 0.13.

A record that passes all four is rare and costs effort. A record that passes one is still useful, so long as it is reported under the claim it can carry.

04Where it breaks

Not every decision needs all four

Small decisions need small claims. A teacher deciding mid-lesson whether to move on needs to know who is ready for the next step now. A well-built question answers that in thirty seconds, and holding it to a test of lasting learning would be asking it to do another job.

And some of what schools care about most, such as a habit of checking a source, shows up over years and in places no task reaches. The four tests describe what convincing evidence would look like. They do not promise it can always be collected.

05Where it lands

One sentence before any data

The first change is in what gets written before a dashboard is opened: one sentence naming the claim. “Learners can set up a ratio from a word problem they have not seen, three weeks after the unit.” If that sentence cannot be written, no amount of data will supply it.

The second is in how a report is laid out. Each figure sits under the claim it can support. Completion goes under “reached the end”. A start and end score goes under “changed”. An item repeated weeks later goes under “lasted”. A figure that supports no claim about learning stays on the page, under its own name.

The third is a modest kind of honesty. Most of the time a team can afford one or two of the four tests. Saying which were met and which were not costs less than collecting everything, and is worth more than claiming it.

The point

What transfers

The reason this page exists, rather than the story that produced it.

  1. Evidence belongs to a claim. The same record can be strong evidence for one sentence and weak evidence for another.
  2. Write the claim first, as something a learner could do. If it cannot be written, no data will answer it.
  3. Four tests separate a record of activity from evidence of learning: was it there before, was it the learner or the support, did it last and does it travel.
  4. Small decisions need small claims. A question that shows who is ready now is doing its job, and should not be asked to prove more.
  5. Report every figure under the claim it can carry, and say which of the four tests it did not meet.
Singapore