Gerald Ajam

Evidence of learning

Educational technology produces an extraordinary amount of observable activity. Clicks, completion, time on task, submissions, views, attempts, logins, and engagement are easy to count. Learning is harder.

Status Position

Published September 2026

All work

Why it matters

The important question is not simply what the learner did. It is what the learner can now understand or do that they could not reliably understand or do before.

Most of what a platform records answers the first question well and the second not at all. That is not a failure of the data. It is a failure to notice which question the data was collected to answer.

The cost shows up in decisions. What gets counted gets managed. A dashboard that reports completion invites everyone to raise completion, and completion is easy to raise without anyone learning more.

What I have come to believe

Evidence of learning is evidence of change. A single snapshot tells me where a learner is. It does not tell me how far they moved or what moved them. To claim that an experience taught something, I need to know what the learner could do before, what they can do after, and ideally whether the gain lasted.

Performance during learning is a poor guide to learning. Conditions that make practice feel smooth, such as massed repetition and hints on demand, often produce strong scores on the day and weak retention a week later. Conditions that make practice feel harder can do the reverse. So I distrust any measure taken while the support is still in place.

The most useful evidence is usually a task that asks the learner to use the idea somewhere new. Recall shows that something was stored. Transfer shows it was understood well enough to travel.

Activity data is still worth having. It shows who turned up, where people stopped and what they skipped. It misleads only when it is reported under the name of learning.

How I approach it

I start by writing down the change we are hoping for, as something a learner could do. If that sentence cannot be written, no amount of data will answer the question.

Then I look for the smallest honest measure of that change: a question asked before and after, a check a few weeks later, a task in an unfamiliar setting. Where a platform offers a convenient proxy, I ask what links it to the thing it stands for before I let it stand in.

In short

The position in four claims

The argument above, cut down to what travels.

  • Activity is not learning

    Observable participation may be necessary for learning but does not demonstrate that learning occurred.

  • Performance is not always learning

    Strong immediate performance can coexist with weak retention or transfer.

  • State is not change

    Knowing what a learner can do now is different from knowing what changed as a result of the learning experience.

  • Proxies need validation

    Convenient measures should not automatically become the construct.

In practice

The questions I ask

Before designing anything in this domain, and again when something is not working.

  1. What should a learner be able to do afterwards that they could not reliably do before?
  2. What could they do at the start, and how do we know?
  3. Is this measure taken while the support is still in place, or after it has been removed?
  4. Does the task ask for recall, or for the idea used somewhere new?
  5. What links this proxy to the thing it claims to stand for?
  6. How long after the learning are we measuring?

Count activity by all means. Just stop calling it learning.

In preparation

Notes this domain still needs

Listed so the gap is visible. Each will appear in the notes when it is worth reading, not before.

  • What counts as evidence of learning?
  • Completion is not competence
  • Measure the change, not the state
  • What dashboards cannot tell you
  • Engagement is evidence of activity
  • Designing questions that reveal thinking
  • Why delayed measures matter

Unsettled

What I would still argue about

The honest list. These are open in my own head, not rhetorical.

  • What counts as convincing evidence that learning has changed?
  • Which behaviours are meaningful signals, and which are merely convenient to measure?
  • How long after the learning a measure has to wait before it says anything about retention rather than rehearsal.
Singapore