Evidence of learning
Educational technology produces an extraordinary amount of observable activity. Clicks, completion, time on task, submissions, views, attempts, logins and engagement are easy to count. Learning is harder.
Published September 2026
Why it matters
The first question is what the learner did. The important one is what the learner can now understand or do that they could not reliably understand or do before.
Most of what a platform records answers the first question well and the second not at all. The data is doing its job. The failure lies in not noticing which question it was collected to answer.
The cost shows up in decisions. What gets counted gets managed. A dashboard that reports completion invites everyone to raise completion, and completion is easy to raise without anyone learning more.
What I have come to believe
Evidence of learning is evidence of change. A single snapshot tells me where a learner is. It does not tell me how far they moved or what moved them. To claim that an experience taught something, I need to know what the learner could do before, what they can do after and ideally whether the gain lasted.
Performance during learning is a poor guide to learning. Conditions that make practice feel smooth, such as massed repetition and hints on demand, often produce strong scores on the day and weak retention a week later. Conditions that make practice feel harder can do the reverse. So I distrust any measure taken while the support is still in place.
The most useful evidence is usually a task that asks the learner to use the idea somewhere new. Recall shows that something was stored. Transfer shows it was understood well enough to travel.
Activity data is still worth having. It shows who turned up, where people stopped and what they skipped. It misleads only when it is reported under the name of learning.
How I approach it
I start by writing down the change we are hoping for, as something a learner could do. If that sentence cannot be written, no amount of data will answer the question.
Then I look for the smallest honest measure of that change: a question asked before and after, a check a few weeks later, a task in an unfamiliar setting. Where a platform offers a convenient proxy, I ask what links it to the thing it stands for before I let it stand in.
In short
The position in four claims
The argument above, cut down to what travels.
Activity is not learning
Observable participation may be necessary for learning but does not demonstrate that learning occurred.
Performance is not always learning
Strong immediate performance can coexist with weak retention or transfer.
State is not change
Knowing what a learner can do now is different from knowing what changed as a result of the learning experience.
Proxies need validation
Convenient measures should not automatically become the construct.
In practice
The questions I ask
Before designing anything in this domain, and again when something is not working.
- What should a learner be able to do afterwards that they could not reliably do before?
- What could they do at the start, and how do we know?
- Is this measure taken while the support is still in place, or after it has been removed?
- Does the task ask for recall, or for the idea used somewhere new?
- What links this proxy to the thing it claims to stand for?
- How long after the learning are we measuring?
Unsettled
What I would still argue about
Open in my own head. If you have an answer, I would like to hear it.
- What counts as convincing evidence that learning has changed?
- Which behaviours are meaningful signals, and which are merely convenient to measure?
- How long after the learning a measure has to wait before it says anything about retention rather than rehearsal.
Where next
Follow this idea
Evidence
Assessment systems: validity, reliability and high stakes (opens in a new tab)Guide
Operationalise it
Learning analytics (opens in a new tab)Guide
Related argument
A myth is a finding that travelledNote
Challenge it
Engagement is evidence of learningUnconvinced by
Follow the concept
The usefulness of an educational metric decreases when the metric becomes a substitute for the construct it was meant to approximate.Working proposition
To take away
Count activity by all means. Just stop calling it learning.
The next time a dashboard is put in front of you, ask what the learner can now do that they could not do before, and how long after the learning that was measured.