Gerald Ajam

Engagement is evidence of activity

Most arguments about engagement data end in a demotion. It is not learning, so it matters less. That is half right. Engagement data is a poor measure of learning and a good measure of something else, and a team that only demotes it throws away the part it is good at.

Status Position

Published November 2026

All notes

In short

Engagement figures are weak evidence of learning and good evidence of activity, so read them for what they show well: who got in, where they stopped and what the design made hard.

The shift it turns on

The usual reading

A weak signal of learning

Discount it, and look for something better.

The reading it moves to

A strong signal of activity

Read it under its own name, for the questions it can answer.

01The situation

Read upwards, and found wanting

Engagement figures are usually read upwards. Logins, minutes, sessions and streaks are treated as early, rough evidence of learning, and then criticised for being early and rough. The criticism is fair. The habit of reading upwards is what causes the trouble.

Read that way, the figures mislead twice. They overstate learning whenever they rise for some other reason: a reminder, a reward, a deadline or a tab left open. And they hide what they are good at, because nobody asks them the questions they could answer.

02The tension

My interpretation

What is the log a record of?

What if engagement data is precise evidence about something other than learning? A log line is a fact about behaviour. This learner opened this page at this time, stayed this long and left from this screen. Facts like these are far more dependable than any inference drawn from them about understanding.

Even the behaviour is less direct than it looks. Most platforms record only the moments a learner did something, so time on task is estimated from the gaps between actions. Kovanović and colleagues (2015) note that the method is rarely described in detail, and found that the choice of method can change the results of the analyses built on it. A child’s log also records the household: a shared device, a parent who logs in, a weekly slot in the computer lab. Low activity can mean no device. High activity can mean a parent doing the tasks.

So the log is evidence of activity, shaped by access, by the timetable and by how the platform chose to count. Those are exactly the things a product team and a school most need to know, and they are rarely asked about.

Engagement data is precise evidence about something other than learning.

03The move

Ask it activity questions

  1. Rename it

    Report activity as activity. “Logged in this week” and “minutes on the platform” are accurate labels. Under those names nobody mistakes them for learning, and nobody has to defend them as learning either.

  2. Ask who got in

    Before comparing classes or schools, read activity as a map of access. Who never started? Who works only at certain hours, or only during school? A gap in the log is often a gap in devices or connection, and it falls hardest on the learners with least. That is worth knowing, and it calls for a response that has nothing to do with motivation.

  3. Ask where they stopped

    Drop-off is the most useful thing activity data shows. When many learners leave at the same screen, the screen is telling you something: an unclear instruction, a broken step, a task pitched too far ahead. That is a design finding, and it is far firmer than anything the same data says about understanding.

  4. Check the link before leaning on it

    Where activity is used as an early warning, test it against real evidence of learning in the course where it will be used. Gašević and colleagues (2016) studied nine courses and found that ignoring how each was taught could lead to over- or under-estimating the effect of platform features on students’ success. The link depends on how each course uses the platform.

04Where it breaks

Activity is honest until it is a target

Even as evidence of activity, the figures hold only while nobody is judged by them. When schools are ranked on platform minutes, minutes rise, and the log starts to record the ranking. Campbell (1979) described this for social indicators in general. It applies to activity data even when no claim about learning is attached.

05Where it lands

Better questions for the same data

The first change is in what a team asks of its engagement figures. Who got in, where they stopped and what they skipped are questions activity data answers well. Asked those questions, a dashboard that was useless for judging learning becomes useful for fixing the product and finding the learners it is missing.

The second is a cleaner split in reporting. Activity goes in one column, under its own name. Evidence of learning goes in another, collected on purpose with tasks designed to show it. The columns are allowed to disagree, and when they do, the disagreement is worth a conversation.

The third is less comfortable. Some engagement features exist mainly to move the activity column. Read under its own name, a streak counter is a feature that raises activity. Whether it raises anything else is a separate claim, and it needs separate evidence.

The point

What transfers

The reason this page exists, rather than the story that produced it.

  1. Engagement data is weak evidence of learning and good evidence of activity. Read it for what it is good at.
  2. Report activity under its own name. A figure that makes no claim to be learning does not need defending as learning.
  3. Read the log as a map of access before comparing anyone. Low activity can mean no device.
  4. When many learners stop at the same screen, treat it as a design finding.
  5. Activity data stays honest only while nobody is judged by it.
Singapore