A WORKING GUIDE TO INTERACTIVE LEARNING

Designing interactive experiences

Where the learner has to commit. What the system should say back, and when. How to tell careful reasoning from an efficient exploit.

Four sections and twenty-five numbered moves for anyone who makes, buys or signs off a simulation, an interactive video, a practice tool or a tutor. Each move says what to do, what to watch for and how much evidence sits behind it.

Every move carries an evidence mark. Six rest on replicated findings. Thirteen rest on a real source that is narrower than the advice. Four are design conventions with no direct test. Two are contested, and say why in their own panels.

WHO THIS IS FOR

For people who make, buy or sign off interactive learning

Designers and developers building it. Teachers and leads choosing it. Anyone asked whether a prototype is worth the money.

You do not need to build software to use this guide. Most of the moves are questions you can ask of a prototype in a review meeting, before much code exists.

The subject is the moment inside a product where a learner acts and the system answers: a simulation, a branching scenario, an interactive video, a practice set, a hint system, a chat tutor. General interface design gets one move here (C·01), the one that separates the effort to remove from the effort to keep.

RESPONSIVE AND INTERACTIVE

Most interactive learning is merely responsive

A product can answer every click and still never give the learner a chance to be wrong.

FIG 01: SIX SCREENS, NONE OR THREE CHANCES TO BE WRONG

The top row is common. The learner watches, clicks Next, opens a hotspot, clicks Next, reveals an answer and clicks Next. Every click gets a response. Nothing the learner thinks changes what happens.

The bottom row uses six screens of the same length. Three of them ask the learner to commit to something they could get wrong: a prediction, an explanation, a choice with a consequence. That difference is the one I care about, and it is cheap to build once you look for it.

PREDICT BEFORE YOU LOOK · ILLUSTRATIVE CASE

Two versions of the same circuit simulation. Which teaches more?

VERSION A · CHECK AND MOVE ON

Explore the circuit, then answer

• Learners drag bulbs and batteries around freely
• A quiz at the end: five multiple-choice items
• Each item marked right or wrong at once
• Almost everyone finishes, and scores are high

VERSION B · COMMIT FIRST

Predict, then run the circuit

• Before each run, learners predict what the bulbs will do
• They run it and set the result beside their prediction
• They type one sentence on why it happened
• Fewer finish in the lesson, and early answers are often wrong

Both versions are invented to make a point, but both are common. Commit to an answer before reading on.

Version A looks better on every number a dashboard shows. Version B has three places per run where a learner can be wrong, and a record of what they thought. The close of this guide comes back to the two versions once the moves are in place.

HOW TO READ THE MARK ON EACH MOVE

Four marks

• REPLICATED Several studies or a meta-analysis agree, and the direction of the effect is stable
• SUPPORTED A real source exists, but it is narrower than the advice drawn from it
• CONVENTION A design practice with a clear rationale and no direct test
• CONTESTED The evidence is mixed, or points different ways in different settings

REPLICATEDSix of twenty-five Several independent studies with a consistent direction of effect, usually summarised in a meta-analysis. The feedback findings and the worked-example findings make up most of this group.

SUPPORTEDThirteen of twenty-five A study or review says something real, but in a narrower setting than the advice: one kind of tutor, one age group, one subject or a laboratory task. Most of this guide sits here, which is normal for design advice.

CONVENTIONFour of twenty-five Practice with sound reasoning behind it and no study that tests the move itself. Worth following. Not worth citing as research.

CONTESTEDTwo of twenty-five Studies point different ways, or the answer depends on conditions nobody has pinned down. Each of the two says so in its own panel, with my position marked as mine.

The mark sits in the eyebrow of every move, with the section letter and number. Where a mark is anything other than replicated, the panel says why.

THE CLAIM

Count the moments where a learner can be wrong

If there are none, the product is responsive. It may be pleasant to use. It is not yet asking anyone to think.

Everything in the four sections hangs off this one count. A commitment gives the system something to reply to. A reply gives the learner something to revise. And only a commitment leaves a trace in the data that can tell careful reasoning from a shortcut.

Section A

Where the learner commits

Six moves for building a moment where the learner can be wrong.

• A Where the learner commits
• B What the system says back
• C When to help, and how much
• D Careful reasoning or efficient exploit

Six moves for building a moment where the learner can be wrong. They come before any reply is designed, because a reply needs something to reply to.

01

A MOMENT TO BE WRONG

Section A

01

A moment to be wrong

Before anything else, find where the learner has to commit to something.

A commitment is any moment where the learner puts something on the record that could turn out wrong: a prediction, a choice with a consequence, an answer in their own words. Without one, the product has nothing to respond to except clicks.

The three moves here find those moments, ask for more than a selection and put a prediction ahead of every result.

A · 01 · SUPPORTED

Find the moment the learner can be wrong

Five kinds of interactivity are common. Only two of them can show a learner that their idea was wrong.

FIG 02: FIVE KINDS OF INTERACTIVITY, TWO THAT TEST AN IDEA

Moreno and Mayer (2007) sort interactivity into five kinds: dialoguing, controlling, manipulating, searching and navigating. Pausing a video and opening a menu change what is on screen. They never test whether the learner’s idea was right. Setting a variable in a model and answering a question can.

List every screen in the prototype. Mark the ones where the learner commits to something that could be wrong. If the count is zero, start there.

Hotspots, drag-to-reveal and click-to-continue. They look interactive in a demo and ask for nothing.

SUPPORTED · Moreno & Mayer, 2007

Searching sits halfway. A learner who has to form a query has to think about what they need, but the system answers the query and never the thinking behind it. The classification comes from a review, so the mark is supported. The same review sets out principles for guided activity, reflection, feedback, learner control and pretraining that the rest of this guide draws on. The step from the classification to counting commitments is mine.

A · 02 · SUPPORTED

Ask the learner to produce something

Selecting from options is easier to build. Producing an answer, a prediction or an explanation is where more of the learning happens.

FIG 03: FOUR MODES, AND LEARNING TENDS TO RISE

Chi and Wylie (2014) separate four modes of engagement. Passive is receiving. Active is doing something physical, like clicking or highlighting. Constructive is producing something new, like a prediction or an explanation. Interactive is building on a partner’s contribution. Learning tends to rise in that order.

Wherever a learner picks from options, ask whether they could type it, sketch it or set it instead. Keep selection for when the options are the point.

A drag-and-drop that feels hands-on. Moving the right label to the right box is still active, and it can be done by elimination.

SUPPORTED · Chi & Wylie, 2014

The mark is supported because ICAP is a framework, built from a review of studies that were not designed to test it. The ordering holds up well across that review. It is still a weaker claim than one experiment comparing two versions of your own product. In most products the cheapest step up is from active to constructive: one free-text box where there used to be four buttons.

A · 03 · SUPPORTED

Ask for a prediction before you show the result

A prediction costs the learner a few seconds. It turns the result into the answer to their own question.

FIG 04: CURIOSITY PEAKS WHEN THE LEARNER IS UNSURE

People are most curious about an answer when they are unsure of it, and curiosity about a question predicted better recall of surprising answers a week or two later (Kang et al., 2009). Short tests placed inside an online lecture cut mind wandering and improved learning (Szpunar et al., 2013).

Before a simulation runs or a video reveals, ask what will happen. Make the learner commit, then show the result beside their prediction.

A prediction the learner can skip, or one that disappears. Show it again when the result arrives, or it does no work.

SUPPORTED · Kang et al., 2009 · Szpunar et al., 2013 · Brame, 2016

The commitment can be a choice, a slider or a sketch. Neither study tested predictions inside a simulation. Kang and colleagues measured curiosity about trivia questions, and found it peaked when people were moderately confident they knew the answer. Szpunar and colleagues put short memory tests between segments of a recorded lecture. Brame (2016) draws the practical conclusion for educational video: give viewers something to do with it, such as questions to answer while they watch. The move follows from all three. It has not been tested as a move.

02

COMMITMENTS THAT CARRY THE IDEA

Section A

02

Commitments that carry the idea

A commitment teaches only if it is about the thing being learned.

Once a product has places where the learner commits, the next question is what they commit to. A learner can commit to which button comes next, or to how the circuit works. Only the second carries the idea.

The three moves here put the learner’s hand on the variable that matters, give an open simulation one question to answer and ask for the reason along with the answer.

A · 04 · CONVENTION

Put the learner’s hand on the idea

A control that only paces the content teaches pacing. A control that is the variable lets the learner test the idea.

FIG 05: A BUTTON THAT PACES, A SLIDER THAT TESTS

In a well-made simulation the learner changes the thing the lesson is about. Drag the resistance and watch the current fall. The PhET team built their physics simulations around direct, immediate response: move a control and the model answers at once (McKagan et al., 2008).

For each control, ask what the learner learns by using it. If the answer is “how to reach the next screen”, make the control the variable instead.

Controls that need a manual. Effort spent working out the interface is effort taken from the idea.

CONVENTION · McKagan et al., 2008 · Johnson-Glenberg, 2018

The mark is convention because I found no study that compares a pacing control with a variable control on the same content. The PhET work is design research with student interviews. It supports the practice without testing it against an alternative. Johnson-Glenberg (2018) makes the same argument for gesture and hand controls in immersive media: the action should match the concept being learned. Hers is a set of design principles. It does not test this move either.

A · 05 · REPLICATED

Replace “explore” with one question

Open discovery leaves most learners wandering. A question and a place to record each run turn the same simulation into a test of an idea.

FIG 06: THE SAME SIMULATION, WANDERED AND TESTED

Learners left to discover a model alone struggle to form hypotheses, design tests and read the results (de Jong & van Joolingen, 1998). Minimal guidance does worse than guided instruction for novices (Kirschner et al., 2006).

Simulations work well when they carry scaffolding, and which scaffolding helps depends on what the learner already knows (Chernikova et al., 2020).

Give the simulation one question to answer and a table to record each run. Ask for one change at a time.

“Explore freely” as the first instruction. Novices change three things at once and cannot say which one mattered.

REPLICATED · de Jong & van Joolingen, 1998 · Kirschner et al., 2006 · Chernikova et al., 2020

Chernikova and colleagues pooled 145 studies of simulation-based learning in higher education and found a large average effect (g = 0.85). Learners with little prior knowledge gained most from examples. Those with more gained from reflection. So the right amount of structure depends on the learner, and a product that serves both groups needs both kinds of support. Kirschner and colleagues’ paper is an argument, and a contested one. The mark rests on the reviews either side of it.

A · 06 · REPLICATED

Ask for the reason with the answer

The same right answer can rest on a right idea, a wrong idea or a guess. Only the reason tells them apart.

FIG 07: ONE ANSWER, THREE DIFFERENT IDEAS UNDER IT

Prompting learners to explain why something works is one of the most reliable moves in the literature. A meta-analysis of 64 reports found a moderate benefit (g = 0.55) for prompts that induce self-explanation (Bisra et al., 2018). The reason is also what makes a right answer worth trusting.

After a choice, ask for one sentence on why. Where free text is too slow, offer a short list of reasons that includes the common wrong ones.

A reason box that accepts anything. If “because” scores the same as a real reason, learners will learn that.

REPLICATED · Bisra et al., 2018

A list of reasons is weaker than free text, because the learner selects instead of producing. It is still much better than nothing, and it gives the system something precise to reply to. An answer followed by a reason also makes the log readable. A learner who gets the answer right and the reason wrong is telling you something a score never will.

Section B

What the system says back

Six moves for a reply that leaves the thinking with the learner.

• A Where the learner commits
• B What the system says back
• C When to help, and how much
• D Careful reasoning or efficient exploit

Six moves for a reply that leaves the thinking with the learner. A reply can show what happened, explain why or simply say right and wrong. The evidence on which of these teaches is unusually clear.

03

CONSEQUENCE AND EXPLANATION

Section B

03

Consequence and explanation

What the system should say once the learner has committed.

A verdict tells the learner whether they were right. A consequence shows them what their choice did. An explanation tells them why. These are three different replies. The research separates the first from the last sharply.

The first two moves are about what the reply carries: show the consequence, and say why. The third is about where the reply is aimed. It should point at the work.

B · 01 · SUPPORTED

Show what happened, and let the learner read it

I hold that a verdict ends the thinking. A consequence the learner can see gives them something to explain.

FIG 08: A VERDICT, AND A CONSEQUENCE TO EXPLAIN

When a learner wires two bulbs in series and both go dim, the circuit has replied. They can see the result of their choice and ask why. “Incorrect, try again” leaves them nothing to reason with but the remaining options.

Moreno and Mayer’s (2007) feedback principle points the same way: novices learn more from feedback that explains than from feedback that only corrects.

Where the domain allows it, let the model reply. Show the result of the learner’s choice, then ask them to explain it.

A consequence the learner cannot read. If the bulbs dim and nobody notices, the reply has not arrived.

SUPPORTED · Moreno & Mayer, 2007

This move is the core of my position on interactive learning: feedback should show consequences, and a verdict should come last if it comes at all. The mark is supported because the studies behind it compare explanatory feedback with corrective feedback. They do not compare a consequence shown in a model with a verdict. That step is mine. I take it because a consequence is an explanation the learner has to build, and building it is the point.

B · 02 · REPLICATED

Say why, or the reply teaches almost nothing

In computer-based learning, telling learners only whether they were right had almost no effect. Explaining why had the largest.

FIG 09: WHAT THE REPLY CARRIES, AND WHAT IT DOES

Van der Kleij et al. (2015) pooled 40 studies of feedback in computer-based environments. Right or wrong alone averaged an effect of 0.05. The correct answer averaged 0.32. Elaborated feedback, which explains, averaged 0.49.

A broader meta-analysis found the same ordering across education: the more information feedback carries, the more it does (Wisniewski et al., 2020).

Every reply to a wrong answer should say what that answer implies and why it does not hold. Write these for the commonest wrong answers first.

A green tick and a red cross with nothing else. They are the cheapest reply to build and the one that does least.

REPLICATED · Van der Kleij et al., 2015 · Wisniewski et al., 2020

Both meta-analyses report wide variation between studies, so the averages say more about direction than size. Van der Kleij and colleagues also found that elaborated feedback did especially well against the other two for higher-order outcomes. Writing elaborated feedback is real work. A practical order is to write it for the three or four wrong answers most learners give, and use a plain correct-answer reply for the rest.

B · 03 · REPLICATED

Point the reply at the work

Feedback raises performance on average. In one large review, more than a third of feedback interventions made it worse.

FIG 10: OVER A THIRD OF EFFECTS BELOW ZERO

Kluger and DeNisi (1996) found that feedback improved performance on average, and that over a third of interventions lowered it. Feedback that drew attention to the self, and away from the task, did most harm.

Hattie and Timperley (2007) reached a similar view. Feedback about the task and the process works. Praise of the person does little.

Write replies about the work: what the answer shows and what to try next. Cut “Great job!” unless it comes with something to use.

Scores, rankings and streaks shown at the moment of an error. They turn attention to standing, away from the task.

REPLICATED · Kluger & DeNisi, 1996 · Hattie & Timperley, 2007 · Butler, 1988

Butler (1988) gave pupils comments, grades or both on their work. Interest and performance were highest after comments alone, and grades with comments did no better than grades. That is one study, in twelve classes, and it is often cited further than it reaches. Points and badges are also replies about standing. On average they do no harm: a meta-analysis found small positive effects of gamification on learning, and the effect on cognitive outcomes held up best in rigorous studies (Sailer & Homner, 2020). They still carry nothing about the task, so they cannot stand in for a reply that does.

04

A REPLY THE LEARNER CAN USE

Section B

04

A reply the learner can use

A reply only teaches if the learner does something with it.

A well-written reply still fails if the learner cannot see enough of the system to understand it, never judges their own work or has nothing to do once they have read it.

The three moves here make the model visible, turn the learner into a judge before the system scores and end every reply with something to do.

B · 04 · CONVENTION

Let the learner see enough of the model to reason with

A learner cannot explain a consequence produced by a black box. Show the parts that do the work.

FIG 11: TRIAL AND ERROR, OR A THEORY TO TEST

I argue that if the rules are hidden, the only strategy left is trial and error. PhET simulations show what is usually invisible, like electrons moving in a wire, so that learners can form a theory of the system and test it (Wieman et al., 2008).

I hold a stronger version: an interactive system should make its own model visible enough for a learner to predict it.

Show the variables the system uses and how they change. If a score or a recommendation is computed, show what went into it.

Showing everything. A visible model with forty variables is as opaque as a hidden one. Show the few that carry the idea.

CONVENTION · Wieman et al., 2008 · McKagan et al., 2008

The mark is convention because the stronger version is my position, argued from design practice and the PhET work. Nobody has tested it as stated. It matters most for adaptive products. When a system decides what a learner sees next, the learner should be able to tell why. Otherwise the only theory they can form is about how to make the system move on, and that is where gaming starts (Section D).

B · 05 · SUPPORTED

Make the learner a judge of the work

Judging work against a standard teaches the standard. Learners who review a peer’s work improve their own.

FIG 12: THE LEARNER JUDGES, THEN THE SYSTEM SCORES

Across 54 studies, peer assessment improved performance compared with no assessment (g = 0.31) and with teacher assessment (g = 0.28), according to Double et al. (2020). In one study of writing, students who only gave feedback improved more than those who only received it (Lundstrom & Baker, 2009).

Before a submission is scored, ask the learner to check it against a strong example and a short list of criteria. Then let them revise.

Peer review before learners have seen what strong work looks like. They will praise each other and miss the same errors.

SUPPORTED · Double et al., 2020 · Lundstrom & Baker, 2009 · Panadero & Jonsson, 2013

The writing study was of second-language students at university. Double and colleagues found peer assessment did about as well as self-assessment (g = 0.23, with no significant difference), and the effect held across a wide range of settings. Rubrics help through several routes, including clearer expectations and better self-assessment, though the effect is not automatic (Panadero & Jonsson, 2013). Most of these studies ran in classrooms, so the evidence is for the routine and says little about any particular software. In a product, the cheapest version is a self-check screen before submission: one example, three criteria and a revise button.

B · 06 · CONVENTION

Give the learner something to do with the reply

A reply followed by the same item invites elimination. A reply followed by a new item asks the learner to use it.

FIG 13: A REPLY, THEN A FRESH ATTEMPT

Hattie and Timperley (2007) describe good feedback as answering three questions: “Where am I going?”, “How am I going?” and “Where to next?” Most products answer the second and stop. The third is the one that makes the learner act.

End every reply with an action: a new item that uses the idea, a revision or a prompt to explain what changed.

“Try again” on a multiple-choice item. With four options it guarantees a tick within four clicks and teaches nothing about why.

CONVENTION · Hattie & Timperley, 2007

When the next item repeats the idea, vary the numbers so the old answer does not carry over. The mark is convention because the three questions are a model of feedback drawn from a review, and following a reply with a fresh item is design practice more than a tested finding. It also protects the data. A learner who gets a new item right after a reply has shown something. A learner who gets the same item right by elimination has shown nothing, and the log cannot tell which happened unless the item changed.

Section C

When to help, and how much

Seven moves for timing, hints and the struggle worth keeping.

• A Where the learner commits
• B What the system says back
• C When to help, and how much
• D Careful reasoning or efficient exploit

Seven moves for timing, hints and the struggle worth keeping. The design question is when the system should step in, and how much it should do once it does.

05

THE RIGHT DIFFICULTY

Section C

05

The right difficulty

Some struggle is the learning. Some is the interface getting in the way.

Effort spent on the interface is waste. Effort spent on the idea is often how the learning happens. A product that removes both looks easy, tests well in a demo and teaches less.

The four moves here sort the two kinds of difficulty, decide when confusion is worth letting run, choose what comes first and time the reply to the goal.

C · 01 · SUPPORTED

Tell the two kinds of difficulty apart

Remove the struggle with the interface. Keep the struggle with the idea, and expect learners to dislike it.

FIG 14: SORT EVERY STRUGGLE BEFORE YOU FIX IT

Learners often judge their learning by how easy it felt. Students in actively taught physics lectures learned more, and felt they had learned less (Deslauriers et al., 2019).

People also rated massed study of paintings as more effective than spaced study, even when spacing had helped them more (Kornell & Bjork, 2008).

After a test session, sort every struggle into two lists: the interface and the idea. Fix the first now. Keep the second while it produces new attempts.

Satisfaction scores used to judge difficulty. Learners may rate the version that teaches more as harder, and feel they learned less from it.

SUPPORTED · Kornell & Bjork, 2008 · Deslauriers et al., 2019

The note “Ease of use is not ease of learning” on this site makes the longer argument. The short version for a review meeting: when someone reports that learners found a task hard, ask what they found hard. A hidden button is a defect. A prediction that keeps failing may be the lesson working. The mark is supported because both studies are with adults, one in a laboratory and one in a lecture hall, and neither tested an interactive product.

C · 02 · SUPPORTED

Judge confusion by what the learner does next

Resolved, confusion leads back to engagement. Unresolved, it tends to sour into frustration and then boredom.

FIG 15: WHERE CONFUSION GOES NEXT

D’Mello and Graesser (2012) tracked learners’ emotions while they worked with a computer tutor. Confusion could go either way. Resolved, it gave way to engagement. Left alone, it tended to turn into frustration and then boredom.

A later study, in which animated tutors disagreed on purpose, found that confusion can help learning when it is induced and then resolved (D’Mello et al., 2014).

My rule of thumb: while a confused learner keeps trying something new, let them work. When the attempts start to repeat, offer the first level of help.

Help that fires on the first wrong answer. It removes the confusion before it can do any good.

SUPPORTED · D’Mello & Graesser, 2012 · D’Mello et al., 2014

The rule about repeating attempts is mine. It turns a finding about emotions into something a product can see, because a log can record attempts. It cannot record feelings. Treat it as a design hypothesis. The research says confusion can help and can turn sour. It does not say how many attempts to wait. Learners who already doubt they can succeed may give up sooner, so the wait should be shorter for them.

C · 03 · REPLICATED

Open with the problem when the goal is a concept

For a new concept, letting learners attempt a problem before instruction beat teaching first, on average. Younger pupils were the exception.

FIG 16: WHICH COMES FIRST DEPENDS ON THE GOAL

A meta-analysis of 53 studies compared problem solving followed by instruction with the usual order. Problem first came out ahead (g = 0.36), and further ahead when it followed the principles of productive failure closely (Sinha & Kapur, 2021).

The idea is that the attempt shows learners what they cannot yet do, so the teaching that follows has somewhere to land.

For a concept, open with a problem learners can attempt several ways but not yet solve. Then teach by setting their attempts beside the expert solution.

Younger pupils and general skills. For grades two to five, and for domain-general skills, the studies favoured teaching first.

REPLICATED · Sinha & Kapur, 2021

This sits inside an argument the field has not settled. The case for teaching first rests on cognitive load: novices solving unfamiliar problems spend working memory on search (Kirschner et al., 2006). The case for problems first rests on studies like these. I think both sides agree on more than the argument suggests. A procedure with no idea to discover should be taught directly, with worked examples (C·07). A concept with something to discover is where the attempt earns its place. In a product, the attempt has to be one the learner can make several ways, or the struggle is with the interface.

C · 04 · CONTESTED

Time the reply to what is being learned

I time the reply to the goal: at once for a basic skill, after a whole answer for judgement. The evidence on timing points both ways.

FIG 17: TWO TIMINGS FOR TWO KINDS OF GOAL

Classroom studies of quizzes have mostly favoured immediate feedback. Laboratory studies of learning test content have mostly favoured a delay (Kulik & Kulik, 1988). In an engineering course, feedback delayed by a week led to better exam results on new problems (Mullet et al., 2014).

For fluency in a basic skill, reply after each step. For judgement, let the learner finish and commit, then reply to the whole.

Instant feedback by default. It is easy to build and learners like it. Neither fact shows that it builds judgement.

CONTESTED · Kulik & Kulik, 1988 · Mullet et al., 2014 · Van der Kleij et al., 2015

The engineering students said they preferred immediate feedback, even though the delay served them better. Decide which goal an activity serves before choosing its timing. My position is the one stated on this site: timing should follow the goal. Immediate replies suit a skill that has to become automatic. A delay suits judgement, because it leaves the learner time to commit and to check their own reasoning. The mark is contested because the studies do not line up neatly behind that rule. In computer-based feedback, delay went with smaller effects across the studies, and the pattern of immediate for lower-order and delayed for higher-order outcomes appeared without reaching significance (Van der Kleij et al., 2015). Treat the rule as a design hypothesis worth testing in your product.

06

HELP THAT WITHDRAWS

Section C

06

Help that withdraws

Good help waits for an attempt, comes in small steps and shrinks as the learner improves.

Most products offer help on demand and in full. A review of help seeking found that learners often use on-demand help badly, even though using it well goes with better learning (Aleven et al., 2003).

The three moves here put help behind an attempt, give it in levels that never reach the answer and fade it over time.

C · 05 · SUPPORTED

Make the learner try before the system helps

Help on demand lets the learner skip the thinking. Help that waits for an attempt replies to what the learner actually did.

FIG 18: HELP ON DEMAND, AND HELP AFTER A TRY

A review of help seeking found that learners often use on-demand help badly (Aleven et al., 2003). Learners who click through hints until the answer appears learn less (Baker et al., 2004). In a field experiment, an unguarded chat tutor raised practice grades and then lowered exam grades (Bastani et al., 2025).

Put help behind an attempt. The learner submits something, even a wrong first step, and the first hint replies to it.

A gate that is easy to satisfy. If any keystroke counts as an attempt, learners will type one and move on.

SUPPORTED · Aleven et al., 2003 · Baker et al., 2004 · Bastani et al., 2025

Koedinger and Aleven (2007) call the underlying problem the assistance dilemma. Give too much help and learners stop thinking. Give too little and they flounder. No single setting resolves it, which is why the next two moves are about how much help to give and when to take it away. The mark is supported because I found no study that tests an attempt gate on its own. The tutor version in Bastani and colleagues’ experiment carried several safeguards, so its result cannot be credited to any one of them.

C · 06 · SUPPORTED

Make every hint smaller than the answer

Raise help one level after a miss and drop it one level after a success. Leave out the level that gives the answer away.

FIG 19: HELP RISES AND FALLS WITH EACH ATTEMPT

Scaffolding has three features: contingency, fading and the transfer of responsibility to the learner (van de Pol et al., 2010). Wood, Bruner and Ross (1976) described the tutor’s job as taking over the parts of a task the learner cannot yet manage, so the learner can complete the rest.

Write help for each hard step in levels that run from general to specific. The most specific level should still leave the learner one thing to do.

A bottom-out hint, the last level that simply states the answer. It turns the hint button into an answer button, and the log will show learners racing to it.

SUPPORTED · van de Pol et al., 2010 · Wood et al., 1976 · Baker et al., 2004

The evidence that scaffolding helps is thinner than its reputation. Van de Pol and colleagues found many descriptive studies and only a small number that tested effectiveness, and those suggested it works. Most of that research is on teachers talking with students. The levels-of-help design comes from intelligent tutors, where hints run from general to specific and the last one usually gives the answer. That last hint is the one learners who game the system click through to (Baker et al., 2004).

C · 07 · REPLICATED

Take the support away as the learner improves

Worked examples beat problem solving for novices. The same examples get in the way once learners know the method.

FIG 20: THE LAST STEP IS THE FIRST TO GO

Novices learn a procedure faster from studying worked examples than from solving the same problems (Sweller & Cooper, 1985). As they gain skill the effect reverses, and detailed guidance starts to get in the way (Kalyuga et al., 2003).

Fading the steps of an example one at a time manages the handover (Renkl & Atkinson, 2003).

Start with a complete worked example. On each later problem, leave one more step for the learner, starting from the last.

Support that never changes. Hints and worked steps that help a beginner can slow down a learner who already knows the method.

REPLICATED · Sweller & Cooper, 1985 · Renkl & Atkinson, 2003 · Kalyuga et al., 2003

Drop the support altogether for learners who show they no longer need it. Renkl and Atkinson describe fading from the end and fading from the start. I prefer fading from the end in a product, because the learner always finishes the problem, so every item ends with a result they produced. The harder question is when to fade. The cleanest signal is performance on an unassisted item. Time spent and items completed are weaker signals. A learner who solves a problem with no worked steps has earned one fewer next time.

Section D

Careful reasoning or efficient exploit

Six moves for closing shortcuts and reading what the log can show.

• A Where the learner commits
• B What the system says back
• C When to help, and how much
• D Careful reasoning or efficient exploit

Six moves for closing shortcuts and reading what the log can show. The first three are design decisions. The last three are about what interaction data can and cannot tell you.

07

THE SHORTCUT

Section D

07

The shortcut

If a product can be finished without the thinking, some learners will finish it that way.

Baker and colleagues define gaming the system as getting correct answers and moving on by systematically exploiting regularities in the software’s feedback and help. In the classrooms they observed, no other off-task behaviour was as strongly tied to poor learning (Baker et al., 2004).

The three moves here find the shortcuts before learners do, price them out and change what counts as success.

D · 01 · SUPPORTED

Play it badly before the learners do

Some learners will look for the fastest way through. Find it first, and decide whether it is a problem.

FIG 21: TWO ROUTES TO THE SAME FINISH

In the tutor classrooms Baker and colleagues observed, gaming the system was the off-task behaviour most strongly tied to poor learning. How often a student gamed predicted their post-test score about as strongly as their prior knowledge did (Baker et al., 2004).

Their analysis suggested learned helplessness or a focus on performance as likelier reasons for gaming than a lack of interest.

Before release, try to reach the end without learning anything. Write down every route that works, and decide which ones to close.

Treating gaming as a character flaw. If it comes from feeling unable to succeed, a harder lock will not fix it. Easier first steps might.

SUPPORTED · Baker et al., 2004

Guess, click through hints and repeat old answers. The finding comes from one family of tutors, in the classrooms one team observed, with gaming coded by observers. It is still the clearest evidence that a shortcut in the software can cost learning. The point about motive matters for design. If learners game because they expect to fail, closing every shortcut leaves them stuck with nothing to do. The fix is partly in the lock and partly in the difficulty of the first steps (Section C).

D · 02 · CONVENTION

Make every shortcut cost more than thinking

A shortcut gets taken when it is cheaper than the thinking. Raise its price above the thinking, and the thinking becomes the easier route.

FIG 22: PRICE EACH SHORTCUT ABOVE THE THINKING

Three shortcuts are built into most interactive products. Guessing until right works wherever a wrong answer costs nothing. Clicking to the last hint works wherever the last hint is the answer. Reusing the last answer works wherever the item stays the same after a reply.

Ask for a reason with each answer (A·06), make help wait for an attempt (C·05) and give a new item after each reply (B·06).

Penalties that punish honest mistakes. Docking points for a wrong answer also teaches learners not to try.

CONVENTION · Baker et al., 2004 · Aleven et al., 2003

The mark is convention because I found no study that tests these fixes as a set. Each one follows from the move cited beside it. One check on any fix: after it, does a careful learner take any longer? If honest learners are slowed as much as the ones gaming, the fix has raised the price of the thinking too.

D · 03 · SUPPORTED

Judge it by what the learner can do without it

A tool can raise scores while it is switched on and lower them once it is gone. Only an unassisted check can tell.

FIG 23: PRACTICE ROSE, THE EXAM DID NOT FOLLOW

In a field experiment with nearly a thousand high school maths students, a standard chat interface raised practice grades by 48 per cent. Once access was taken away, those students did 17 per cent worse on the exam than students who never had it (Bastani et al., 2025).

A version with prompts designed to protect learning raised practice grades by 127 per cent and largely avoided the harm.

Measure learning with the tool switched off: a short unassisted item after each section, or a check a week later. Report it beside usage.

Practice scores and completion rates offered as evidence of learning. On practice grades, the standard chat interface looked like a success.

SUPPORTED · Bastani et al., 2025

This is one study, in one country and one subject, with one model generation. It is still the clearest case I know of a product that looked better on its own numbers while it left learners worse off. The design lesson travels further than the AI. Any interactive product that does part of the work for the learner can raise scores during use. The only way to find out whether it taught anything is to take it away and look.

08

READING THE LOG

Section D

08

Reading the log

The same fast click can be expertise or an exploit. The log alone often cannot say which.

Interaction data records what learners did and when. It does not record why. Telling careful reasoning from an efficient exploit using that data alone is still an open question.

The three moves here read speed against the item, read it again against what the learner has mastered and then admit the limit. When the log cannot tell, ask.

D · 04 · SUPPORTED

Read the response time beside the answer

An answer given faster than the item can be read is a guess, whatever it scores.

FIG 24: SOME ANSWERS ARRIVE BEFORE THE READING DOES

Wise and Kong (2005) proposed a measure of effort built on response time. An answer given too fast to have read and considered the item counts as a rapid guess. The share of items answered with real effort gave a usable index of how hard a test taker was trying, with supporting evidence for its reliability and validity.

Set a threshold for each item from its length and difficulty. Log every answer below it as a rapid response, and report the share beside the score.

One threshold for every item. A short item read in two seconds and a long one skimmed in two seconds are different events.

SUPPORTED · Wise & Kong, 2005

The measure was built for low-stakes computer-based tests, where the worry is that test takers do not try. In a learning product the question is a little different, but the logic carries. A fast right answer to a hard item is weak evidence of mastery until you know it was not a guess. A rapid wrong answer tells you nothing about what the learner believes. Neither should move a mastery estimate the way a considered answer does.

D · 05 · SUPPORTED

Look at where the speed happens

Fast on a step the learner has mastered is efficiency. Fast on a step they have never got right is the exploit.

FIG 25: THE SAME SPEED, READ AGAINST MASTERY

Detectors of gaming built from tutor logs use context as well as speed: how fast, after how many errors and on which kind of step. Baker et al. (2008) built one that worked across different tutor lessons, and it separated gaming that went with poor learning from gaming that did not.

Read each fast response against the learner’s record on that skill. Flag speed on steps they have never got right. Leave speed on mastered steps alone.

Reading the log as a record of thinking. It records actions and times, and a learner can think off screen, on paper or out loud.

SUPPORTED · Baker et al., 2008 · Aleven et al., 2003

The two-by-two is a simplification. Slow is not always reasoning: a slow learner may be stuck or away from the screen. Fast is not always gaming: a learner who has worked the answer out on paper will enter it quickly. The cell to worry about is fast on steps the learner has not mastered, and even there the log raises a question without answering it. The detector was built on one family of tutors.

D · 06 · CONTESTED

When the log cannot tell, ask the learner

In my judgement, logs raise questions about intent and rarely settle them. A short question to the learner often can.

FIG 26: THE LOG FLAGS, THE LEARNER EXPLAINS

Baker et al. (2004) coded gaming by watching classrooms. Baker et al. (2008) built a detector from logs that can flag likely gaming. Neither can see a learner who worked the answer out on paper and entered it quickly. Whether interaction data alone can tell careful reasoning from an efficient exploit is still open.

When the log flags a pattern, ask. One question at the next pause, “How did you get that?”, gives you evidence the log does not hold.

Asking as an accusation. If the question feels like being caught, learners will learn to look slow instead of thinking.

CONTESTED · Baker et al., 2004 · Baker et al., 2008

The mark is contested because the field has not settled how far logs can go, and some researchers would argue the log can carry more of the weight than I give it. My position is cautious. The log is good at finding where to look and poor at saying why. A question to the learner, or a short look at their working, is cheap and settles more than the data can. It also tells the learner that their reasoning is what counts.

09

PUTTING IT TO WORK

09

Putting it to work

Six questions for the next prototype you review, and where the evidence runs out.

The six questions come from the position on interactive learning on this site. Each one points back to the moves that answer it.

The appendices hold the questions as a table to print, the log signals with their innocent explanations, the terms and an evidence note for every move.

SIX QUESTIONS

Ask these of the next prototype

• 01 Where does the learner have to commit to something they could be wrong about?
• 02 What does the learner generate, and what do they only select?
• 03 Does the feedback report a consequence or a verdict?
• 04 Is the feedback timed for building a skill or for building judgement?
• 05 Can the learner see enough of the system to form a theory of it?
• 06 What would a learner gaming this look like in the data, and could we tell?

01Where does the learner have to commit? Moves A·01 and A·03. Count the moments. If there are none, nothing else in this guide has anything to work on.

02Generate or select? Moves A·02 and A·06. Look for free text, sketches and settings where the idea matters. Look for a reason beside each answer.

03Consequence or verdict? Moves B·01 to B·03. Look for replies that show what happened, say why and point at the work.

04Skill or judgement? Move C·04. Immediate replies for fluency. A whole answer committed before the reply for judgement. This is the contested one.

05Can the learner form a theory? Move B·04. The few variables that carry the idea, visible and changing.

06Could we tell? Moves D·01 to D·06. A list of the shortcuts someone found by playing it badly, a response-time threshold, an unassisted check and a plan to ask.

These six come from the position on interactive learning on this site. Appendix A sets them out as a table to take into a review.

BACK TO THE TWO SIMULATIONS

Version B gives the learner more chances to be wrong

VERSION A

Looks better on the dashboard

• Every click answered, no idea tested
• Verdicts with nothing to reason with
• A high score that cannot tell a guess from a reason
• Nothing in the log to separate careful from quick

VERSION B

Gives you something to read

• Three commitments per run
• The circuit replies with a consequence
• A reason recorded beside every answer
• A log that can separate a guess from a reason

Version A wins on completion and on scores, and those are the numbers most reviews ask for. Version B has the moves in this guide: commitments that carry the idea (Section A), replies that show consequences (Section B), struggle kept on the idea (Section C) and a record that can tell reasoning from a shortcut (Section D).

Neither version has been tested. The claim here is narrower. Only Version B gives you the evidence to find out which one teaches.

Count the moments where a learner can be wrong.

Reply with what happened, and make help wait for an attempt.

Then judge the product by what learners can do without it.

END

Count them on the next prototype

Before anyone shows you the click data.

• WEB geraldajam.com
• THE POSITION geraldajam.com/ideas/interactive
• GUIDES geraldajam.com/guides

Reference layer

Appendix

Appendix A

Six questions for a review

The six questions from the position on interactive learning, with what a good answer looks like and the moves behind each one.

Appendix B

Signals in the log, and their innocent explanations

Every signal below has a version that is gaming and a version that is fine. The log shows the signal. It rarely shows which version you are looking at.

Appendix C

Terms

The terms this guide uses, in the sense it uses them.

01Assistance dilemma How much help to give. Too much and learners stop thinking. Too little and they flounder (Koedinger & Aleven, 2007).

02Attempt gate A rule that help appears only after the learner has submitted an attempt.

03Bottom-out hint The last hint in a sequence, which states the answer.

04Commitment A moment where the learner puts something on the record that could turn out wrong: a prediction, a choice with a consequence or an answer in their own words.

05Consequence A reply in which the system shows what the learner’s choice did, for the learner to explain.

06Contingency Help that responds to what the learner just did: more after a miss and less after a success (van de Pol et al., 2010).

07Elaborated feedback A reply that explains why an answer holds or does not, or gives a hint or a worked step (Van der Kleij et al., 2015).

08Expertise reversal effect Guidance that helps novices can get in the way of learners who already know the material (Kalyuga et al., 2003).

09Fading Withdrawing support as the learner improves, for example by removing the steps of a worked example one at a time (Renkl & Atkinson, 2003).

10Gaming the system Getting correct answers and moving on by exploiting regularities in a system’s feedback and help (Baker et al., 2004).

11Generation and selection Producing an answer, a prediction or an explanation, compared with picking one from a list. Generation sits higher in the ICAP framework.

12ICAP Passive, active, constructive and interactive: four modes of engagement, in rising order of expected learning (Chi & Wylie, 2014).

13Levels of help Help for one step written from general to specific, with each level opened by a further missed attempt.

14Productive failure A design in which learners attempt a problem before being taught the concept, then compare their attempts with the expert solution (Sinha & Kapur, 2021).

15Productive struggle Effort on the idea that leads to learning, as distinct from effort spent working out the interface.

16Rapid guessing Answering faster than the item could be read and considered (Wise & Kong, 2005).

17Response time effort The share of items answered with enough time to have engaged with them (Wise & Kong, 2005).

18Responsive A product that answers clicks. An interactive one answers ideas, so the learner can find out they were wrong.

19Unassisted check An item or test taken with the tool’s help switched off, to see what the learner can do alone.

20Verdict A reply that says only whether the learner was right. The research calls it knowledge of results.

Appendix D

Evidence notes

Every move, its mark, what the mark rests on and where the evidence stops. References were checked against Crossref and OpenAlex records, and claims against abstracts or full text, on 12 November 2026. Where a finding was read only through a secondary account, the row says so.

Source trail

References

Every source the twenty-five moves rest on. Entries follow APA 7th edition, and links point to the DOI. Each was checked against a Crossref or OpenAlex record while this guide was assembled. Appendix D says which findings were read only in an abstract or through a summary.

1. Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(3), 277–320. https://doi.org/10.3102/00346543073003277
2. Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the cognitive tutor classroom: When students “game the system”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 383–390). ACM. https://doi.org/10.1145/985692.985741
3. Baker, R. S. J. d., Corbett, A. T., Roll, I., & Koedinger, K. R. (2008). Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, 18(3), 287–314. https://doi.org/10.1007/s11257-007-9045-6
4. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), Article e2422633122. https://doi.org/10.1073/pnas.2422633122
5. Bisra, K., Liu, Q., Nesbit, J. C., Salimi, F., & Winne, P. H. (2018). Inducing self-explanation: A meta-analysis. Educational Psychology Review, 30(3), 703–725. https://doi.org/10.1007/s10648-018-9434-x
6. Brame, C. J. (2016). Effective educational videos: Principles and guidelines for maximizing student learning from video content. CBE-Life Sciences Education, 15(4), Article es6. https://doi.org/10.1187/cbe.16-03-0125
7. Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58(1), 1–14. https://doi.org/10.1111/j.2044-8279.1988.tb00874.x
8. Chernikova, O., Heitzmann, N., Stadler, M., Holzberger, D., Seidel, T., & Fischer, F. (2020). Simulation-based learning in higher education: A meta-analysis. Review of Educational Research, 90(4), 499–541. https://doi.org/10.3102/0034654320933544
9. Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
10. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179–201. https://doi.org/10.3102/00346543068002179
11. Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116
12. D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. https://doi.org/10.1016/j.learninstruc.2011.10.001
13. D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170. https://doi.org/10.1016/j.learninstruc.2012.05.003
14. Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32(2), 481–509. https://doi.org/10.1007/s10648-019-09510-3
15. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
16. Johnson-Glenberg, M. C. (2018). Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers in Robotics and AI, 5, Article 81. https://doi.org/10.3389/frobt.2018.00081
17. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31. https://doi.org/10.1207/S15326985EP3801_4
18. Kang, M. J., Hsu, M., Krajbich, I. M., Loewenstein, G., McClure, S. M., Wang, J. T., & Camerer, C. F. (2009). The wick in the candle of learning: Epistemic curiosity activates reward circuitry and enhances memory. Psychological Science, 20(8), 963–973. https://doi.org/10.1111/j.1467-9280.2009.02402.x
19. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1
20. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284. https://doi.org/10.1037/0033-2909.119.2.254
21. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239–264. https://doi.org/10.1007/s10648-007-9049-0
22. Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: Is spacing the “enemy of induction”? Psychological Science, 19(6), 585–592. https://doi.org/10.1111/j.1467-9280.2008.02127.x
23. Kulik, J. A., & Kulik, C.-L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97. https://doi.org/10.3102/00346543058001079
24. Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18(1), 30–43. https://doi.org/10.1016/j.jslw.2008.06.002
25. McKagan, S. B., Perkins, K. K., Dubson, M., Malley, C., Reid, S., LeMaster, R., & Wieman, C. E. (2008). Developing and researching PhET simulations for teaching quantum mechanics. American Journal of Physics, 76(4), 406–417. https://doi.org/10.1119/1.2885199
26. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326. https://doi.org/10.1007/s10648-007-9047-2
27. Mullet, H. G., Butler, A. C., Verdin, B., von Borries, R., & Marsh, E. J. (2014). Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition, 3(3), 222–229. https://doi.org/10.1016/j.jarmac.2014.05.001
28. Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. https://doi.org/10.1016/j.edurev.2013.01.002
29. Renkl, A., & Atkinson, R. K. (2003). Structuring the transition from example study to problem solving in cognitive skill acquisition: A cognitive load perspective. Educational Psychologist, 38(1), 15–22. https://doi.org/10.1207/S15326985EP3801_3
30. Sailer, M., & Homner, L. (2020). The gamification of learning: A meta-analysis. Educational Psychology Review, 32(1), 77–112. https://doi.org/10.1007/s10648-019-09498-w
31. Sinha, T., & Kapur, M. (2021). When problem solving followed by instruction works: Evidence for productive failure. Review of Educational Research, 91(5), 761–798. https://doi.org/10.3102/00346543211019105
32. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2(1), 59–89. https://doi.org/10.1207/s1532690xci0201_3
33. Szpunar, K. K., Khan, N. Y., & Schacter, D. L. (2013). Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences, 110(16), 6313–6317. https://doi.org/10.1073/pnas.1221764110
34. van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22(3), 271–296. https://doi.org/10.1007/s10648-010-9127-6
35. Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511. https://doi.org/10.3102/0034654314564881
36. Wieman, C. E., Adams, W. K., & Perkins, K. K. (2008). PhET: Simulations that enhance learning. Science, 322(5902), 682–683. https://doi.org/10.1126/science.1161948
37. Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. https://doi.org/10.1207/s15324818ame1802_2
38. Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, Article 3087. https://doi.org/10.3389/fpsyg.2019.03087
39. Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

End of the guide

Back to geraldajam.com