Probing with options open

Where a plan cannot know its result, it can still learn quickly and cheaply. This move swaps detailed rules for a few enabling ones, runs several small probes at once with the criteria for spreading or stopping each written before they start, and audits which commitments in the plan cannot be undone.

Complexity: the four moves

Move 3 of 4 · Learns by acting, without spending the future

Try several small things at once, and decide in advance what would stop them

Time: 2 hours
People: 4–10, including the people who will run the probes
You need: The reading from the second move, the plan, the probe grid and the option audit, printed
You leave with: A direction statement, three to five probes with written criteria, and an option audit that names every one-way door

Needs some practice

The idea

What the move is, and why it works

What the move is

Probing is acting in order to find out. It replaces one large rollout with a portfolio of small, different tries, run at the same time. Each probe is safe to fail: if it goes wrong, the harm stays small and can be undone. Each has two criteria, written down before it starts. One says what would make you spread it. The other says what would make you stop or shrink it.

The second half of the move is an audit. It looks at the plan’s commitments and asks which of them spend the future: contracts, public targets, anything hard to take back. Those get decided slowly and on purpose.

The idea behind it

In a complex situation the result comes from interaction, so it cannot be specified in advance. What a plan can shape is the conditions that results grow from. Some rules hold behaviour inside limits. Others let good patterns form. A short rule everyone keeps does more than twenty nobody reads.

The line between a probe and a pilot matters. A pilot is run to show that something works, so stopping it counts as failure. A probe is run to find out, so stopping it on its own criteria counts as the method working. That is why the criteria go in first.

Why it works

It lowers the cost of being wrong and shortens the time to learning. A blueprint learns at the end of the year, when the report comes in. A probe learns in a fortnight. Running several together also stops any one of them becoming a cause someone has to defend.

The option audit works for a different reason. The value of being able to change course is invisible until the day it is needed, so it is usually given up by default, for a discount or a quick win. The audit makes someone decide to give it up, or not.

When

When to reach for it, and when not to

Use it when

A complex part of the plan is about to be rolled out to everyone.
A contract, target or public commitment that nobody could undo is about to be signed.
The reading has found someone already doing what the plan wants, and you want to know if it travels.
Staff disagree about what will work, and arguing has stopped producing anything.

Skip it when

The part is complicated. Plan it and do it.
A probe could do harm that cannot be undone, for example to a student’s record or a family’s trust. Find a smaller probe, or do not probe.
The outcome is already promised to the principal or the cluster. Call it a pilot and be honest about it.
Little more will be learned and delay is costly. Commit.

Before

Before anyone is in the room

Bring the reading from the second move: the attractor, the slow variables, and who is already doing something.
List every commitment in the plan that has a date, a signature or a public figure attached.
Work out what each likely probe would cost in teacher time per fortnight.
Agree who has the authority to stop a probe without calling a meeting.
Print the probe grid and the option audit.

Steps

How to run it

7 steps, about 2 hours in all. The questions in quotation marks are meant to be read out as they are.

01
Write a direction, not a destination
15 min
Rewrite the plan’s aim as a direction the school can move in, with no number at the end. A destination says “95 per cent of books compliant by the end of the year”. A direction says which way is better and leaves room for the route. Keep it to one sentence a student could understand.
Ask the room
- If we could not use a number, how would we say what we want?
- Would a student recognise this sentence as being about them?
- Which way would this sentence tell a teacher to move on a Tuesday afternoon?
Watch for A direction so vague that anything satisfies it. If no practice could fail it, sharpen it.
02
Swap a detailed rule for an enabling one
15 min
Take the rule in the plan that most tells people how to do something, and ask what minimum it really needs. Replace the full protocol with the few things that must be true, and let departments find the rest. Write the minimum down.
Ask the room
- What is the least this rule must say for the direction to hold?
- What are we specifying that a good teacher would decide better in the room?
- What must never happen, whatever else changes?
Watch for The minimum growing back into a protocol. If it runs past five lines, it is the old rule again.
03
Design three to five different probes
30 min
Build a small portfolio. Make the probes different from each other, not variants of one idea. Start from what the reading found already happening. Include one you expect to fail. Each should cost little and run for a few weeks in a few classes.
Ask the room
- Who is already doing something like this, and could we watch it on purpose?
- Which probe do we expect to fail, and what would we learn if it did?
- If this went wrong, who would be hurt, and could we undo it?
- Which department would agree to which probe, and which would not?
Watch for A portfolio of one idea at three sizes. Or a probe that is really a pilot already promised to someone.
04
Write amplify and dampen criteria first
25 min
For each probe, write what you would see if it were working, which would make you spread it, and what would make you stop or shrink it. Use things people can observe, not scores. Set a review date and an owner. Do not start a probe until both criteria are written down.
Ask the room
- What would we see in four weeks if this were working?
- What would make us stop, even if people liked it?
- What would tell us people are performing for the probe instead of doing it?
- Who looks, and when?
Watch for Criteria that only allow success. If nothing could trigger the dampen line, it is not a probe.
05
Audit the options the plan spends
20 min
Go through every commitment with a date, a signature or a public figure. For each, ask whether it could be undone next term, and at what cost. Sort them into two-way doors, which you can walk back through, and one-way doors, which you cannot. Name what each one-way door gives up.
Ask the room
- If we were wrong about this, could we undo it within a term?
- What does this commit us to that nobody actually decided?
- What would we lose the chance to do if we signed this today?
Watch for Discounts. A saving for committing early is the price someone is paying for your options. Ask whether it is enough.
06
Place each big decision on the optionality grid
10 min
For each one-way door, ask two questions. Is more learning expected soon? Is delay costly? High learning and low cost of delay: stage it and defer. High learning and high cost of delay: move now, keep a route to switch, and set triggers. Low learning and high cost of delay: commit firmly, even lock in. Low learning and low cost of delay: decide on efficiency.
Ask the room
- Will we know much more in six months than we know now?
- What does waiting cost, and who pays it?
- If we stage this, what signpost would tell us to commit?
Watch for Paralysis. Optionality is not a reason to defer everything. Some decisions get better with lock-in.
07
Name owners and review dates
5 min
Each probe and each staged decision gets one owner and one date. Put them in the calendar before the meeting ends.
Ask the room
- Who owns this, by name?
- When do we look, and what will we bring?
Watch for Probes owned by “the team”. Nobody will read the results.

Worksheets

Two sheets to print and fill in

Each one prints on its own sheet of A4. The grey line under each is an example from the worked situation.

Complexity · Probing with options open · Worksheet

The probe portfolio

Sets out three to five probes, each with what would spread it and what would stop it, agreed before any of them starts.

Direction

Start date

Review date

Can stop a probe without a meeting

Probe	Where and who	What we hope to learn	Amplify if…What we would see if it were working	Dampen if…What would make us stop or shrink it	Owner and review

Example Ten-minute quiz every fortnight, never recorded | Secondary 2, every department that agrees | Whether misconceptions show up before the checkpoint | It finds something the checkpoint would not show until the end of the semester | Anyone records the scores, or students start revising for it | Mr Ravi, Term 2, Week 6

geraldajam.com/work/toolkit

Complexity · Probing with options open · Worksheet

The option audit

Names every commitment the plan cannot easily undo, what it gives up, and where it sits on the optionality grid.

Plan

Date

Done by

Commitment	One-way or two-way door	What it gives up	Learning expected soon?High or low	Cost of delayHigh or low, and who pays	What the grid says

geraldajam.com/work/toolkit

Worked example

Three probes in Term 1, Week 10, and a contract kept to one year

One school changes how it assesses

By Term 1, Week 10, the last week of the term, the school had most of a term’s evidence. Green pen was ritual, the book check was producing longer comments written for the checkers, parents had read Developing as a fall, and Mr Ravi’s whole-class feedback had halved his marking. Mrs Lim brought the reading from Term 1, Week 8 to a staff meeting and asked for probes, not a new policy.

A direction in place of the target The plan’s target was 95 per cent of books showing feedback in the agreed format. The meeting rewrote it as a direction: every student, and every parent, can tell what the student knows and what to do next. Nobody could put a percentage on that, which was the point. The HOD English said it was too vague to check. Mrs Lim said the probes would check it.
Three probes, and English agreed to one Whole-class feedback in science and geography, starting from Mr Ravi’s practice. A ten-minute quiz each fortnight, never recorded. And students annotating one piece of work to say what they would change and why. English agreed only to the quiz. The meeting let that stand: a probe a department has been pushed into will be run to fail.
Criteria, written first Whole-class feedback: spread it if marking time falls and the next lesson starts from the common mistakes; shrink it if students stop acting on feedback at all. Quiz: spread it if it finds misconceptions earlier than the checkpoint would; stop it the moment anyone records the scores. Annotation: spread it if students name a specific change and a reason; shrink it if the annotations converge on stock phrases, as green pen had on “I will use more evidence.”
What the probes did In Term 2, Week 2 the quizzes found a Secondary 2 misconception about energy that the checkpoint would not have shown until the end of the semester. That met the amplify criterion. Students liked the quizzes because they did not count, and one class asked for them to count. The criterion said no, and it held. Whole-class feedback met its criterion and spread to two more departments by Term 2, Week 10, changed on the way. Nobody was named to read the annotations, and by Term 2, Week 10 there was no verdict on them either way.
The option audit and the contract The audit flagged one large one-way door. In Term 2, Week 6 the vendor offered a further discount to extend the contract to five years and lock in the five-point scale. Learning was high: parents were contesting the scale, and guidance on reporting for lower secondary had not arrived. Delay was cheap, because a one-year extension kept the system running. The grid said stage it. Mrs Lim took one year at a higher price. The finance committee wanted the saving, and she had to defend the premium to the principal.

What came out of it

Three probes with written criteria: two met them, and one went quiet for want of an owner. One contract kept to a year. The one-year deal mattered sooner than anyone expected. In Term 2, Week 10 the school cut the scale parents see to three points, a change that a five-year lock-in on the five-point scale would have made much harder to argue for. The awkward part was the quiz. Leaders kept asking for the data, and the probe worked only because nobody had it. The criterion that protects it is one line on a sheet, and it has to be defended every term.

Signs

How to tell it is working, and when it has gone hollow

Working

At least one probe is stopped or shrunk on its own criteria, and nobody treats it as a failure.
A probe surprises the people running it.
Someone defends a premium or a delay because it keeps an option open, and can say which option.
Departments change a probe as it spreads, and the criteria still apply.

Gone hollow

Every probe succeeds, and the report was drafted before they started.
The probes are one idea at three sizes.
The criteria are written after the results are in.
Every decision is deferred in the name of keeping options open, and nothing is ever committed.

Variations

Other ways to run it

In 45 minutes

Pick the one complex card that matters most. Write a direction, one probe and its two criteria, and audit the one commitment that worries you most. One probe is not a portfolio, so say so, and add a second at the next meeting.

On your own

A teacher can probe in one classroom. Choose a practice to try for four weeks. Write in your planner what would make you keep it and what would make you drop it, and look on the date you set. Mr Ravi’s whole-class feedback began as a probe like this, though he wrote nothing down.

With students

Ask a class to help write the dampen criteria for a probe that affects them. Ask what students would do to make it look as if it were working. They will tell you, and you can write it in.

What to do with what you made

Put the review dates in the calendar and protect them. When a probe meets its amplify criterion, spread it by invitation before you formalise it, because formalising can stall what was spreading. Before any result is reported upward, run the fourth move on it.

Holding the frame lightly The next move: check the results and the measures before they travel.Provisional commitment The optionality grid in full, and the case for when to lock in.Conditions before outcomes Enabling constraints and safe-to-fail probes, followed through the semester.

Tools

Tools, templates and guides worth having

Published elsewhere, and worth the time.

Routine
Min Specs, from Liberating Structures (Lipmanowicz & McCandless, 2013)
Strips a protocol down to its few must-dos and must-not-dos. The quickest way to run step 2 with a group.
Method
Patton, Developmental Evaluation (2011)
Evaluation built for work that is meant to change as it runs. How to judge probes without waiting for a verdict at the end.
Book
Pascale, Sternin & Sternin, The Power of Positive Deviance (2010)
How to find people already succeeding against the odds and spread what they do. A good source of first probes.
Book
Wiliam, Embedded Formative Assessment (2011)
Practical ways to check understanding without grading it. Useful for designing assessment probes such as the unrecorded quiz.
Method
Haasnoot et al., Dynamic Adaptive Policy Pathways (2013)
Signposts, triggers and switching routes for decisions under deep uncertainty. The long version of step 6.

Reading

Where it comes from

Start with the first on the list.

Snowden, D. J., & Boone, M. E. (2007). A leader’s framework for decision making. Harvard Business Review, 85(11), 68–76.
Probe, sense, respond, and safe-to-fail portfolios.
Plsek, P. E., & Greenhalgh, T. (2001). The challenge of complexity in health care. BMJ, 323, 625–628.
Minimum specifications and attractors, from health care, the nearest neighbour to education.
Patton, M. Q. (2011). Developmental Evaluation. Guilford Press.
Evaluation as learning, not a verdict. The accountability frame for probes.
Sarasvathy, S. D. (2001). Causation and effectuation. Academy of Management Review, 26(2), 243–263.
Affordable loss: commit only what you can afford to lose.
Haasnoot, M., Kwakkel, J. H., Walker, W. E., & ter Maat, J. (2013). Dynamic adaptive policy pathways. Global Environmental Change, 23(2), 485–498.
Signposts, triggers and switching routes.

Singapore