Every rule of thumb has an opposite number

The situation

Decisions in a school system are made faster than they can be investigated. A platform team has a sprint. A steering group has an hour. So people reach for defaults, rules that say what to assume before the evidence is in: build only what is needed now, take the simplest explanation, ask who benefits, pilot it because the door swings both ways.

Most of the time the defaults are right. They are compressed experience, which is why they carry weight in a room. The trouble is that they come in pairs. The engineer says build only what is needed. The operations lead says leave a margin. The data protection officer asks for precaution and the product owner says it is a two-way door. Each has picked the rule that suits their job, and the meeting turns into a contest between two sayings, won by whoever quotes theirs with more conviction.

I have won that contest with a well-placed razor. It felt like rigour at the time.

The question

What if a rule of thumb were read the way a bias is? A bias is a rule that earns its keep most of the time and fails in a describable set of cases. A rule people adopt on purpose is the same kind of thing with a better reputation. Occam is right until the data leave residue, and Hanlon until the error survives being pointed out.

The failure is built into the rule. The zebra rule says that when you hear hoofbeats you should think of horses, and it is right because common things are common. It fails for the same reason, on the day it is a zebra. The precautionary principle overlooks the cost of waiting because it only prices the cost of acting. So the useful unit is bigger than a rule. It is the rule, its opposite number and the condition that says which one applies.

Set sixteen of those collisions side by side and seven conditions settle almost all of them: evidence, base rates, interests, reversibility, error costs, scale and time horizon. That is stated so that it could be wrong. One collision that no condition settles would count against it.

The move

Say the rule out loud. A default that is spoken can be questioned. One that stays unspoken simply decides.

Then find its counterweight. If nobody can say which rule pulls the other way, the room is holding a slogan.

Check the condition that decides between them. Each condition comes with a question. For precaution against the two-way door, ask what exactly would undo this, what it would cost and which part could never be undone. A pilot that sends student essays to a vendor’s model splits cleanly. Teachers seeing drafts they can discard is reversible, so go. Essays kept by the vendor to train its model are not, so that part is settled by contract before anything runs.

For building lean against leaving a margin, ask whether being short would be an inconvenience or a failure, and who would pay. A marking workflow nobody has asked for can wait. A submission service that falls over at ten on deadline night cannot, because the cost lands on students at a time they did not choose.

For assuming error against asking who benefits, ask whether honest mistakes would have gone both ways. A dashboard that counts inactive accounts as reached, thirty-second visits as sessions and duplicate logins twice has made three defensible choices. All three inflate the figure the contract renewal depends on. Count which way the errors point before deciding on good faith.

Say in advance what would make you switch. A rule without an exit is a belief. This is the step most often skipped, and the cheapest.

And apply every rule symmetrically. A razor can be aimed: used on claims you dislike and waived for claims you hold. Aimed like that, it is a bias with a Latin name.

What it produced

The first was a fourth field guide: 103 razors, laws and principles in six families, and sixteen collisions between them. Every entry says where its rule misleads and names the rule that pulls against it. Writing that section 103 times showed that the failure can nearly always be read off the rule itself.

The second was a one-page instrument: the seven conditions, asked in order before a rule is applied. Evidence and base rates come first, because they are about what is true. The rest are about what to do. Skip the first two and argue from error costs alone, and the precautionary principle becomes a veto while the two-way door becomes an excuse.

The third was a habit for platform decisions. Split every decision into the parts that can be undone and the parts that cannot. Move fast on the first. Move slowly on the second, and check again as schools come on board, because doors close as dependants accumulate.

The fourth was some humility about famous names. Of the 103 rules, twenty-three have been tested. Fifty-two are working practice or wit. Several of the best-known laws hold far more narrowly than they are quoted, so each entry carries a tag for what stands behind it, and a famous name gets the standing it has earned.