Guardrails without burying the question you actually asked

Guardrails are how a squad pre-agrees what would make a “win” unacceptable. They fail when the memo treats them as a second primary metric, or when they are so many that nothing can veto anything.

One question, a short list of vetoes

The hypothesis is allowed to be narrow: will this new onboarding step raise the share of users who complete identity verification in seven days? Guardrails then name the harms you will not trade for that rise — drop in successful logins, spike in verification retries, collapse in a region you cannot ignore.

Four guardrails is a list a briefing can hold. Fourteen is a way of never deciding. If a metric would not actually stop the ship, it is not a guardrail. Put it in monitoring after launch, or drop it from the experiment record.

When a guardrail moves

A guardrail that moves against the feature should interrupt the celebration, not appear after three pages of primary-metric charts. Write the size, the count of users underneath it, and whether the movement is concentrated in a device class or a week.

A guardrail that does not move deserves a calm sentence, not a victory lap. The point of the metric was to watch for harm, not to collect extra green ticks.

Do not let side charts elect a new hypothesis

Briefings go wrong when someone finds a lively chart in a slice and proposes that this was the real question all along. If you want that question, design the next experiment. The present readout has to answer the present question, with the vetoes you named, or admit that the design cannot answer it.

We keep a paper copy of the original metric list in the room for this reason. It is easier to point at a page than to argue with a memory of what “we all knew” the experiment was for.

Request a readout if this is the conversation your squad is already having.