What belongs in a feature experiment readout, and what can wait

A readout is not a gallery of every chart the experiment produced. It is a document a product manager can carry into a launch review and still recognise as the same experiment the squad designed.

Start with the question that assigned traffic

The first paragraph should repeat the hypothesis in the words used when the flag was turned on. If those words are embarrassing, that is useful information. Rewriting the hypothesis to match a prettier metric belongs in a later experiment, not in the readout of this one.

Name who qualified. Feature experiments in Malaysian apps often exclude staff accounts, a corporate cohort, or users who never finished onboarding. If the launch review later talks about “all customers,” the readout should have already said who was never in the table.

Primary result, then the size of the thing

Say what happened to the primary metric, in both relative and absolute terms, for the window you actually ran. A two-point rise on a rate that was already high is a different story from a two-point rise on a rare event. Thresholds matter; they are not the whole sentence.

If the squad peeked, say so next to the result, not in a footnote. Peeking changes how much weight the number can carry. Hiding it does not restore the weight.

Guardrails as vetoes, not as extra winners

A guardrail that did not move can be a short sentence. A guardrail that moved against the feature needs space, because it may cancel a ship even when the primary metric looks kind. What does not belong is a tour of twenty slices that were never listed in the design.

Declared segments — Android versus iOS, new versus returning, Peninsular versus East Malaysia if you truly planned that cut — can sit after the guardrails. Undeclared slices can be listed as questions for a follow-up, with counts so nobody mistakes a thin slice for a national story.

What can wait for the appendix

Funnel screenshots that do not change the decision, secondary metrics that were never going to veto a ship, and a history of every dashboard the analyst opened while waiting for sample: those can wait. The people in the briefing have seventy-five minutes. Spend them on the decision.

End with one of three words the squad already agreed to use: ship, iterate, or stop. Then the reasons. Then the owner of the next step. If the memo cannot end that way, it is still a notes file, not a readout.

Request a readout if this is the conversation your squad is already having.