Journal / How to pack an experiment folder for someone who was not in the room
15 June 2026 · Aisha Rahman
How to pack an experiment folder for someone who was not in the room
If an outside reader needs a week of meetings to reconstruct the hypothesis, the folder was not ready.
An outside reader should be able to write a fair readout from a folder, not from a week of hallway conversations. Product squads in Johor and KL already have the pieces. They are rarely in one place.
The smallest honest pack
Put these in one folder with dates in the filenames: the hypothesis as written at launch, the variant descriptions (screenshots count), the assignment rule, the start and stop dates, the sample sizes by variant, the primary metric definition, the guardrail definitions, and the results table for the same window.
Add a short note on anything that disturbed traffic: a store outage, a festive campaign, a bug that sent some users to both variants. The note can be ugly. Missing it is worse than ugliness.
What we should not have to reverse-engineer
Who was excluded, and why. Whether staff accounts were filtered. Whether the experiment ran on web, Android, iOS, or a subset. Whether anyone peeked, even “just to check tracking.” Whether a later copy change landed on all variants.
If the product manager, the engineer who built assignment, and the analyst who pulled the table would each tell a different story, write the disagreement down. We would rather read an argument than invent a clean history.
A pack that is not ready
Screenshots of a dashboard with no metric definitions, a hypothesis written after seeing the result, or a results table whose window does not match the flag dates: those packs go back. The delay is shorter than writing a readout that has to be withdrawn.
If you want a briefing date in the same week, send the pack before you ask for the date. We read in the order complete packs arrive, not in the order calendar holds are requested.
Request a readout if this is the conversation your squad is already having.