Prompt LibraryData
Experiment Metric Sanity Checker Prompt
Sanity-check experiment metrics before trusting results.
The prompt
Before trusting the experiment results below, run these sanity checks:
1. Sample ratio mismatch (is the split actually 50/50?).
2. Peeking: was the test stopped at a peak? (check the day-by-day p-value).
3. Bot/traffic contamination.
4. Metric definition drift (did the metric formula change mid-test?).
5. Simpson's paradox in any segment.
Flag any check that fails and what it means for the result.
Results:
{{results}}Replace the {{fields}} with your own context and tighten the rules to match your domain.


