The 75-Bet Rule — Why Signal Won't Promote Until Forward Sample Is Real
The 75-Bet Rule — Why Signal Won't Promote Until Forward Sample Is Real
> Signal Syndicate Research Library · Model Validation · Educational intelligence · Not betting advice
A backtest can look brilliant on eighteen bets. Forward testing on three bets can look brilliant too. Neither is enough to ship a lane to users. What failing this bar actually looks like is documented in the P43 case study.
Signal uses a hard rule: 75 out-of-sample (OOS) shadow bets before promotion is even on the table. This page explains the gate in plain English — and walks through a live example from our P43 summer UNDER shadow lane.
The idea in one sentence
Out-of-sample means bets graded on games and slates the model never used while tuning. We log them in shadow mode (`do_not_apply=true`) — real metrics, zero user exposure — until the sample is large enough to trust.
What happened on P43 (Phase 72)
| Metric | Value (historical estimate) |
|---|---|
| OOS shadow bets logged | **3** |
| Minimum for promotion review | **75** |
| Shortfall | **72 bets** |
| Shadow ROI on those 3 | ~27% (illustrative — tiny n) |
| Court verdict | **MONITOR only** |
| User-facing picks | **None** (`do_not_apply=true`) |
Three winning-looking shadow rows do not override the gate. The lane stays internal until forward sample grows or the hypothesis fails on expanded holdout.
Why 75 — not 10, not 500?
There is no magic number that guarantees profit. The gate is a variance control:
- Below ~20 OOS bets: ROI swings are mostly noise — a hot week and a cold week look like "systems."
- At 75: We can start separating luck from structure without pretending certainty.
- Above 75: Promotion still requires multi-AI courtroom, founder sign-off, and honest replication — not automatic ship.
Think of it as the difference between a promising lab result and a clinical trial. We publish both stages; we only productize the second.
What this means if you are learning to bet seriously
1. Ask for forward sample size, not just backtest ROI.
2. Treat small-n PASS as hypothesis — our shadow labels say so explicitly.
3. Use Ask Signal: "What is the 75-bet promotion gate?" — this page is the grounded answer.
What we did next
- Continued shadow logging at zero user exposure
- No production JSON policy change
- Linked transparency reports (P43 not promoted, shadow mode explainer) in the Research Library
Research only · estimates only · not betting advice · past results do not predict future performance.