13 Strategies, 0 Survivors: A Pre-Registered Kill Protocol in Python (DSR, PBO, CSCV)

Leader ●1 ●11 ●30
calendar_today ago • schedule3 min read
— Originally published at guetaquant.com

We pre-registered 13 simple strategies, ran them through four fixed gates, and published
the result even when it was unflattering: 13 → 0. Then we repeated it in four more
markets. 65 pre-registered tests, zero survivors in all five markets.

This is the protocol, the numbers after our published correction, and the part most
backtests skip: deciding the kill rule before you see the equity curve.

Educational content: methodology only, no signals, no return claims.

By Mahdi Goodarzi (g.dev/mahdigoodarzi), Gueta Quant —
Spanish source guide: validar un backtest sin autoengañarse.


1. The four gates, fixed before execution

Every candidate had to pass all four, in order:

Gate Rule
G1 Out-of-sample net return > 0 and Sharpe > 0
G2 Walk-forward cumulative result > 0
G3 Deflated Sharpe Ratio (DSR) ≥ 0.95, deflation family N = 14
G4 Probability of Backtest Overfitting (PBO) < 0.50 via CSCV with S = 16

The thresholds were written down first. That is the whole trick: a rule you can move
afterwards is not a rule.

2. What actually survived the funnel

Market Funnel Best PBO
EUR/USD 13 → 8 → 7 → 0 → 0 0.5639
Gold 13 → 9 → 8 → 0 → 0 0.1737
S&P 500 13 → 10 → 8 → 0 → 0 0.5695
Bitcoin 13 → 5 → 4 → 0 → 0 0.2139
Ethereum 13 → 7 → 4 → 0 → 0 0.1557

These are the corrected counts from our published addendum (2026-08-16), which fixed an
execution-model error and a PBO convention. The zero survived the correction.

The most instructive number is the one that almost passed: the best DSR on gold was
0.8818 (SMA Cross) after the addendum — previously reported as 0.9444. Both are below
0.95, so the strategy was rejected either way. But note what happened: a correction moved
the "best" candidate substantially and the decision did not move, because the threshold
was fixed in advance. Had we chosen 0.94 after seeing 0.9444, we would have shipped a
strategy on a number we later revised to 0.8818.

3. The deflation is doing the work

SMA Cross, Momentum, Bollinger Reversion and Ichimoku passed G1 in all five markets.
They looked consistent. They still failed G3/G4.

That is the bias most backtests never price in: testing many variants and keeping the
winner. Deflation by the number of trials absorbs the apparent edge, because a search over
14 variants produces a maximum that is expected to be large even when nothing is real.

# Sketch of the gate logic — thresholds are arguments, not constants you edit later.
def passes_gates(trial, *, dsr, pbo, n_family=14, dsr_min=0.95, pbo_max=0.50):
    g1 = trial.oos_return > 0 and trial.oos_sharpe > 0
    g2 = trial.wfo_cumulative > 0
    g3 = dsr >= dsr_min          # deflated by the family of trials
    g4 = pbo < pbo_max
    return all([g1, g2, g3, g4])

The important line is not dsr >= dsr_min. It is that dsr_min and n_family were
declared in the pre-registration file before the run.

4. Why report the zero?

Three reasons, all practical:

  1. A published zero is auditable. Anyone can rerun the funnel. A published winner
    invites a search for the parameter that makes it look better.
  2. It calibrates expectations. If 13 textbook strategies die on four gates in five
    markets, a strategy that sails through deserves suspicion, not celebration.
  3. It kills the survivorship story. Our own losing variants stay in the record.

Our research notes are public, including the addendum that corrected our own numbers:
guetaquant-research.

5. Checklist you can copy

  1. Write the gates and thresholds before running anything.
  2. Fix the trial family size and record every variant, including the ugly ones.
  3. Report the funnel counts, not just the winner.
  4. Publish corrections as addenda, with the old and new numbers side by side.
  5. Treat "almost passed" as a rejection. It is the most seductive number in the table.

Study material under Colombia SFC Decreto 2555 de 2010 — financial-engineering education,
not investment advice. No signals, no managed accounts. Related:
walk-forward and the Deflated Sharpe ·
Monte Carlo permutation testing.

AI assistance was used for drafting and editing; the numbers, gates and corrections come
from the published research files.

🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

Dashboard Operasional Armada Rental Mobil dengan Python + FastAPI

Masbadar - Mar 12

The Sovereign Vault — A Comprehensive Guide to Protocol-Driven AI

Ken W. Algerverified - Jun 4

Python for commerce students, from a former bio kid

rahuldk1105 - Oct 30

Forecast Kebutuhan Bahan & Produksi Konveksi dengan Python (Praktis + Template)

Masbadar - Mar 8

We pre-registered a Polymarket insider score. It passed, and we still didn't ship it

orcalayer - Oct 22
chevron_left
1.6k Points • 42 Badges
Bogotá, Colombia • guetaquant.com
15Posts
2Comments
13Connections
Founder & Lead Quantitative Engineer at Gueta Quant (https://guetaquant.com).

I design open-source... Show more

Related Jobs

View all jobs →

Commenters (This Week)

2 comments
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!