Our Polymarket news bot backtest showed t = 10. It was look-ahead bias

Our Polymarket news bot backtest showed t = 10. It was look-ahead bias

●7 ●21
calendar_today ago • schedule2 min read
— Originally published at orcalayer.com

We tested four ways to trade news on Polymarket in 2026, on non-sports event markets: a 5-second bot, following a move a minute late, copying wallets that were early on past news, and copying fresh wallets that agree. None made money. The full results are on our blog, and every price shock and detector trigger is on GitHub with a standard-library script that recomputes the tables.

This post is about the one test that did look profitable, because the reason is a bug that is very easy to write into any event-driven backtest.

The bot that made +$2.63 per $10

The idea: Polymarket prices react to headlines, and a bot that sees the first trade of a move can buy within seconds. Our first backtest bought $10 in the direction of a news move 5 seconds after its first trade, paid the taker fee, and valued the position at the price 2 to 15 minutes later.

Result: +$2.63 per $10, t-statistic 10.1, on 510 events. Held to resolution: +$3.22.

The bug: the event definition knew the future

Here is roughly how a "news jump" was defined:

# BUGGY: the event is defined by where the price went later
def is_news_jump(trades, t0):
    p0 = trades.price_at(t0)
    later = trades.price_between(t0 + 120, t0 + 900)   # 2 to 15 minutes later
    return later is not None and abs(later - p0) >= 0.10

The move only counts if the price is still 10 cents away minutes later. So the test only ever buys moves that are already known to have continued. In that version, 88% of the "jumps" had moved further by the time they were valued. Not because news moves continue, but because those were the only moves the filter let through.

A live bot never sees the price 15 minutes later when it decides. It sees the first trade.

The fix: only use what is known at the decision time

# HONEST (simplified): the event is defined only by data up to t0
def is_price_shock(trades, t0):
    p0 = trades.price_at(t0)
    ref = trades.median_price(t0 - 300, t0)    # median of the previous 5 minutes
    return ref is not None and abs(p0 - ref) >= 0.05

Same markets, same 30 days (September 14 to October 14, 2026), same entry 5 seconds after the shock, same fee. 4,303 shocks qualified; 1,218 had a trade 5 seconds later and enough later trades to value the position:

Version $ per $10 at the price 2-15 min later Held to resolution Move continued
Event picked by the later price +2.63 (t 10.1) +3.22 88%
Event picked by what was known +0.18 (t 1.5), before exit costs -0.34 37%

+0.18 before the cost of getting out is not a strategy. The whole edge was the filter.

A quick test for this bug

Every column that feeds the event filter or the selection must be computable at the decision timestamp. A cheap way to enforce it is to build the filter from a view of the data that physically stops at t0:

def known_at(trades, t0):
    return trades[trades.ts <= t0]       # nothing after the decision

assert is_price_shock(known_at(trades, t0), t0) == is_price_shock(trades, t0)

If your filter gives a different answer when you cut off the future, it is using the future. The same trap shows up as "goals" detected by a later score, "breakouts" confirmed by a later close, or wallets labelled "smart" on their full history.

What news trading returns without the bug

The other three tests used rules fixed before looking at results:

  • Follow a move a minute late. A detector on past data only, moves of 10 cents or more, buy the rising side over the next 3 minutes with 2 cents of slippage and the fee. 1,713 moves, May to October 2026: sold 1 to 24 hours later, -13% to -23% of the stake, in both test periods.
  • Copy wallets that were early on past news. No better than a control group of similarly active wallets; no difference is significant. Wallets that bought in the first minutes of earlier moves kept doing it (77% on the rising side vs 44% for the control), but copying them still returned about the same as copying anyone: by the time a copy fills, their price is gone.
  • Copy fresh wallets that agree. 86,228 new wallets: -7.2% per dollar after fees with one wallet, -4.9% with six agreeing.

Limits

One half-year of non-sports event markets; nothing faster than 5 seconds; no sports, no market making around news. Plain t-statistics, and trades on one event are not independent.


Originally published on orcalayer.com. OrcaLayer is an independent Polymarket analytics provider, not affiliated with Polymarket.

🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

We pre-registered a Polymarket insider score. It passed, and we still didn't ship it

orcalayer - Oct 22

Backtesting a Polymarket copy trading bot without fooling yourself

orcalayer - Oct 10

AAIF Hits 10 Projects. Its CTO on What to Adopt Now and the Security Gaps Left Open

Tom Smithverified - Oct 23

Backtesting live football on Polymarket: 6 traps in a recorded order book

orcalayer - Oct 14

Dashboard Operasional Armada Rental Mobil dengan Python + FastAPI

Masbadar - Mar 12
chevron_left
424 Points • 28 Badges
Ukraine • orcalayer.com
10Posts
6Comments
1Connections
OrcaLayer is a read-only analytics layer over Polymarket, the prediction market running on Polygon. ... Show more

Related Jobs

View all jobs →

Commenters (This Week)

5 comments
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!