Published
Every config in my backtest database gets a single composite score -- a blend of profit factor, win rate and net result -- so I can sort 12,331 rows and find the best ones without reading each one by hand. Today I ran ORDER BY score DESC LIMIT 1 out of habit, expecting the usual suspects. What came back was a pullback_reclaim config on BSBUSDT: 200 trades, 100% win rate, score 49,908.87. The second-highest score in the entire table, any symbol, any strategy family, is 427.78. The "best" result in my database is 116x higher than the runner-up. That gap is the tell.
What a score of 49,908 actually is
Profit factor is gross wins divided by gross losses. When a 200-trade sample happens to land zero meaningfully-sized losing trades, that denominator collapses toward zero and the ratio wants to go to infinity. My pipeline catches that and caps profit factor at a sentinel value of 999 instead of returning inf -- a reasonable guard. What isn't reasonable is that the score formula takes that capped 999 and runs with it, producing a composite number two full orders of magnitude above anything a real result can produce. The guard rail stopped a divide-by-zero crash. It didn't stop the number from becoming meaningless.
| Config | Trades | Win rate | PF | Net | Score |
|---|---|---|---|---|---|
| BSBUSDT pullback_reclaim (#1 overall) | 200 | 100% | 999* | +$47.88 | 49,908.87 |
| BILLUSDT micro_momentum (#2 overall) | 200 | 99.5% | 9.28 | +$55.81 | 427.78 |
| BSBUSDT pullback_reclaim, same family, full sample | 2,000 | 95.0% | 1.96 | +$471.18 | 53.79 |
*Sentinel cap, not a real profit factor -- the pipeline substitutes 999 whenever gross losses round down to near-zero on a small sample. Figures pulled directly from backtest_results, ids 2, 35 and 102.
The same coin, scored 861x lower, making 10x more
BSBUSDT pullback_reclaim doesn't just exist as a 200-trade fluke -- it's one of the more heavily tested symbol/family combinations in the whole database, run across sample sizes from 200 up to the standard 2,000-trade window dozens of times. Among those full-size runs, the best one nets +$471.18 on a believable 95% win rate and a profit factor of 1.96 -- a real number, not a cap. Its score: 53.79. That's 861x lower than the 200-trade fluke, for a config that made roughly 10x more money. If I had trusted the score field alone, I'd have picked the worse trade every time.
The part that actually held
Here's what didn't break: the shortlist of 50 configs my pipeline flags is_top_candidate -- the ones that actually get walked forward into deeper validation -- all require a minimum of 808 trades. The 200-trade fluke never got near that gate; it isn't on the shortlist, and never was. A second, completely independent filter caught what the score alone couldn't. That's the system working as intended one layer down. But the score field itself -- the one a human (or a careless query) reads first -- has sat in the database this whole time reporting that a $47.88 trade is the single best result out of 12,331, by a factor of 116.
What I actually check now
Score is a sort key, not a verdict. Before I trust any number near the top of a sorted list, I now check trade count first -- anything under roughly 800 trades in this pipeline gets read as "too early to tell," full stop, regardless of what the headline number says. I also stopped trusting profit factor above single digits on sight; a PF of 9 is suspicious, a PF of 999 is a bug report. The same logic applies outside my own database: if you're evaluating any backtest -- yours or someone else's -- and the sample size isn't stated next to the headline number, run the one you do have through risk of ruin before you size a real position off it. A great score on a small sample and a mediocre score on a large one are not the same kind of claim, even when the small one reads higher.
Method: queried directly against our own backtest pipeline's SQLite store (backtest_results, 12,331 rows across all symbols/families; is_top_candidate shortlist, 50 rows). Scores, trade counts and profit factors are the pipeline's own stored output, not recomputed by hand. Research findings on paper/simulated strategies, not a live trading record. Not financial advice.
- Risk of ruin calculator β how bust probability scales with a thin, uneven edge
- Required win rate calculator β the win rate your payoff ratio actually needs
- Win rate calculator β track your own numbers trade by trade
- Break-even win rate calculator β where fees push the line