Forex Robot Backtest Scams: How Perfect Backtests Are Made and How to See Through Them

A forex robot's backtest is the easiest thing in trading to make look perfect. Smooth equity curves, 90 percent win rates and tiny drawdowns can all be produced from strategies that lose money live. Some sellers do it deliberately. Others do it by accident. Either way, the buyer pays. Here is how it is done and how to see through it.

Five ways backtests lie

1. Curve fitting

Every adjustable setting, such as moving average length, stop distance, session hours or filter levels, gives a strategy more freedom to match the past. Try enough combinations and one will fit history beautifully by chance. Like flipping coins until someone throws ten heads, the result says nothing about the future.

Tell-tale sign: small changes to a setting produce large changes in results.

2. Cherry-picked periods

Showing only the years when the strategy worked. A trend-following robot tested only on strong trends, or a range robot tested only on quiet markets, will look brilliant.

Tell-tale sign: a backtest that starts or ends at an oddly specific date.

3. Missing costs

Spreads, commissions, swaps and slippage are left out. For a robot trading many times a day, costs can be larger than the entire edge.

Tell-tale sign: very short targets, high trade frequency and no mention of spreads.

4. Impossible fills

The test assumes every order fills at the exact price, even during news spikes, gaps and the daily rollover when spreads widen.

5. Hidden look-ahead

The simulation uses information that would not have been known at the time: a swing high before the candles that confirm it, a daily high before the day ends, or trade selection that depends on what happens later.

This one catches honest developers too. While building the TebotechSignals engines, a research script of ours showed a steady profit across 32 currency pairs for almost four years. A candle-by-candle replay through the live engine's code showed the profit was an illusion: the script had quietly used knowledge of the future to skip losing trades. We describe the mistake in The Hybrid Trader.

The tests that reveal a real edge

1. Out-of-sample testing. Was the robot designed on one period and then tested once on a later period it never saw?
2. Walk-forward results. Repeated development and test windows moving through time.
3. Many pairs and regimes. Does it work on markets it was not tuned for?
4. Costs included. Realistic spreads and commissions on every trade.
5. Enough trades. At least about 100, preferably many more.
6. A live record. Weeks or months of live or demo trading compared with what the backtest predicted for the same period.

If a seller cannot provide these, the backtest is marketing.

The martingale disguise

Some backtests look perfect because the robot never closes a losing trade at a loss: it adds positions until the market comes back. The closed-trade curve rises steadily while open losses build in the background. One long trend eventually exceeds the account. Look at drawdown including open positions and at whether trades are added at worse prices. Our guide to no-risk forex Expert Advisors explains this pattern.

Questions to ask before buying

• Which period was used to design the robot, and which was kept aside for testing?
• What spread and commission were assumed?
• What was the maximum drawdown including open trades?
• How did it perform live over the last six months compared with the backtest?

Bottom line

A beautiful backtest is not evidence of a working robot. It is evidence that someone can make a beautiful backtest. Trust out-of-sample tests, costs, enough trades and, above all, a complete live record. For the full list of deceptive techniques, read Signal or Scam?.