Paper Trading Before Real Money: How We Validate Trading Bots
We maintain two algorithmic trading systems. One swing-trades equities on daily signals; the other trades event contracts on sports predictions, fed by a machine-learning model with hundreds of engineered features. Both run automatically every market day. Combined real dollars at risk so far: zero.
That's not cold feet. That's the methodology, and it's the part most "I built a trading bot" write-ups skip.
Backtests lie to you politely
Everyone backtests. The problem is that a backtest is a simulation of the market and a simulation of your own execution, and both simulations flatter you. Look-ahead bias sneaks in through innocent-looking joins. Fills happen at prices you wouldn't have gotten. The dataset quietly excludes the delisted losers. You tune parameters until the curve looks good, and what you've actually done is memorize the past.
A backtest tells you a strategy could have worked. It cannot tell you that your system — the code, the data feed, the scheduler, the order logic, the error handling — does work, unattended, on data it has never seen. Only running it forward does that.
The paper-trading harness
So both systems trade every day with fake money and real everything-else:
- A scheduled job fires on market days, pulls live data, runs the signal logic, and records the trades it would have placed — entries, exits, sizing — to a persistent ledger. Same code path that would touch a broker, minus the final call.
- The runs are monitored for silence. A cron that dies quietly is indistinguishable from a strategy with no signals unless something is counting heartbeats — our watchdog opens an issue if a daily run goes missing.
- The suite around it is unsentimental. Hundreds of tests cover the unglamorous machinery: date handling around holidays, partial data, API failures, duplicate signals, restart recovery. The strategy is maybe five percent of the code. The other ninety-five percent is what actually breaks.
Months of forward paper results now sit in version control — a ledger nobody can retroactively tune. It has already killed one strategy variant that backtested beautifully and forward-tested like a coin flip, which is exactly the kind of thing you want to learn for free.
What this has to do with your business
You probably don't run a trading bot. But if you're automating anything with real consequences — invoices, payroll adjustments, customer messages, inventory orders — you face the same gap between "the demo worked" and "this runs unattended with my money on the line."
The trading world's discipline transfers directly:
- Run the real system against real inputs with the consequences stubbed out. Not a test environment — the production pipeline, with the last dangerous step replaced by a ledger entry.
- Log what it would have done, immutably. Review that ledger the way you'd review an employee's first month.
- Alarm on silence, not just on errors. The automation that stops running is more dangerous than the one that crashes loudly.
- Only then connect the wire — and even then, with limits, the way a new trader gets a small book.
"Move fast" is fine advice for software whose failures are free. Automation's failures are usually somebody's money. The boring path — simulate, log, watch, then commit — isn't slower in the long run. It's just where the speed comes from later: once a system has months of clean unattended dry-runs behind it, you can trust it with the real thing and stop thinking about it.
That last part is the entire point of automation.