AI in trading: what machine learning does well, and where it quietly fails

Search interest in AI trading tools has exploded, and so have the promises. Some of it is real. But the published research is clear on one point: the most dangerous thing a machine-learning model can do in finance is look brilliant in a backtest.

Where machine learning genuinely helps

• Reading at scale. Language models can read every corporate announcement, filing and headline and surface the ones that matter. Structured news and corporate announcement APIs are the natural input.
• Classifying regimes. Is the session trending, rotating or range-bound? Classification is far more robust than point prediction.
• Detecting anomalies. Spotting bad ticks, gaps and outliers in data before they reach a strategy.
• Explaining complex indicators. Turning a market profile or an order-flow imbalance into a plain-language read of the session.

That last use is where AI trading tools are most practical today. The partner MirraCharts suite, for example, pairs AI Market Profile and AI Order Flow indicators with a guidance panel that describes the session in words — see the NinjaTrader indicators. The value is interpretation, not prophecy.

Where it quietly fails: overfitting

A flexible model given enough attempts will find patterns in pure noise. The more parameters you tune, the better the backtest looks — and the worse the live result tends to be.

This is not a theoretical worry. Researchers have shown that trying many strategy variations on the same history makes a high backtested Sharpe ratio almost meaningless without correcting for the number of trials, and that a large share of published return “factors” fail stricter statistical hurdles.

A checklist before you trust any AI model

1. Hold out time, not rows. Train on the past, test on a later period you never looked at.
2. Walk forward. Retrain and retest across several consecutive windows.
3. Count your trials. Every variation you tried is a lottery ticket; correct for them.
4. Hunt for leakage. Any feature computed with future information — including unadjusted corporate actions — invalidates the result.
5. Demand enough trades. The arithmetic in our backtesting guide applies to models too.
6. Start with clean data. A model trained on snapshot data learns the gaps; see why free data costs more.

What AI can do for each kind of trader

What trustworthy AI output looks like

Whether you build a model or buy a tool, good AI output in trading shares four traits. It shows its inputs — which data, which session, which timeframe. It explains rather than commands, so you can disagree with it. It is timestamped, because a read of the market goes stale in minutes. And it admits uncertainty: a tool that is never unsure is not measuring anything real.

The data an AI model actually needs

• Tick-accurate history so features like volume at price and order-flow imbalance mean what they say.
• Consistent adjustment across every resolution, or the model learns that bonus issues are crashes.
• Point-in-time context — index membership, lot sizes and expiries as they were, not as they are.
• Text with timestamps — announcements and news aligned to the tick they arrived, for any language-model work.

Those four are exactly what a recorded archive provides and a scraped one does not. The Pix API suite exposes realtime and historical ticks, Greeks, corporate data and announcements from the same source, which keeps features and labels in step.

Research referenced: White, H. (2000), “A Reality Check for Data Snooping”, Econometrica. Bailey, D., Borwein, J., López de Prado, M. and Zhu, Q. (2014), “Pseudo-Mathematics and Financial Charlatanism”, Notices of the AMS. Harvey, C., Liu, Y. and Zhu, H. (2016), “… and the Cross-Section of Expected Returns”, Review of Financial Studies.

Questions traders ask