ML in trading: why it doesn't predict prices

Machine Learning in Algorithmic Trading: Real Applications and Common Mistakes

Machine learning (ML) in trading doesn’t predict stock prices directly, but excels at solving specific, narrow tasks: identifying market regimes, filtering trading signals, optimizing order execution, and analyzing order book microstructure. In Russia, solo algorithmic traders handle the entire pipeline—from data preparation to production deployment—unlike hedge fund teams where roles are divided. Most ML-based strategies fail due to poor data quality and flawed testing.

The Solo Algo Trader’s Role

Today’s lone algo trader acts like a full team of specialists. They manage:

• Data sourcing and cleaning;
• Feature engineering;
• Model building;
• Hypothesis testing;
• Strategy assembly;
• Risk calculation;
• Production deployment.

This demands home GPU-equipped hardware and AI tools. Unlike funds with specialized roles, one person carries it all. A survey at a meetup revealed: 70% have experimented with ML, but only 30% run profitable strategies with real money. The core issue? Lack of meaningful content—discussions are full of vague hints, and everyone reinvents the wheel, repeating the same mistakes.

Why Direct Price Prediction Doesn’t Work

The myth of neural nets predicting tomorrow’s Sberbank price collapses under real-world testing. Regression on historical prices yields noise. In high-frequency trading (HFT), ML is limited—models introduce delays critical at nanosecond precision.

Signal generation via ML often fails. Instead, ML shines in:

• Market regime detection: turning off trend-following strategies during sideways markets;
• Signal filtering: eliminating losing trades from base strategies;
• Execution optimization: minimizing slippage and fees;
• Order book microstructure analysis: spotting patterns in order flow invisible to the naked eye.

These applications extract alpha without the illusion of prediction.

Classic Testing Pitfalls

Most strategies break during backtesting. Key traps:

• Garbage in = garbage out: 80% of time spent finding and cleaning data. Missing a candle day or poorly stitched futures create false patterns;
• Looking into the future: flawed train-test splits let models see future data (see Marcus de Prado). Backtest looks perfect; live results trigger margin calls;
• Forecast ≠ PnL: model guesses direction right 60% of the time, but slippage, fees, and latency eat all profits.

The gap vs. Western funds isn’t in ideas—it’s infrastructure. Non-stationarity and overfitting are universal challenges.

Validating Ideas Before Coding

Before writing Python code, ask yourself five questions:

• Is the task well-defined? (Not ‘add a neural net,’ but ‘reduce execution costs’);
• Do you have honest data? (Clean, no future leaks, real-time accessible);
• Can you test it including fees and latency?
• Does the forecast translate into action? (Clear path from model to trade);
• Is there an economic impact? (Profit after all costs).

A negative answer to any means it’s too early to deploy ML.

What Matters Most

• ML in trading solves focused problems—not price prediction;
• 80% of success lies in data—cleaning and validation are critical;
• Avoid future leakage in backtests;
• Factor in PnL with slippage and fees;
• Validate ideas using the five-question checklist before coding.

ML isn’t a magic button—it’s a tool for precise needs. Closed communities lead to repeated errors; sharing insights accelerates progress.

— Editorial Team