DiffQuant: Sharpe optimization in a differentiable simulator

DiffQuant: Differentiable Trading Simulator for Direct Sharpe Optimization

In ML-based algorithmic trading, models are traditionally trained to predict returns or price direction using MSE or cross-entropy, while performance is evaluated via Sharpe ratio accounting for costs. This creates a disconnect: accurate micro-movement forecasts don't guarantee out-of-sample profitability. DiffQuant eliminates proxy objectives by building a single differentiable graph from market features to final PnL and Sharpe. Walk-forward testing delivers a Sharpe of +1.73 and +8.22% return after commissions.

Challenges with Traditional Proxy Objectives

The standard pipeline separates prediction from trading:

• Model minimizes MSE on return_{t+1}.
• Position sized via heuristics.
• Backtester applies commissions after the fact.

This leads to:

• Mismatch in error spaces: Accuracy on noisy fluctuations causes overtrading and bleed from costs.
• No gradient on position size: Model doesn't learn entry aggressiveness.
• Ignoring costs during training: Slippage and commissions sit outside the computation graph.

DiffQuant integrates position sizing, PnL simulation, and metrics into a PyTorch graph, passing gradients end-to-end.

Differentiable Trading Simulator

The simulator computes PnL as tensor operations over horizon t ∈ [0, H-1]:

$$r_t = \frac{c_t - c_{t-1}}{|c_{t-1}| + \varepsilon}$$

$$gross_t = p_{t-1} \cdot r_t$$

$$cost_t = smooth\_abs(\Delta p_t) \cdot (commission + slippage)$$

$$pnl_t = gross_t - cost_t$$

Key innovation—smooth_abs for differentiability:

$$smooth\_abs(x) = \sqrt{x^2 + \varepsilon}, \quad \varepsilon = 10^{-6}$$

This ensures C^∞ smoothness near zero, where the policy starts flat, avoiding subgradient discontinuities.

Gradients from -Sharpe flow through PnL → positions → model, training it to account for costs end-to-end.

Policy Architecture with iTransformer

Backbone is iTransformer (ICLR 2024): an inverted transformer where tokens are feature channels, not time steps. For financial data, this captures cross-channel dependencies (price-volume-volatility).

Config: d_model=32, n_layers=4, n_heads=2, d_ff=64 (52k parameters).

Full graph:

• Z-score normalization over context window (B, ctx, F) without look-ahead.
• iTransformerEncoder.
• Concat extras: [prev_pos, prev_delta, t/H, (H-t)/H].
• PolicyHead: direction_head × gate_head.

Position: $$p_t = \tanh\left(\frac{d_t}{\tau_{dir}}\right) \times \sigma\left(\frac{g_t}{\tau_{gate}}\right)$$

Gate signal masks uncertain directions (like action masking). gate_bias=-1.0 initialization stabilizes near-flat starts.

Rolling rollout:

for t in range(H):
    window = full_seq[:, t : t + ctx, :]
    window_norm = normalize_context(window)
    extras = [prev_pos, prev_delta, t/H, (H-t)/H]
    pos_t = model(window_norm, extras)
    positions_list.append(pos_t)
positions = cat(positions_list)
step_pnl = simulator.simulate(closes, positions)
loss = hybrid_loss(step_pnl, positions)
loss.backward()

Hybrid Loss to Prevent Pathologies

Pure Sharpe leads to churning, flat collapse, long bias, terminal exposure, drawdown blindness. Hybrid fixes them:

$$\mathcal{L} = \lambda_1 \cdot (-Sharpe) + \lambda_2 \cdot turnover + \lambda_3 \cdot drawdown_{log} + \lambda_4 \cdot |p_H| + \lambda_5 \cdot (\hat{f} - f^*)^2 + \lambda_6 \cdot |\bar{p}|$$

• $\lambda_2$: Turnover penalty against churning.
• $\lambda_3$: Log-drawdown for stability.
• $\lambda_4$: Zero exposure at horizon end.
• $\lambda_5$: Target flat fraction via sigmoid.
• $\lambda_6$: Anti-bias, crucial on bull markets.

$$drawdown_{log} = mean(cummax(cumsum(log(1+pnl))) - cumsum(log(1+pnl)))$$

Mirror Augmentation for Symmetry

Training data (2024–2025) is bull BTC. Without intervention, model goes long-only. Mirror augmentation flips prices and features on batch subsets, creating symmetric pairs.

Result: short_fraction 17.3% (test), 20.9% (backtest).

Experiments: Data and Splits

| Parameter | Value |

|-----------|-------|

| Instrument | BTCUSDT Binance Futures |

| Resolution | 30-min bars (aggregated from 1-min) |

| Period | 2021–2025 |

Splits:

• Train: Jan 2024 – Mar 2025.
• Test: Jul–Sep 2025.
• Backtest: Oct–Dec 2025.
• Walk-forward OOS.

Key Takeaways

• Single differentiable graph from features to Sharpe solves proxy issues, including costs and position sizing.
• iTransformer + direction×gate enables cross-channel attention and confident trading.
• Hybrid loss with anti-bias prevents pathologies like churning and long-only.
• Mirror augmentation yields symmetric policy on asymmetric data.
• Walk-forward Sharpe +1.73 after commissions on BTC OOS.

— Editorial Team